* feat(webui): add space key quick preview for files and folders
Allow users in file list views to press the Space key to quick preview the most recently selected item.
- Add FilePreviewDialog supporting both folder previews (icon, name, count) and real file previews (reusing FileViewer with viewer area and control bar).
- Constrain dialog to max 1920x1080 or 70vw/70vh.
- Support dismissing via close button, clicking outside, or pressing Space again (intercepted in capture phase).
- Enable across project file lists, share links, and public share views.
* feat(webui): auto play video in quick preview dialog
- Pass autoPlay prop from FilePreviewDialog through FileViewer to VideoViewer
- Configure videojs with autoplay: 'any' and handle policy rejection with muted autoplay fallback
- Add test coverage for autoPlay prop forwarding in FilePreviewDialog
* fix(webui): resolve video quick preview initialization and autoplay pause bugs
* fix(webui): support folder symlinks and password-protected shares in quick preview
* feat(core): export comments to FCP, Premiere, Media Composer, and Resolve
Add backend support to export video comments matching Frame.io export capabilities:
- DaVinci Resolve (.edl): CMX 3600 EDL with ResolveColorPurple markers and reply threading
- Avid Media Composer (.xml): OMFI StreamItems XML with TC1 / V1 track allocation and sequential frame positioning
- Adobe Premiere Pro (.xml): FCP7 XML sequence with dual marker placement on sequence and generatoritem
- Final Cut Pro 10.4.0+ (.fiojson): Flattened JSON schema with frame, timestamp, and annotation serialization
Includes DTO schema validation, AssetService export method, API route (GET /files/:fileId/comments/export), SMPTE drop-frame timecode calculations, unit tests, golden master tests, service integration tests, and API endpoint tests.
* test(core): move comments export test fixtures to test directory
* fix(core): escape asset name in Premiere XML, align Resolve FCM, and support CJK
- Escape XML special characters in <name> tag for Premiere Pro XML sequence
- Harmonize Resolve EDL FCM header and emitted timecodes using getEffectiveDropFrame
- Use RFC 5987 / RFC 6266 buildContentDisposition for CJK filenames in download headers
- Add test coverage for CJK filenames, comments, and author names across all 4 NLE formats
* fix(core): implement timezone-aware comment exports and sanitize resolve edl
* feat(sandbox): abstract SandboxService and hot-reload allowed domains
Introduce SandboxService with pluggable SandboxProvider in @shumai/core. Eliminate disruptive per-session sandbox resets by hot-reloading allowed domains via updateConfig, isolate violations per commandId, and eagerly initialize sandbox at worker/server boot.
* fix(sandbox): isolate command violations and serialize concurrent domain sync
Strictly isolate getBlockedHost per commandId to prevent false-positive blocked reporting on concurrent allowed commands. Serialize syncAllowedDomains to ensure concurrent requests during initialization or updates are never silently dropped. Add comprehensive unit tests for LocalSandboxProvider and concurrent bash commands.
* fix(agent): do not synchronize sandbox when allowedDomains is omitted
Make allowedDomains optional in CreateAgentSessionParams and only invoke sandboxService.syncAllowedDomains when allowedDomains is defined. Omit allowedDomains in generateSessionNameActivity so background chat title generation never overwrites the live sandbox allowlist.
* feat(transcode): optimize video operations using ffmpeg http range requests
* fix(transcode): preserve non-retryable errors and ensure mapConcurrent settles on rejection
* fix(s3): stream downloadToFile and putObject to prevent OOM on large files
* fix(agent): preserve file MIME detection on download_asset and align local download directory creation
- Upgrade @inlang/paraglide-js to 2.25.0 in root devDependencies
- Untrack 2,700+ generated paraglide files from git index
- Ignore packages/webui/paraglide in .gitignore and .dockerignore
- Remove --no-emit-git-ignore flag from i18n:compile and i18n:watch
- Chain bun run i18n:compile into build and build-npm scripts
- Add i18n compile step in Dockerfile builder stage
- Add i18n compile step in CI workflows (.github/workflows/ci.yaml, release-npm.yaml)
- Update README.md and README_CN.md with local setup instructions for i18n compilation
* chore(deps): upgrade bun to 1.4.2 and update packages
- Upgrade Bun to 1.4.2 in Dockerfile, CI workflows, and release workflows.
- Update dependencies across workspace with Bun 1.4.2.
- Preserve @earendil-works/pi-agent-core at 0.82.1.
- Retain stable compatible versions for database adapters and pdfjs-dist.
- Add explicit Buffer type annotation in transcode service.
* chore(deps): upgrade prisma to 7.10.0, pg to 8.23.0, and pdfjs-dist to 6.3.289
* chore(deps): revert dotenv-expand to 13.0.0
* feat(settings): redesign image and video generation settings with tabs, card switch, and 2-box model selector
* fix(settings): remove model type checkmark, remove search inputs, and fix select box scrolling in add model dialog
* refactor(settings): remove display name input from add model dialog
* feat(media-generation): add gemini-omni-1.1-flash as default video model via google interactions api
* feat(media-generation): forward video parameters to gemini-omni-1.1-flash via system instructions and prompt
* feat(agent): add built-in image and video generation tools and settings
* fix(settings): refine media generation layout and expose media tools in agent and quota forms
* feat(media-generation): add alibaba, bytedance, minimax, prodia, and azure providers
* feat(agent): support s3 key references for media generation and s3KeyOnly in read_asset
* feat(agent): store preset avatars as real files in s3 storage
* fix(agent): use inline base64 previews in dialog and remove static avatar files
* feat(agent): add 4 avatar presets and track avatarPreset in agent config
* refactor(agent): consolidate preset avatar base64 data in dtos
* feat(asset): record and display agent provenance on assets and audit logs
* style(webui): apply 30% translucent frosted glass background to bot badges
* style(webui): match bot icon badge style to thumbnail media overlay (bg-black/60 text-white)
* style(webui): use "AI" text badge with pdf overlay style instead of bot icon
* feat(provider): vendor and adapt pi model generator
- Vendor generate-models.ts, model-data.ts, and models-dev-reasoning-options.ts from @earendil-works/pi tag v0.82.1
- Adapt imports to use @earendil-works/pi-ai and expose generateModels programmatic function
- Add compare.test.ts to verify vendored generator against original built-in catalog
- Add packages/core/src/provider/generator to eslint ignores
* feat(provider): add provider and model sync api and ui
* feat(webui): add checkbox for existing providers and clean dialog description
* feat(webui): add expand all and collapse all to sync providers dialog
* refactor(provider): remove compare.test.ts and cli overhead from generate-models
* fix(provider): add 30s timeout and strict error handling to sync catalog fetching
* fix(e2e): eliminate drag-and-drop race condition on webkit
Allow requestAnimationFrame collision updates to settle in dragCard helper before releasing pointer, and scope fileCard locator via data-testid to avoid matching sidebar tree items.
* feat(provider): optimize provider and model configuration management
* style(provider): use default tabs variant in provider settings dialog
* style(provider): make tabs list full width
* feat(provider): make model card row clickable and remove pencil edit icon
* feat(settings): persist active settings tab in url hash
* feat(agent): add search box and alphabetical sorting for provider and model selectors
* refactor(providers): remove frontend pagination and display all models in scrollable list
* fix(providers): address code review findings for model editing, i18n, and api protocol propagation
* feat(agent): consolidate media tools into read_asset and add S3 key support to download_asset
* fix(agent): refine parameter descriptions and remove download URL from read_asset text content
* fix(agent): resolve owning asset and enforce RBAC permissions when downloading by S3 key
* feat(agent): improve timestamp precision and proxy selection for video screenshots
* feat(agent): support dynamic annotationId resolution across chat and comments
* fix(agent): scope annotation resolution to asset and format has_markup consistently
* fix(webui): strip styles and preserve multiline text on paste in chat input
* fix(db): remove custom log wait strategy in test global setup to prevent postgres startup race condition
* feat(chat): support markups, attachments, and timestamps in agent 1-to-1 chat
* feat(chat): improve message card layout with inline badges and integrated assets
* fix(chat): enable image attachment thumbnails in 1-to-1 chat cards
* fix(chat): address code review feedback on input state, mentions, i18n, and session deletion
* fix(chat): replace ring with border color on selected message card
* fix(asset,webui): disallow dropping version stacks onto files and version stacks
* fix(webui): derive isSingleFile from total drag selection and remove unused hasVersionStacks
* feat(webui): add mobile friendly file detail and public share view
* fix(webui): address review feedback for mobile file detail and public share view
* fix(webui): use dynamic viewport and flex column layout for mobile file viewer
* fix(webui): support iOS WebKit video fullscreen and guard requestFullscreen
* fix(webui): compact page indicator and layout in mobile PDF control bar
The project has no security policy, so a researcher with a finding today has
only a public issue or Discord, both of which disclose before a fix exists.
Routes reports to GitHub private vulnerability reporting. Names deployment
manifests as in scope, since the documented quickstart is what most operators
actually run. Adds operator guidance for the three things a self-hoster owns:
the session secret, not publishing the database port, and treating the agent
as a code-execution surface.
Response timelines are left for the maintainer to set.
* feat(webui): add simplified mobile-friendly layout for file and share lists
* refactor(webui): switch mobile file list layout to 1-column grid
* feat(webui): show foldertree by default in child sidebar and add close button
* feat(webui): add collapsible section headers and fix folder card full-width on mobile
* feat(webui): improve mobile breadcrumb bar with middle truncation and dual ellipsis
* fix(webui): unify breadcrumb DOM structure across desktop and mobile
* fix(webui): remove unused breadcrumbs variable in BreadcrumbNav
* fix(webui): complete and confirm file uploads on mobile and handle errors properly
* fix(webui): remove active scale animation on mobile FAB to stabilize dropdown menu anchor
* refactor(webui): use PresignedUrl type from dtos for uploadFiles and eliminate any cast
* fix(webui): scope folder card max width to desktop with responsive md:max-w-[340px]
* feat(transcode): add hardware-accelerated video transcoding support
* feat(transcode): use encoder quality modes and source-capped maxrate for video transcode
* fix(transcode): scale max bitrate for downsampled preview framerates
* fix(transcode): propagate videoBitRate in getMediaInfoActivity and update auto description
* feat(project): add recent viewed files section to project folder tree
* feat(webui): disable toolbar buttons and context menu in recents page
* test(core): optimize recents cap test execution time
* feat(webui): show hint text in recents page top toolbar
* feat(webui): rename recents section to Recently Viewed
* refactor(core): consolidate recents methods into AssetService
* fix(api): forward pagination parameters in folder children endpoint (#349)
* fix(project-recents): scope views and disable recents mutations
* fix(api): allow recent views in demo mode
* fix(api): limit demo recent view exception to post
* feat(auth): add admin password reset CLI command and reset password web UI
* fix(auth): invalidate previous reset tokens and use AsyncLocalStorage for link capture
* feat(kanban): allow editors to change task status and default reporter to creator
* fix(kanban): hide create task button for reviewers and eliminate duplicate error toast
* fix(webui): update sidebar selection styling, kanban active state, and pointer cursors
* feat(webui): move dashboard to left sidebar and prevent mount animation
* fix(e2e): update create-project test to click Home button in sidebar
* feat(kanban): implement AI-native kanban database models, services, and REST APIs
* feat(webui): implement team kanban board and task management
* style(webui): increase kanban dialog widths and form container
* fix(webui): ensure kanban dialogs override sm:max-w-lg base class
* style(webui): set edit task dialog max width to 5xl
* chore: untrack kanban plan markdown files
* feat(webui): update kanban page layout to 2 columns and refine task activity UI
* fix(webui): constrain kanban column and board heights for vertical scrolling
* feat(kanban): unify task status transition into updateTask API and allow unconstrained human task transitions
* fix(webui): add optimistic updates to kanban drag-and-drop to eliminate flash
* refactor(webui): remove client-side search box from kanban header
* feat(kanban): redesign task dialog comments area and add attachment support
* refactor(webui): remove top status action bar from task info dialog
* fix(webui): narrow query cache predicate and add defensive guards for kanban drag optimistic update
* feat(kanban): add drag-and-drop task reordering with fractional indexing (#331)
* feat(kanban): add drag-and-drop task reordering with fractional indexing
* fix(kanban): improve card drop zones, snap indicators, and column highlight
* fix(kanban): enable card droppables only during active drag
* fix(kanban): refine hover styles and drop indicators for column boundaries
* fix(kanban): unify bottom drop indicator and fix top indicator clipping
* fix(kanban): ensure full column highlight and full coverage drop target
* minor change
* feat(kanban): allow agent tasks free drag-and-drop and enforce status change permissions (#332)
* refactor(kanban): rename task_type_agentic translation keys to task_type_agent
* feat(kanban): refactor task type enum to isAgentTask boolean (#333)
* refactor(kanban): remove phase 2 agent execution logic and streamline phase 1 kanban
* feat(kanban): add notifications and audit logging for kanban (#334)
* feat(kanban): redesign config area layout and add parent tasks multi-selector (#335)
* feat(webui): redesign kanban task dialog config area
* feat(kanban): simplify task dependencies with parent tasks multi-selector
* refactor(kanban): remove unused links routes and legacy dependency methods
* feat(kanban): connect assets and kanban tasks with many-to-many relationship (#336)
* feat(kanban): connect assets and kanban tasks with many-to-many relationship
* feat(kanban): improve asset picker layout, infinite scrolling, and breadcrumb icon styling
* fix(kanban): ensure asset picker scrollarea is scrollable with min-h-0 and shrink-0 flex styling
* fix(kanban): hide recently deleted in folder tree picker and adjust header padding
* fix(kanban): use lucide files icon instead of paperclip for linked assets
* feat(kanban): improve task deletion, goal sidebar interaction, and card styling (#337)
* feat(kanban): improve task deletion, goal sidebar interaction, and card styling
* feat(kanban): move agent task badge after priority and update to orange style
* feat(kanban): add status Compare-And-Swap (CAS) to prevent drag-and-drop race conditions (#338)
* feat(kanban): add status Compare-And-Swap (CAS) to prevent drag-and-drop race conditions
* style(kanban): rotate kanban icon 180 degrees in sidebar and top nav
* style(kanban): remove rotate-180 from SquareKanban icons
* feat(kanban): add unassigned goal record and fix all tasks count (#339)
* feat(kanban): add unassigned goal record and fix all tasks count
* feat(kanban): toggle task count with three-dot action button on goal card hover
* fix(ci): pin bun version to 1.3.14 across CI and Docker (#340)
* fix(webui): truncate goal name to max 2 lines in kanban sidebar (#341)
* fix(quota): exclude agents from quota rules and usage records
* refactor(quota): remove agent_skill_call_count resource type and related logic
* chore(db): replace manual migration with prisma-generated migration
* feat(quota): add database-backed resource quota system
* feat(webui): add resource quota management in team settings (#319)
* feat(webui): add resource quota settings in team management
* refactor(webui): update quotas list to full row layout and restrict skill/mcp to select list
* feat(quota): refactor quota system to 2-level rules and records
* fix(webui): fix overflow and scroll area in quota usage records dialog
* fix(webui): ensure quota records scroll area has explicit max height
* feat(webui): improve quota cards interaction and restrict editable fields in edit mode
* fix(quota): serialize initial records, validate merged updates, and reset on dimension changes (#320)
* feat(quota): replace network quota with agent tools quota (#321)
* feat(webui): show remaining time until next refresh in quota usage records
* fix(webui): rename bash quota resource to Agent Bash Calls
* refactor(quota): remove unused reserved field from quota records
* refactor(quota): remove quota rule cache (#322)
* feat(authz): enforce project-aware roles for agent/skill/mcp permissions
Introduce a shared resolveEffectiveRole helper that mirrors AuthzService
precedence rules (projectMember.role override -> team role for team-scoped
members -> restricted), and route all agent/skill/mcp runtime permission
checks through it. Previously these checks used the raw teamMember.role,
which let project-only members execute team-level agents/skills/MCP tools
and ignored per-project role overrides.
Changes:
- authz: add resolveEffectiveRole(teamId, projectId, userId, db) and
refactor hasPermission to use it (behavior-preserving)
- chat/asset/agent-activity: gate agent usage by the effective project role
- read_skill: enforce skill permission via effective role (projectId
threaded from the agent session)
- mcp-service: checkPermission and buildAgentTools filter by effective
role; un-permitted servers' tools are no longer advertised to the model
- agent activity: only advertise skills the user's effective role permits
- listProjectMembers: filter included agent bots by the requester's
effective project role
- new GET /projects/:projectId/chat-agents API returning chat agents
scoped to the project and the user's effective role; WebUI chat selector
now uses it (settings keeps the team-level API)
- tests: effective-role resolver, chat/mention/MCP/skill project-role
matrices, listProjectChatAgents, listProjectMembers filter, API route
* fix(agent): forward projectId to getAgentChatContextActivity in agentChat workflow
The agentChat workflow called getAgentChatContextActivity without passing
projectId from the task payload, so project-scoped members (scope:
'project') resolved a null effective role in the activity and were denied
with 'Insufficient role to use agent' even though the chat selector and
chat.ts both authorized them. Forward payload.projectId so the activity
enforces the user's effective project role, and assert the forwarding in
the workflow unit test (fails without the fix).
* feat(agent): add permission controls for chat agents
* refactor(agent): extract shared agent permission helpers
Consolidate the duplicated role-hierarchy logic into
packages/core/src/agent/permissions.ts:
- getRoleLevel(): numeric level for a team role (fail closed at 0)
- getAgentRequiredLevel(): required level for an agent permission
(defaults to reviewer/1)
- getAllowedAgentRoles(): inclusive role set for filtering
The typed Record over TeamMemberRole makes the mapping exhaustive,
so adding a new role fails compilation until all sites are updated.
Used by ChatService.startOrContinueChat, AssetService comment
mentions, getAgentChatContextActivity, AgentService.listAgents, and
TeamService.getTeamMembers, replacing four inline copies of the
hierarchy and the allowed-roles conditionals.
Also fix the API permission-denial test to mock HTTPException(403)
and assert a 403 response instead of a generic Error/500, matching
real authzService behavior.
* test(agent): fix flaky comment DAG tests by spacing ULID id generation
ULID ids have 1ms timestamp granularity; back-to-back comment creates
frequently land in the same millisecond, in which case Prisma ULIDs sort
by their random suffix instead of creation order. The DAG spine in
initializeAgentSessionActivity chains top-level comments by id, so a
shuffled order produces paths that omit preceding comments, making the
path-to-root assertions flaky (~80% of full-file runs locally).
Add a 2ms sleep between consecutive comment creates in the affected
tests, guaranteeing strictly increasing ULID timestamps. A 2ms delay is
sufficient because the next create always fires at least 2ms after the
previous ULID was generated. Production is unaffected: real comments
arrive seconds apart via separate requests.
* feat(version-stack): add manage versions dialog and version removal
* fix(webui): remove ellipsis and add pointer cursor to manage versions button
* fix(version-stack): handle symlink deduplication and name sync for share links
* refactor(version-stack): extract shared symlink dedup helper
The 'repoint first symlink per share folder, delete duplicates with
fileCount decrement' logic was triplicated across createVersionStack,
reparentAssets, and reparentFileToFile with subtle variations. Extract
it into dedupeSymlinksToTarget() in packages/core/src/asset/symlink.ts
and call it from all three sites (behavior unchanged), with a dedicated
test suite covering the edge cases.
* feat(agent): add nested agents.md support across backend and webui
* fix(webui): update agentsmd route paths and unnest folder agentsmd route
* fix(agent): flush agents.md autosave on close and address review feedback
- Flush pending debounced AGENTS.md autosave on close/unmount so edits
made within 750ms of navigating away are no longer silently lost
- Serialize autosaves through a promise chain so an older in-flight
request can never overwrite newer content on the server
- Internationalize the hardcoded "Failed to save" and
"Loading AGENTS.md..." strings (en + zh)
- Validate that AGENTS.md is only stored on folder/root assets in
AssetService, preventing bogus virtual paths for file assets
- Cap updateAgentsMd content at 100k chars via Zod validation
- Remove unused getAgentsMd/updateAgentsMd response DTO schemas
- Drop unused teamId prop from AgentsMdEditor
- Add tests: non-folder asset rejection, oversized content 400,
and agent-chat workflow assetId passthrough
* refactor(db): isolate AGENTS.md content in dedicated 1:1 AssetAgentMd table
* style(webui): unify top bar button hover styles and add wand icon to agentsmd button
* style(webui): update agentsmd button state colors and remove presence dot
* fix(webui): fix close button overflow and stabilize debounced autosave in agentsmd editor
* fix(webui): update debounce to 1s and await pending save before closing agentsmd editor
* feat(webui): add terminal AGENTS.md breadcrumb on agentsmd editor page
* fix(webui): use ScrollArea for markdown editor scrolling
* feat(webui): replace legacy markdown editor with new lexical wysiwyg editor
* refactor(webui): organize markdown editor files into components/markdown-editor folder
* feat(webui): redesign select and multi-select metadata fields with search box and dynamic option creation
* fix(webui): stabilize select popover placement and preserve field visibility in store
* refactor(webui): enforce 2-row collapsed height and whitespace-pre-wrap on LongTextField
* feat(agent): support dynamic newOption in select and selectMulti metadata fields
* fix(metadata): address code review feedback on selectMulti item validation and WebUI option deduplication
* feat(webui): redesign select and multi-select metadata fields with search box and dynamic option creation
* fix(webui): stabilize select popover placement and preserve field visibility in store
* refactor(webui): enforce 2-row collapsed height and whitespace-pre-wrap on LongTextField
* feat(agent): type-safe CREATION_CONTEXT metadata on create_file/create_version
Build the create_file and create_version tool metadata parameter from the
project's CREATION_CONTEXT fields at harness creation time, instead of the
schemaless key-value map that required the agent to call list_autofill_fields
first.
- Thread projectId from the chat workflow into createAgentSession and build
a strict, all-required (nullable) metadata schema via fieldsToTypeBoxSchema
(additionalProperties: false, option IDs enforced via enum).
- Remove the list_autofill_fields tool and its activity case.
- Reject metadata keys that are not CREATION_CONTEXT fields of the target
project with a clear non-retryable error telling the agent to omit metadata.
- Skip null metadata values when persisting (create_file/create_version and
the autofill workflow).
- Switch the touched schemas from @sinclair/typebox 0.34 to typebox v1 to
match the harness validator.
* refactor(agent): migrate remaining tools from @sinclair/typebox 0.34 to typebox v1
All agent tool schemas now use the same typebox v1 package as the harness
validator; drop the @sinclair/typebox 0.34 dependency from @shumai/agent.
* test(agent): reproduce null coercion in harness tool validation
The pi-ai harness (<=0.82.1) coerces null values in [string, null] unions to
'' during tool argument validation for typebox v1 schemas. A valid create_file
call with path: null + data: {...} therefore arrives at execute with path: ''
and fails with 'Provide exactly one of path or data'. Null metadata values
(the 'unknown' sentinel) are likewise turned into ''.
This test reproduces both failures. It is expected to fail until
@earendil-works/pi-ai is upgraded to >=0.84.0 (#7328), which preserves values
that already match a union arm.
* fix(agent): patch pi-ai 0.82.1 to preserve null in nullable union tool args
Apply the upstream fix for earendil-works/pi#7328 (PR #7373) via a bun
patch instead of upgrading pi-ai. coerceWithUnionSchema now returns the
original value when it already validates against any union member,
instead of coercing null in [string, null] unions into "".
Fixes the create_file false "Provide exactly one of path or data" error
when the model passes path: null with data. Reproduced by
packages/agent/src/reproduce-tool-null-coercion.test.ts, which now passes.
* fix(docker): copy bun patch dir before frozen-lockfile install
The pi-ai patchedDependencies entry makes bun install --frozen-lockfile
fail in the builder stage because patches/ was not present in the image
yet. Copy it alongside package.json/bun.lock so the patch applies.
* refactor(agent): simplify metadata guidance in system prompt and error message
Shorten the create_file/create_version metadata instructions in the system
prompt and reword the invalid-metadata-key error to not reference
CREATION_CONTEXT internals. Update the matching test assertion.
* feat(metadata): replace aiAutofill boolean with autofillSource enum
* fix(agent): validate creation-context metadata and make writes transactional
Address code review feedback on the autofillSource migration:
- Update the agent system prompt to reference the 'metadata' parameter of
create_file/create_version instead of the removed 'context' parameter,
which was being silently discarded by the activity
- Write metadata inside the caller's transaction via the new
metadataService.updateAssetMetadataInTx, keeping metadata rows atomic
with asset creation instead of opening a separate non-atomic transaction
- Reject metadata keys that are not defined CREATION_CONTEXT fields of the
project with a non-retryable ApplicationFailure before creating the asset
- Remove dead agentContext handling from autofillAiActivity and its tests,
and drop the unused ai_autofill i18n keys (paraglide recompiled)
- Derive the DTO autofillSource enum from the AutofillSource const instead
of a hardcoded z.enum to prevent drift
* docs(mcp): add MCP support documentation
* docs(readme): simplify MCP feature description
* docs(mcp): fix inaccuracies in MCP servers documentation
- Clarify that discovery runs on refresh, OAuth completion, or first
tool call, not automatically at server creation
- Note that advanced settings (transport, auth, permission, direct
tools) are configured in the edit dialog after creation
- Correct agent assignment UI description: switch toggle, not checkbox
* docs(mcp): explain proxy vs direct tool configuration
Add a dedicated section covering the three tool states (Off / Proxy /
Direct), what the mcp proxy tool does and why it exists, and when and
why to use direct tools, including the context-token trade-off.
* feat(webui): render pdf pages via live canvas with selectable text
Replace the rasterize-to-PNG pipeline in the PDF viewer with direct
canvas rendering that tracks the current zoom, so text stays crisp when
zoomed in, and overlay a pdf.js TextLayer that makes text natively
selectable when not in draw mode.
- pdf-viewer: render pages onto a live canvas at a zoom-proportional
scale (debounced, memory-capped) instead of a fixed 2x PNG snapshot
- pdf-viewer: add pdf.js TextLayer overlay per page, transformed in
lockstep with the Konva stage so spans stay aligned while panning
and zooming
- drawing-canvas: accept an optional live canvas element and force a
layer redraw when it is swapped
- add text-layer.css with the minimal pdf.js text layer styles
- add e2e spec verifying text selection in view mode and that draw
mode routes drags to the canvas
* fix(webui): show pdf text selection highlight above the page canvas
The pdf.js text layer was stacked below the opaque Konva page canvas, so
the native selection highlight was painted underneath it and invisible
(selection itself worked). Move the overlay above the canvas and gate its
pointer-events on draw mode: auto when not drawing (select text), none in
draw mode so drags reach the Konva stage below (draw markup).
Verified with screenshot pixel analysis: the selected region now renders
the highlight (blue) where it was pure grayscale before.
* fix(webui): transparent selection background for pdf text layer br elements
* fix(webui): stop pdf text selection over-selecting in empty space
The text layer container was user-selectable (user-select: auto), so when
a drag's caret moved into empty space Chromium targeted the container and
extended the selection to a container-content boundary, selecting across
rows (e.g. dragging from line 1 into the gap selected lines 1-2).
Make the container user-select: none while keeping the text spans
user-select: text, so the caret clamps to the nearest selectable text and
the selection stays put when dragging into empty space.
* fix(webui): deselect pdf text when clicking empty space
With user-select: none on the text layer container, Chromium no longer
clears the selection when clicking non-selectable areas. Clear it manually
on mousedown over the container or <br> (not a text span) so clicking
empty space between lines or outside text deselects, matching native
behavior.
* fix(webui): add content-only pinch-zoom and two-finger pan to media viewers
Intercept trackpad pinch (ctrl+wheel), touchscreen pinch (two touches),
and two-finger move inside the image, video, and PDF viewers so they
transform only the media content instead of triggering browser page
zoom (matching Frame.io behavior).
- New shared pan-zoom math (zoomAtPoint, fitScale, isZoomed) and a
usePanZoomGestures hook using native non-passive wheel/touch listeners
with touch-action: none on the media container.
- Plain wheel / two-finger scroll pans the content, but only once the
user has zoomed in; drag-pan is gated the same way.
- Image and PDF viewers no longer reset zoom to fit on container resizes
while the user has manually zoomed (matches the video viewer).
- DrawingCanvas ignores multi-touch moves so single-pointer pan/draw
doesn't fight the pinch handler; the video viewer keeps play/pause on
the container div to avoid double-toggling when the overlay is active.
Includes unit tests for the pan-zoom math.
* fix(webui): disable click-drag pan for pdf and video viewers
When zoomed, PDF and video previously panned on left-button (or
single-finger touch) drag via the shared DrawingCanvas. That conflicts
with left-click semantics: PDF wants the drag for text selection and
video reserves the click for play/pause.
Remove the gated onPan from both viewers so the Konva overlay stays
pointer-events:none when not drawing - clicks pass straight through
(video click-to-play now works cleanly at any zoom) - while panning
remains available via mouse wheel and two-finger gestures. Image keeps
drag-pan.
* feat(mcp): support tool-level directTools array and 3-state UI configuration
* refactor(mcp): refine tool hint explanation and separate server instructions block
* style(mcp): display tool count in header title and split option hints into 3 lines
* style(mcp): update tools description paragraph and place bulk buttons on a separate row
* fix(mcp): enforce excludeTools in proxy path and align direct-tool counts
- Filter the per-agent tool registry by excludeTools at build time (DB cache
and lazy-connect paths), so excluded tools are unreachable through the mcp
proxy (call, search, list, describe) — matching pi-mcp-adapter where
isToolAllowed gates both direct and gateway exposure.
- Add shared isDirectTool predicate (matches original or prefixed name) used
by both buildDirectTools and the proxy description counts, so the
advertised direct/proxy counts can no longer drift from actual
registration; toolCount now reflects allowed tools only.
- Remove the legacy boolean directTools branch (no legacy data exists).
- Drop the now-unused mcp_excluded_tools_hint i18n key and regenerate
Paraglide messages.
- Extend unit tests: excluded tools are not counted, listed, searched, or
callable via the proxy; prefixed-name directTools configs count correctly.
* fix(agent): resolve version stack name and size in list_assets
* fix(agent): resolve version stack to latest version in download_asset, analyze_image, and read_pdf_pages
* feat(mcp): add MCP server support with proxy tool and OAuth flow
Add URL-based MCP (Model Context Protocol) server support to the backend:
- DB: McpServer / AgentMcpServer / McpServerCredential models with
typed JSON columns (auth config, cached tools, stored OAuth tokens,
pending auth state) and a migration.
- Core (@shumai/core/src/mcp): DB-backed credential store, OAuth
client provider (authorization_code + client_credentials) with a
Hono-hosted redirect callback, connection lifecycle manager
(Streamable HTTP with SSE fallback, lazy connect, 401 -> needs_auth),
JSON Schema -> AgentTool parameter conversion, in-memory tool registry
with ranked search, the single 'mcp' proxy AgentTool (search /
describe / list / call / connect / status / auth-start), per-server
opt-in direct tools ({server}_{tool}), and McpService (CRUD,
discovery, OAuth, per-agent assignment via AgentMcpServer, and
permission gating on tool execution mirroring skills).
- Agent: chat agents receive the 'mcp' proxy tool (plus direct tools
for opted-in servers) built from their assigned, enabled servers.
- API: /teams/:teamId/mcp/servers CRUD, tools refresh/list, auth
start/complete/status/remove, test, permission, and the public
OAuth callback route; Admin authz via new ResourceType.McpServer;
audit logging.
- DTOs: zod schemas + types for servers, tools, auth, and per-agent
mcpServerIds.
- Tests: DB store, schema conversion, registry/search, service
integration against a real in-process MCP server (v2
@modelcontextprotocol/server), permission gating, rename
invalidation, direct tools, per-agent assignment, API route tests,
and workflow E2E (local + temporal) for proxy and direct modes.
* fix(mcp): persist OAuth discovery snapshot and stamp SEP-2352 issuer
The callback leg of the OAuth flow failed with
AuthorizationServerMismatchError ('discoveryState was not available on
the callback leg') because McpDbStore.savePendingAuth overwrote the
entire pendingAuth JSON column, clobbering the discoverySnapshot that
the provider's saveDiscoveryState() had stored inside it.
- McpDbStore.savePendingAuth now preserves an in-flight discovery
snapshot so the callback leg can satisfy the SEP-2352 authorization-
server issuer check.
- McpOauthProvider now round-trips the issuer stamp derived from the
discovery snapshot: saveClientInformation/saveTokens persist it,
clientInformation()/tokens() return it, and tokens() backfills a
missing stamp for pre-existing credentials (removes the
'stored OAuth credential has no issuer stamp' warning).
- Regression tests: savePendingAuth keeps the snapshot; the provider
stamps/round-trips the issuer on both legs.
* fix(mcp): backfill SEP-2352 issuer stamp on stale OAuth client info
clientInformation() returned stored client info without an issuer field
when it had been dynamically registered before the stamping fix, so the
SDK logged 'stored OAuth credential has no issuer stamp' on every read.
Now the stored branch backfills the issuer from the discovery snapshot
(persisting it) exactly like the tokens() backfill.
* feat(webui): add MCP server configuration and agent assignment UI (#290)
* feat(webui): add MCP server configuration and agent assignment UI
* fix(webui): auto-detect auth for unconfigured MCP servers and show connect button
* fix(mcp): auto-generate state in mcp-oauth-provider when missing in DB
* fix(mcp): persist OAuth discovery snapshot and stamp SEP-2352 issuer
The callback leg of the OAuth flow failed with
AuthorizationServerMismatchError ('discoveryState was not available on
the callback leg') because McpDbStore.savePendingAuth overwrote the
entire pendingAuth JSON column, clobbering the discoverySnapshot that
the provider's saveDiscoveryState() had stored inside it.
- McpDbStore.savePendingAuth now preserves an in-flight discovery
snapshot so the callback leg can satisfy the SEP-2352 authorization-
server issuer check.
- McpOauthProvider now round-trips the issuer stamp derived from the
discovery snapshot: saveClientInformation/saveTokens persist it,
clientInformation()/tokens() return it, and tokens() backfills a
missing stamp for pre-existing credentials (removes the
'stored OAuth credential has no issuer stamp' warning).
- Regression tests: savePendingAuth keeps the snapshot; the provider
stamps/round-trips the issuer on both legs.
* fix(mcp): backfill SEP-2352 issuer stamp on stale OAuth client info
clientInformation() returned stored client info without an issuer field
when it had been dynamically registered before the stamping fix, so the
SDK logged 'stored OAuth credential has no issuer stamp' on every read.
Now the stored branch backfills the issuer from the discovery snapshot
(persisting it) exactly like the tokens() backfill.
* fix(mcp): attach McpOauthProvider for auto-detected OAuth servers and discover tools on auth completion
* fix(webui): fix flex min-h-0 and max-h scrolling on MCP tools dialog
* fix(webui): set explicit flex height on McpToolsDialog for active scrolling
* feat(webui): make MCP server cards clickable to edit and widen server config dialog
* feat(mcp): agent-level server enablement and auto-detected server metadata (#291)
* feat(mcp): agent-level server enablement and auto-detected server metadata
- Remove the server-level `enabled` column from mcp_servers; enable/disable
now happens per-agent via the AgentMcpServer join table (mirrors skills).
- Drop the unique (teamId, name) constraint so duplicate server names are allowed.
- Add a `description` column, auto-filled from client.getServerVersion() on
connect; the server name is auto-filled from the reported title (fallback name)
with a URL-derived placeholder until first connect.
- Remove manual name input: the create dialog is URL-only, and the edit dialog
shows the auto-detected name/description read-only.
- Replace the include/exclude raw-text inputs with the tool list in the edit
dialog, managing only excludeTools; remove includeTools support and the
separate McpToolsDialog component.
- Sanitize server-reported names into safe direct-tool prefixes (spaces/punctuation
become underscores) and use the same sanitization in the proxy resolver.
* fix(mcp): give the edit-dialog tool list a definite height
The Shadcn ScrollArea viewport is h-full, which requires a definite height
on the root. Using max-h-64 left the viewport at auto height, so the root
clipped the list at 256px without engaging its own scrollbar — tools past
the cutoff were unreachable. Use a fixed h-64 container (same pattern as
move-copy-dialog) so the inner scroll area scrolls independently of the
dialog's outer scroll.
* refactor(mcp): fixed header/footer with scrollable content in server config dialog
Mirror the skill config dialog layout: the edit dialog now keeps the header
and footer fixed and scrolls only the content area (ScrollArea flex-1
min-h-0 inside an h-[85vh] flex column). The tools list no longer has its
own nested ScrollArea — it flows inside the content-level scroll, so all
tools are reachable without a nested-scroll conflict. The create dialog
stays compact (URL-only) like the skill add dialog.
Verified headlessly that max-height-based ScrollArea roots do not engage
scrolling (viewport height resolves to auto), which is why the previous
nested list clipped; the flex-based definite-height pattern scrolls.
* feat(mcp): make the endpoint URL immutable after creation
Editing a server's URL effectively points it at a different server, and the
create flow is URL-only anyway — so the edit dialog no longer allows changing
it (delete + re-add covers endpoint changes). The URL is displayed read-only
next to the auto-detected name/description, and `url` was removed from the
update DTO and service; auth/transport changes still invalidate credentials
and cached tools.
* refactor(mcp): match MCP server cards to the skill card layout
The settings list cards now share the skill card structure: row 1 is the
server name (with direct-mode/transport badges) plus the three-dot menu,
row 2 is the description (falling back to 'No description provided') with
the endpoint URL and last error as small mono lines, and row 3 holds the
connection-status badge and updated date on the left with the permission
selector and the connect button at the right end.
* fix(mcp): stop card menu clicks from opening the edit dialog
The three-dot menu is portaled, so its item clicks bubble through React's
synthetic events to the card's onClick, which opened the edit dialog before
the delete confirmation appeared. Add e.stopPropagation() to every menu
item (matching the skill card pattern).
* fix(mcp): stop Refresh Tools from submitting/closing the config dialog
The Refresh Tools button lived inside the <form> and, like all buttons
without an explicit type, defaulted to type="submit" — clicking it ran the
form submit (save + close) instead of just refreshing. Set type="button".
Also per UX: while refreshing, the button is disabled with a spinning icon,
the per-tool enable/disable switches are disabled, and the success toast is
removed — the button reverting to its idle state is the success signal
(keep the error toast for failures).
* test(mcp): provide explicit redirectUri in McpOauthProvider test helper
* fix(mcp): scope tool registry to agent-assigned servers
The MCP tool registry was a process-global singleton keyed by server id,
so a chat agent could search, describe, and even call tools belonging to
MCP servers that were not assigned to it (e.g. disabled or unassigned
servers), as long as their metadata had been discovered at some point in
the process. This mirrors the disabled-skill bug: the read path never
checked per-agent enablement.
Replace the singleton with a fresh McpToolRegistry built per
buildAgentTools call and warmed from the persisted McpServer.tools DB
cache. Because the registry only ever contains the agent's assigned
servers, unassigned/disabled servers can neither be suggested, searched,
described, nor executed. Live discovery (lazy connect / connect action)
writes into the per-agent registry via ensureConnected(serverId,
registry?).
Adds test-first coverage: suggestions, search, describe and tool-call
leaks across an unassigned same-team server, staleness after
unassignment, and direct-tools scoping.
* feat(mcp): capture and surface server instructions + server-level refresh (#293)
* feat(mcp): capture and surface server instructions
- Add McpServer.instructions column, populated from getInstructions() at
connect time (legacy initialize + modern discover results)
- Surface instructions at three levels like pi-mcp-adapter: truncated
snippet in the mcp proxy tool description, preview in server listings,
and full text via mcp({ instructions: "name" })
- Fix misleading mcp({ server, connect }) usage line to mcp({ connect })
- Share one renderServerListing helper across list and connect modes
- Expose instructions in McpServerInfo DTO and WebUI config card
- Test server supports instructions; service + API tests updated
* refactor(mcp): server-level refresh endpoint, drop agent connect mode
- Replace POST /mcp/servers/:id/tools/refresh with POST /mcp/servers/:id/refresh
returning the refreshed McpServerInfo (server-level, not tools-level)
- Add McpService.refreshServer(): force-close the connection, re-discover and
persist tools, instructions, name/description and status; used by the refresh
endpoint, Test Connection, and post-OAuth completion (replacing discoverTools)
- Remove discoverTools entirely - refreshServer is the single discovery entry
point (testServer and completeAuth route through it)
- Remove refreshTools from UpdateMcpServerRequest/PATCH (dead param)
- Remove mcp({ connect: "name" }) from the agent proxy tool: lazy tool calls
still auto-connect, so agents are unaffected
- WebUI: remove the dialog 'Refresh Tools' button and create-mode background
auto-refresh; add a server-level Refresh button at the left end of the edit
dialog footer; i18n mcp_refresh_server (+ success message)
* feat(mcp): add in-flight request tracking and idle connection cleanup (#294)
* feat(mcp): add in-flight request tracking and idle connection cleanup
* feat(mcp): add background idle cleanup timer to McpService
* feat(mcp): update default background idle cleanup check interval to 30s
* feat(webui): improve MCP server card footer, instructions, and refresh dialog freeze (#295)
* feat(webui): improve MCP server list card, instructions dialog, and refresh freeze UI
* feat(webui): refine MCP card action menu items
* docs(webui): update direct tools mode description in i18n messages
* style(webui): refine Direct Tools Mode layout in MCP server config dialog
* feat(webui): remove tool search box from MCP server config dialog
* fix(mcp): make in-flight idle tracking test deterministic
The 'tracks in-flight requests and safely prunes idle connections' test
relied on a wall-clock sleep (10ms) to guess when the tool handler had
started executing on the server. On slow/loaded CI runners the HTTP round
trip through the real MCP client/server stack can exceed that window, so
resolveSlowTool() fired the no-op placeholder, the gated handler never
completed, and callPromise hung until the 5000ms test timeout.
The handler now signals via a deferred promise when it actually starts,
and the test awaits that signal (racing it against the call promise to
surface early failures) before asserting in-flight state and releasing
the gate. Also gives the test an explicit 10s timeout.
Disabling a skill in the agent config dialog only removed it from the
agent_skills join table, but the chat flow still advertised ALL team
skills to the model in the <available_skills> system prompt section and
the read_skill tool never checked per-agent enablement.
- getAgentChatContextActivity / getAgentAutofillContextActivity now only
return skills explicitly enabled for the agent (agent_skills), and
expose enabledSkillIds in the execution context.
- createReadSkillTool accepts enabledSkillIds and rejects read_skill
calls for skills not enabled for the agent, even if the model knows
the skill id from earlier conversation history.
- Added unit tests for context filtering (chat + autofill), tool-level
enforcement, and the zero-enabled-skills case.
* fix(webui): enable rename and delete actions in file detail top bar breadcrumb
* fix(webui): select only base file name when renaming from file detail breadcrumb
startDownload() used <a target="_blank" download> anchors, which made
Chrome open a new tab for every file that then auto-closed as soon as the
download started (Content-Disposition: attachment). Replace with hidden
iframes: downloads start in the background without opening tabs, and a
failed/expired link renders inside the hidden iframe instead of
navigating the SPA tab away.
* fix(core): use renamed asset name in download Content-Disposition
Downloads of renamed assets (e.g. test.png renamed to foo.png) previously
saved under the storage-key basename because the presigned URLs / local
file handler only set Content-Disposition: attachment without a filename,
and the HTML download attribute is ignored for cross-origin URLs.
- Add buildContentDisposition() helper that emits a sanitized
'attachment; filename=...; filename*=UTF-8...' header (RFC 5987) so
non-ASCII names survive.
- Thread an optional filename through S3Service.presign():
* S3 backend passes it via contentDisposition (response-content-disposition)
* Local backend carries it as a ?filename= query param consumed by the
/files serveStatic onFound handler
- getDownloadLinks / getDownloadUrl now pass the renamed asset name from
the DB, so all download flows (file browser, viewers, breadcrumb, share)
save files under the current asset name.
* fix(core): keep Content-Disposition header ASCII for non-ASCII names
Bun's Headers API rejects header values containing non-ASCII bytes, so a
download of a renamed file with a non-ASCII name (e.g. 幻境边界.pptx) threw
'TypeError: Header Content-Disposition has invalid value' on the local
storage backend.
Per RFC 5987, non-ASCII names belong only in the percent-encoded
filename* parameter. buildContentDisposition now emits the quoted
filename="..." fallback only when the name is pure ASCII; otherwise the
header carries just 'attachment; filename*=UTF-8''...' (all-ASCII, accepted
by Bun/Node Headers and all modern browsers).
* feat(transcode): add Gotenberg support for Office, HTML, CSV, and Markdown PDF proxies
* fix(transcode): use Gotenberg chromium markdown endpoint with index.html wrapper for markdown conversion
* test(transcode): add complete unit test coverage for HTML, CSV landscape/portrait, TXT, and Gotenberg fallback branches
* fix(gotenberg): force .md extension for Gotenberg markdown conversion endpoint compatibility
* refactor(gotenberg): standardize basic auth environment variables on GOTENBERG_BASIC_AUTH_USERNAME and GOTENBERG_BASIC_AUTH_PASSWORD
* test(e2e): add real Gotenberg testcontainers workflow E2E test for docx, pptx, and xlsx
* docs: add Gotenberg document PDF conversion documentation and update features in README
* docs(gotenberg): add Docker quickstart commands and remove obsolete GOTENBERG_USERNAME note
Split the monolithic test:e2e script into:
- test:e2e:app: fullstack web app e2e (playwright --project=app-chromium)
- test:e2e:webui: harness UI integration tests
(E2E_SKIP_APP_SERVER=true --project=harness-chromium)
Remove the now-unused test:e2e:ui and test:e2e:report scripts, and update
the AGENTS.md submission rules and e2e README to reference the new
commands.
Two issues made the watermarked share-link spec flaky on Firefox (and
silently mis-typed on every browser):
1. test.use inside a for loop leaks the last iteration's value to all
tests: both the 'image' and 'video' cases were actually running the
video flow. Scope each test.use inside a test.describe block.
2. Selecting the file by clicking the card body raced with the preview
area re-rendering (video sprite scrubber hover) and the Radix menu
exit animation, which could swallow the click on Firefox. Select via
the card checkbox instead, and wait for the menu to fully unmount
after Escape before the next interaction.
Verified stable: 6/6 on Firefox, 2/2 on Chromium, 2/2 on WebKit.
Document the two new share link security settings:
- Allow Download: hides all download affordances on the public pages and
rejects the download endpoint with 403 while keeping previews available.
- Watermark: per-share-link lazy watermarking — transcodes are only
triggered when the watermark is enabled on a share link (and again for
files added later), so Shumai never stores watermark copies for files
that are not shared through a watermarked link, saving storage.
Add web app e2e coverage for the frequently-combined watermark +
download-disabled share link configuration. For both watermarkable media
types (image and video) the test asserts:
- no download affordance on the public share list page (card menu,
selection bar) and the public file detail page (viewer control bar,
top-nav file dropdown; video also has no resolution submenu)
- the proxy loaded in the detail page is the watermarked version
(-watermark- S3 keys) and no original transcode key is requested
- the download-url endpoint rejects guest download attempts with 403
Extends the e2e share helpers with apiUpdateShare, apiEnableShareWatermark
and apiPublicDownloadUrl, and documents the new cases in the e2e README.
* feat(webui): remove original-file display fallbacks, use proxies only
WebUI now never uses the raw original file for display. All viewers
(image, video, compare, pdf) render exclusively from transcoded proxies;
when no proxy exists the viewer shows an unavailable placeholder instead
of falling back to media.original.
- getBestTranscode: drop the isRaw fallback candidates
- video-viewer / compare-video-pane: remove the synthetic 'Original'
resolution entry and the original.downloadUrl hasMedia gate
- image-viewer / compare-image-pane: remove original.downloadUrl default
- pdf-viewer: require pdfTranscode.url (no original fallback)
- download menus (video-control-bar, breadcrumb-nav): use
media.original.key for the 'Original' download option instead of the
isRaw transcode marker entry
Backend:
- transcode-video workflow no longer pushes the isRaw:true marker entry
into videoTranscodes (it pointed at the original file key)
- remove media.original.downloadUrl from the DTO, asset service mapping,
MediaInfo DB type, and transcode workflows/activities (it was only
consumed by the removed display fallbacks; downloads already use key
via the download-url endpoints)
Tests/fixtures updated; docs updated to state the proxy-only display
policy.
* refactor: remove isRaw field via data migration, clean legacy media JSON
Add a Prisma data migration that cleans legacy media JSONB on both
assets and watermark_files:
- drop the raw-original marker entries (isRaw: true) from
videoTranscodes / imageTranscodes — they pointed at the original file
key and were only used by the removed display fallbacks / download
menus, which now use media.original.key directly
- strip the isRaw field from remaining transcode entries
- drop original.downloadUrl (removed from the AssetInfo DTO)
The migration is generated via prisma migrate dev --create-only and
applied automatically by migrate deploy in production. It is guarded by
a test (packages/db/src/clean-media-migration.test.ts) that seeds legacy
media JSON, runs the migration SQL, and asserts the transform on both
tables.
With the data cleaned, remove the isRaw field entirely from the codebase:
- DTO schemas + ImageTranscode/VideoTranscode interfaces
- PrismaJson MediaInfo transcode types
- watermark activity isRaw skip (and its test), watermark proxy mapping,
agent-embedding transcode selection, webui transcode filters, and all
test fixtures
AGENTS.md: clarify that the no-manual-migration rule applies to DDL;
data migrations (pure SQL transforms, no schema change) are allowed
inside a --create-only generated migration and must be tested.
* feat(sharelink): add sharelink watermark support
* fix(sharelink): harden watermark feature against races, crashes, and tenant leaks
Addresses review findings on the sharelink watermark feature:
- fix(watermark): read back sharelink via transaction client so the
updateShareLinkWatermark response reflects the just-committed state
(global-client reads on a pooled connection miss uncommitted changes)
- fix(watermark): use prisma upsert on the config hash instead of
find-then-create to avoid P2002 races on concurrent enables
- fix(watermark): resolve nested symlinks when walking the share tree and
batch the walk level-by-level instead of one query per node
- fix(watermark): keep watermark tasks in flight from being purged by
orphan-config GC (avoids FK violations in completing activities)
- fix(transcode): re-transcode stale 'processing' watermark rows instead
of silently skipping (crash recovery); downscale embedded block images;
use pino structured logging; drop unused any casts
- fix(api): scope watermark-template list/create to the caller's teams,
reject teamless template creation, and make teamless templates
read-only; validate watermark config required when enabling (400);
map sharelink-not-found to 404
- fix(api): resolve symlink target ids once in public share children
endpoint instead of twice per item
- refactor(db): dedicated WatermarkFileStatus enum (migration included)
- test: activity unit tests (init crash recovery, image + video transcode
paths), concurrent upsert, nested-symlink resolution, API scoping and
validation, e2e video watermark path
* refactor(transcode): move sharp usage out of watermark activity into transcodeService
The transcode package does not declare sharp as a dependency (only
@shumai/core does), so importing it directly in the watermark activity
broke CI typecheck for the transcode package.
- add transcodeService.renderSvgToPng, downscaleImageToPng and
compositeOverlayToWebpFile encapsulating the sharp logic
- watermark activity now calls these methods instead of importing sharp
- activity tests mock the new methods (no sharp/real fixture needed)
- add unit tests for the new transcodeService helpers
* feat(watermark): restrict watermark transcode to existing image and video transcodes
* refactor(transcode): reuse transcodeVideo in watermark activity
* refactor(watermark): use Bun.hash instead of node crypto for watermark config hash
* refactor(watermark): move computeWatermarkConfigHash from dtos to core watermark-hash module
* refactor(api): delegate watermark DB queries from public-share route to watermarkService
* refactor(share): replace dynamic watermarkService import with static import
* feat(watermark): add waitForWatermarkFileActivity for in-flight watermark transcoding tasks
* refactor(asset): replace dynamic watermarkService import with static import
* test(watermark): harden purgeOrphanWatermarkConfigs coverage
Replace the single happy-path GC test with a dedicated suite covering:
- orphan config with proxies: purged, watermarkFiles removed, all four
proxy keys deleted from S3 while the original key is preserved
- config still referenced by a share link is not purged
- config with pending/processing transcode_watermark tasks is not purged
(in-flight guard), but completed tasks do not block purging
- pending tasks of other types (transcode_video) do not protect a config
- an S3 delete failure is non-fatal: remaining keys still deleted and the
config is still purged
- mixed run: only the orphan config is purged while a referenced one
survives (exact count assertion)
* feat(webui): add sharelink watermark editor and sidebar controls
* style(webui): enlarge watermark editor dialog and allocate 2/3 width to preview column
* fix(webui): override sm:max-w-lg in DialogContent to allow full 1400px width
* fix(webui): stop block click propagation and remove preview mode toggles
* fix(webui): store watermark block size as fraction of canvas width
The editor stored block.size in pixels (slider 10-120px, preview
fontSize: `${block.size}px`) while the SVG generator
(generateWatermarkSvg) interprets size as a fraction (0..1) of canvas
width. A 36px block therefore rendered as a ~36x-canvas glyph covering
the entire image.
- Keep backend fraction semantics (canvasWidth * size) so watermarks
scale proportionally with output resolution across proxies
- Editor: defaults (text 5%, image 15%), slider ranges (1-12% / 2-50%)
and number input now work in percent, storing a fraction
- Preview: scales blocks by the measured canvas width so WYSIWYG
- DTO schema: size constrained to (0..1) to enforce the contract
- Update DTO/editor tests to fraction sizes
* feat(webui): add single-row aspect ratio selector and 5 predefined background colors to watermark editor preview
* feat(webui): update aspect ratio to multi-toggle group and enable dot-grid pattern for all background colors
* fix(webui): update background swatches to distinct colors and share PRESET_BACKGROUND_OPTIONS array
* fix(webui): hide redundant absolute close button on watermark editor dialog header
* fix(transcode): skip raw video transcode when generating watermark proxies
The raw marker entry (isRaw: true) in videoTranscodes points at the
original source file and is only used to offer the 'Original' playback
option. It is not a transcoded proxy, so watermarking it produced a
redundant full-resolution transcode (the most expensive pass) plus an
unneeded S3 upload.
Skip entries with isRaw: true in transcodeWatermarkMediaActivity, and
cover the behavior with a unit regression test plus an e2e assertion
that exactly one watermark proxy is produced for a video asset.
* feat(webui): replace watermark image asset ID with 1MB file uploader
* fix(webui): correct watermark editor preview block sizing
The preview canvas width was measured in a useEffect([open]) that ran
before the Radix Dialog had mounted its content, so canvasRef.current was
still null, canvasWidthPx stayed 0, and every block silently fell back to
the hardcoded 640px width (block.size * (canvasWidthPx || 640)). A 100%
size therefore rendered at ~640px on an ~876px canvas (~72%), making the
preview smaller than the real watermarked output for both image and text
blocks.
- Measure the canvas via a ref callback (fires exactly when the node
mounts, regardless of dialog mount timing) and keep a ResizeObserver
attached for aspect-ratio/window changes.
- Use clientWidth (padding-box width) instead of getBoundingClientRect(),
which includes the dialog's zoom-in-95 animation transform.
- Drop max-w-full/max-h-full on the preview <img> so logos scale up to
the block size like the backend SVG rendering does, instead of being
capped at their natural pixel size.
* fix(webui): keep watermark editor preview aspect ratio and restore config scroll
- Compute the preview canvas size with a JS contain-fit (ResizeObserver on the
frame) instead of relying on CSS aspect-ratio + w-full/max-h-full, which
broke the selected ratio on wide-and-short (or tall-and-narrow) windows.
- Add min-h-0 to the config ScrollArea so it shrinks to the column's remaining
height instead of growing to its content height, restoring the scroll range.
* fix(webui): remove toast notification on template load
* fix(share): serve per-file watermark proxies in public share APIs
Stop gating watermark replacement on the share-link-wide watermarkStatus.
Public share endpoints now resolve each asset's own completed watermark
proxy and:
- replace videoTranscodes/imageTranscodes with the watermarked entries
when the watermark file is completed;
- empty those arrays when no completed proxy exists yet (still
transcoding or failed) so original, unwatermarked media is never
exposed while a newly added file is being processed;
- leave non-video/image media (pdf, audio, ...) untouched, since they
never receive watermark transcodes.
This removes the previous behavior where adding a file to a ready
watermark share flipped the whole share link to processing and served
original media for every file until the new transcode finished.
Adds API tests for detail/children endpoints (ready, in-flight, pdf, and
watermark-disabled cases) and a workflow E2E test for adding an asset to
an already-ready watermark sharelink.
The handleDragEnd handlers in the Manage Fields dialog and Fields
dropdown guarded reordering on source.id !== target.id. With dnd-kit
react's OptimisticSortingPlugin (enabled by default), the drop target
always resolves to the dragged item itself, so the guard was never true
and the order API was never called.
Detect moves via the sortable-specific index/initialIndex properties on
the source instead, and reorder the rendered (grouped/search-filtered)
subset before merging back into the full field list. All field groups are
now reorderable in the dialog.
Add an optional 'context' parameter to the create_file and create_version
agent tools so the chat agent can pass generation/source context (e.g.
'Generated using gemini') to the AI metadata autofill workflow.
- Add Asset.autofillContext column, persisted in the same transaction as
asset creation when context is provided
- Add new list_autofill_fields agent tool exposing autofillable metadata
fields (name, type, description, options) per project
- Autofill workflow reads the stored context and includes it in the
autofill AI prompt
- Add short chat system-prompt guidance to use list_autofill_fields and
pass <=50 word context when creating files/versions
Add an optional 'data' field ({name, content}) to the create_file agent tool
so agents can create files without a local file (e.g. when bash is not
available). The existing 'path' upload mode is preserved; exactly one of
'path' or 'data' must be provided.
Extract mime detection helpers (getFileMimeType, readFileMimeType) into a
new file-mime module, keeping utils/mime.ts free of fs/path imports so it
stays compatible with the Temporal workflow bundle (agent-autofill imports
getProxyType from it). Refactor create_version to reuse the extracted
helpers.
Content uploads reuse s3Service.putObject; mime type is derived from the
filename extension with a text/plain fallback, and the name is sanitized
for the S3 key.
Two bugs found while testing skill loading as a non-owner:
1. The injected bash tool was never actually usable: AgentHarness.setTools
only updates the tool registry unless activeToolNames is passed, so bash
was missing from the active tool set sent to the model on the next turn.
Pass the full post-update tool list as activeToolNames.
2. The active_tools_change session entry (written by setTools) has no chat
representation, so mapEntryToMessage fell back to fabricating an empty
user message that appeared as a stray empty bubble in the chat stream.
Render active_tools_change as a typed custom message and skip entry
types that have no chat representation instead of emitting empty text.
A userId that is not a team member previously resolved to isOwner=true
(role === undefined || role === 'owner'), bypassing bash restrictions.
Treat only flows without a user context (autofill etc.) and explicit
owner memberships as unrestricted; unknown users are now restricted.
Editor/reviewer users can no longer ask the agent to run arbitrary bash
commands. Bash execution for them is gated behind approved team skills:
- Add required source param (enum user/skill) to the bash tool; commands
with source=user are blocked when the user is not a team owner.
- Append a Restricted User Context instruction to the system prompt for
non-owner users: never run user-requested bash, only skill-required
commands with source=skill.
- Delay bash tool injection for editor/reviewer: it is only added after
a skill is successfully loaded via read_skill (or restored when a
skill was loaded earlier in the session). Team owners are unaffected.
Add four project-level webapp e2e tests:
- invite a member to a project; the invitee only sees the invited project
- drag a file into a folder and a folder into another folder
- drag a file onto a file to create a version stack
- drag a file onto a version stack to add a new version
Supporting changes:
- expose rootFolderId on the project fixture
- add seedFolder/seedFile helpers and a cross-browser dragCard helper
(WebKit requires pointer moves in awaited batches for dnd-kit activation)
- add project-members-trigger testid to the file-browser toolbar
- document the new test cases in the e2e README
Add six domain-level web app e2e tests covering team member removal,
file upload/delete/restore, share link creation and password protection,
and collection creation. Introduce a parametrized 'file' fixture with
configurable media types (text/binary/image/video/pdf) for future
type-scoped tests, plus helpers for share APIs and team member seeding.
Also document all web app e2e test cases in apps/web/e2e/README.md and
require this documentation in AGENTS.md.
Both Agents and Members header buttons had 'sticky top-0', so while
scrolling the 256px mention list the Agents header stayed pinned and the
Members header snapped to the same top position, covering it exactly.
Remove 'sticky top-0' so headers scroll away naturally with their
sections.
- Render real avatar images (with initials fallback) for users in the
@-mention popup instead of a colored letter circle; bots keep the
colored fallback since BotInfo has no image field.
- Auto-scroll the highlighted mention row into view when navigating
the list with ArrowUp/ArrowDown keys.
Add comprehensive test coverage for 5 core service methods that were
added in the layered architecture refactor but lacked tests:
- AgentService.getAgent: found/not-found cases
- ProjectService.getUserProjects: team scope, project scope,
multi-team aggregation, limit, and empty result cases
- SkillService.getSkill: full field mapping and not-found cases
- TeamService.hasWritableRoleInAnyTeam: owner, editor, reviewer,
and no-membership cases
- UserService (new file): getUserById and createGuestUser
- Setup E2E package under packages/e2e
- Use testcontainers to spin up postgres and temporal dev server
- Overwrite activities in global registry to mock AI model output
- Track temporal workers and expose shutdownWorkers() method for clean shutdown
- Integrate test:e2e:workflow into root package.json and GitHub actions CI
API layer routes in packages/api/src/chat.ts accessed the database directly via prisma, violating communication layer boundaries.
This refactoring migrates these queries to packages/core/src/chat/chat.ts:
- Added getNewSessionMessages to fetch mapped SSE entries.
- Added getChatWorkflowStatus to check task completions.
- Added abortSession to handle workflow cancellation and task updates in core.
- Updated both core and API tests to match refactored signatures and mock boundaries.
- Update chat API routes and service methods to include and verify teamId context
- Validate that the requesting user is the creator of the chat session during continuation
- Update frontend chatbot store and components to pass teamId parameter
- Update backend API/core and frontend store tests to verify team scoping and creator checks
- Remove chatbotAgentId from team settings and schema definition
- Make agentId a required parameter in ChatRequest DTO and chatService
- Fetch chatbot agents on chatbot sidebar mount and default to first enabled chat agent
- Render an agent selector dropdown inside the message input area
- Update existing chat sessions to follow the selected agent when continuing a conversation
Use Bun's built-in file resolver to fall back to extension-based MIME type detection when the mediaType is missing, empty, or generic ('application/octet-stream').
Avoid deriving total frames from container duration when the audio tail mismatch is below the dynamic playback stall threshold. This prevents the rendering of phantom trailing frames that the playhead cannot reach during playback.
Also adds comprehensive unit and E2E tests covering both small and large audio tail gaps.
currentResolution state and currentSrcRef were only initialized on mount,
so if file.id changed without a remount the video.js init effect (keyed on
file.id) would re-create the player with a stale source. Add an effect
that resets them when file.id changes, before the init effect runs, making
the pane robust even without a per-asset key on the parent.
Switching a version in the public share breadcrumb navigated to the
authenticated /projects/:id/files/:id route, which triggered the
authenticated /api/projects/:id/team call (via resolveTeamIdFromPath) ->
401 -> redirect to login.
- breadcrumb handleVersionClick now routes to /share/:shareId/files/:id
with a version search param when public.
- public share route gains a version param; PublicShareManager fetches the
stack asset (for the version list) and the active version child asset
(for display/comments/fields), mirroring the authenticated page.
- compare exit sets version to the active side for parity.
Track when the cursor is over the control bar (VideoControlBar now
forwards mouse enter/leave) and skip the idle-hide timer while it is, so
the bar no longer disappears when the cursor rests on it. Also reduce the
fullscreen auto-hide delay from 2s to 1.2s.
The fullscreen control bar only auto-hid while playing, and a separate
effect force-showed it whenever paused, so it stayed visible permanently
when paused in fullscreen. Now in fullscreen the bar shows on cursor move
and hides after the cursor is idle (~2s) whether playing or paused;
windowed mode keeps the bar always visible.
The single-file player fullscreened the inner video-area element, which
is not an ancestor of the control bar, so the bar was never visible in
fullscreen. Fullscreen the player root instead so the floating control
bar is included and the existing show-on-move / hide-after-idle logic
works (YouTube-style).
Also make the control-bar colors theme-aware in all cases. The fullscreen
floating overlay now uses bg-card/90 + backdrop-blur + foreground text
instead of a hardcoded dark overlay, so it follows the app light/dark
theme. The former themedChrome prop is renamed to floatOverlayInFullScreen
(default true = single-file floating overlay; compare passes false to keep
its bar in-flow).
Add a themedChrome prop to VideoControlBar that keeps the normal
theme-aware chrome (bg-card / foreground text) while fullscreen instead
of the dark video overlay. Compare mode passes it so its fullscreen bar
matches the app light/dark theme for both video- and image-active panes
(ImageControlBar is already themed). The single-file player is unchanged
(themedChrome defaults to false -> dark overlay). The fullscreen toggle
icon still reflects the actual fullscreen state.
In the fullscreen control bar the whole bar uses text-white (it overlays
the dark video), but the zoom group and Fit button hardcoded bg-muted,
which is light in light mode -> white text on a light background. Use a
translucent bg-white/15 for those controls when fullscreen so they stay
readable; the normal (non-fullscreen) bar keeps bg-muted. Also fixes the
same latent issue in the single-file video player.
Force the dark palette on the compare pane topbar (via a scoped `dark`
class + system tokens, no hardcoded hex) so the bar background is dark in
both light and dark app themes. Active pane text uses text-foreground
(white in the dark scope); inactive pane uses text-muted-foreground
(light gray).
The active version's topbar now uses an inverted bar (bg-foreground /
text-background) so it reads as black background with white text in the
light theme, while the inactive pane keeps the card background with muted
text. Filename/version text color now differs between active and inactive
panes.
Previously clicking a non-active video pane both switched the active
version and toggled playback. Now a click on an inactive pane only makes
it active; play/pause is only toggled when clicking the already-active
pane.
Extract the video player's inline ControlBar into an exported
viewers/video-control-bar.tsx (VideoControlBar + PlayerState +
DisplayTranscode) and the image toolbar from file-viewer into an exported
viewers/image-control-bar.tsx (ImageControlBar). video-player and
file-viewer now consume the shared components with unchanged behavior.
compare-control-bar composes the same two shared bars (video vs image
active) with fan-out handlers, so the compare control bar can no longer
drift from the single-file players (e.g. the time-format dropdown).
Add a Compare Versions mode to the file detail page (authenticated and
public share) that splits the viewer into two 50/50 panes showing two
versions side by side.
- Breadcrumb file menu gains a Compare Versions entry (version stacks
with >=2 versions); the file segment is hidden in compare mode.
- URL-driven state (compare/cmpLeft/cmpRight/cmpActive) on both file
routes so the view is refresh/share-safe.
- Per-side topbar with filename, version, version-switch dropdown, an
active-side highlight, and an X exit button.
- Shared control bar reflects the active pane; for same-type media it
coordinates both panes (play/pause, seek, frame-step, zoom, pan sync),
seekbar length matches the active side while the shorter side holds its
last frame; only the active side plays audio; resolution/download apply
to the active side; fullscreen covers the whole compare view.
- Right sidebar (comments/fields) binds to the active side; comment
selection seeks the active pane and annotations/comments can be created.
- Reuses useFramePlayer + DrawingCanvas without modifying the existing
single-file video player.
- Unit tests for compare selection/clamp logic.
Display the video duration (mm:ss) in the bottom-right corner of the
video file card preview area with white text and a transparent
background. Duration is sourced from media metadata and exposed via the
preview DTO. The overlay is gated behind a showDuration prop so it only
appears on file cards, not folder card child previews.
Add an e2e spec covering restart-on-end for the frame-accurate video
player. Drives the real player to its natural end (seek near the tail,
play out the last frames) and asserts both restart triggers resume from
the start: the play/pause control and clicking the video area.
Adds seekToFrame and waitForEnded helpers to VideoPlayerPage.
Address review feedback: an async predicate with setTimeout inside
page.waitForFunction spawns overlapping executions at rAF rate, and checking
currentTime stability before arrival can resolve prematurely.
Reorder to arrival-first: wait for the target frame via a synchronous rAF-polled
predicate, then wait for currentTime to stabilize using a sequential Node-side
loop so browser evaluations never overlap.
Extend the backend-free video-player harness with the app's real comment↔player
wiring (onTimeUpdate capture, player.currentTime(second) seek on comment click,
and a remount to simulate a second user) and add a Playwright spec.
User A comments on 5 distinct random frames (regenerated each run); user B
reloads and clicks each comment. Asserts the seconds-based roundtrip is
loss-free: frameA -> second=frameA/fps -> seek(second) -> floor(second*fps+0.45)
== frameA, so the comment is stored at the frame the author saw and the reader
lands on that exact frame.
- __root.tsx: move useTeamContextStore/useEffect/useQuery above the
conditional 'return <Outlet />' so hooks run unconditionally in a
stable order (fixes 'Rendered more hooks than during the previous
render')
- team-selector.tsx: move single-team navigate() into a useEffect to
avoid updating the router Transitioner during render
Reflect the useFramePlayer changes: the currentTime/RAF path only drives the
playhead once rVFC has delivered a frame this session (or is unsupported / a
grace elapsed), pause locks to the last presented mediaTime when fresh, and
document the frame-accurate pause/resume audio-replay tradeoff.
On Safari/WebKit, video.currentTime (the playback clock) runs ahead of the
frame actually presented on screen at playback start, and the first
requestVideoFrameCallback arrives only after a delay. useFramePlayer drove the
playhead from currentTime immediately on play, racing it forward, then snapped
it backward when the first rVFC reported the true (lower) presented frame.
Fix 1 (play-start rewind): reset the per-session VFC state on each play and only
use the currentTime/RAF path when rVFC is unsupported, or has delivered a frame
this session and then stalled (e.g. video track ended while audio continues), or
never delivered within a safety grace. The playhead now follows the actually
presented frame instead of the leading playback clock.
Fix 2 (pause snap): on pause, prefer the last presented mediaTime when rVFC is
fresh so the playhead locks to the frame the user is looking at, instead of
jumping forward to the leading currentTime. Falls back to currentTime when rVFC
is stale/unsupported.
Add a deterministic useFramePlayer hook unit test (happy-dom) that simulates
Safari's leading currentTime and lagging mediaTime; it asserts the playhead is
monotonic on play and locks to the presented frame on pause (both fail on the
old logic), and that the end-of-track currentTime fallback still works.
The webui package was migrated from a Vite project but now uses the Bun
fullstack framework (Bun.serve + bun-plugin-tailwind) for dev/prod, and the
e2e harness serves via Bun too. Nothing in webui imports vite, @vitejs/plugin-react,
@tailwindcss/vite or vite-tsconfig-paths (the only vite-tsconfig-paths usage is
the root vitest.config.ts via the root devDependency), so drop them from the
webui package.
Verified: typecheck, lint, vitest and the playwright e2e suite all still pass
(Tailwind compiles via bun-plugin-tailwind).
Add a Playwright UI integration test framework for the frame-accurate video
player. Tests run against a backend-free harness served by Bun's native HTML
bundler (matching the project's Bun-based dev/prod serving), so no database,
auth or transcode pipeline is required.
- Add @playwright/test with chromium/firefox/webkit projects; the browser is
selectable via the Playwright --project flag (webkit = Safari engine).
- Serve the harness with Bun.serve + bun-plugin-tailwind (no separate Vite
toolchain); commit a deterministic 16x16/5s/30fps fixture video and matching
AssetInfo.
- Add a play/pause spec covering both the play control and clicking the video
area, asserting the native video clock, seekbar fill and frame readout stay
frame-locked after pause (settle-aware, tolerance-based assertions).
- Mock all backend API calls (anchored /api/ and /files/ routes) for isolation.
- Exclude e2e specs from Vitest; add a separate CI job (webkit on macOS).
- Store uploaded files as files/{ulid}/{original-name} instead of files/{ulid}/raw
- Store transcoded videos as {stem}-{resolution}.mp4 (e.g. foo-720p.mp4)
- Store transcoded images as {stem}-{width}p.webp (e.g. foo-1920p.webp)
- Add download?: boolean param to presign - only sets Content-Disposition: attachment when true
- Local storage handler only sets Content-Disposition when ?download=1 query param is present
- Add POST /api/files/download-url endpoint for key-based single file downloads
- Uses ResourceType.Asset for permission check
- Validates key belongs to the asset's storage directory
- Fix existing download-links API to pass download: true to presign
- Fix breadcrumb navbar download: video shows resolution submenu, others download directly
- Fix video player download: calls download-url API, shows correct file type label
- Remove filename query param hack from presign URLs
- Add sanitizeFilename() and stemFromKey() utility helpers
- Add key field to VideoTranscode DTO interface
When creating a version stack, the sortIndex was previously extracted
by taking files[0] from the database query result. Because findMany
without orderBy is non-deterministic, this resulted in flakiness
where the sortIndex would sometimes be null (from file2) instead of
'abc' (from file1), causing tests to fail.
We now look up the first file specifically based on the ordered
fileIds array parameter.
Checks if the container is running as root (UID 0) before executing
chown and gosu. If running directly as a non-root user (e.g. in hardened
environments like Kubernetes runAsNonRoot or via docker run --user),
it directly execs the command, preventing EACCES and permission errors.
This adds the `dist` and `data` folders to `.dockerignore` to prevent
them from being transferred as part of the Docker build context.
This reduces the build context size by over 1.5GB+, making the build
much faster.
This updates the Dockerfile and adds a scripts/docker-entrypoint.sh script.
Under native Linux (like Debian), when docker-compose mounts a host folder
as a volume (e.g. `./data:/app/data`), and the folder doesn't exist, the
docker daemon creates the folder as root-owned (`root:root`).
Because the container runs as a non-root user `bun`, it gets permission
denied (`EACCES`) when trying to write to the data directory.
To resolve this in a beginner-friendly way without manual host-side
permissions updates:
1. The runner stage now installs `gosu`.
2. At the end of the Dockerfile, the user switches back to `root` so the
container launches as root.
3. On launch, the entrypoint script executes. Since it runs as root, it
automatically changes ownership of `/app/data` to `bun:bun`.
4. The entrypoint script then uses `gosu` to drop privileges and launch the
actual server processes as the non-root `bun` user.
Capture the files list snapshot at the time the mutation is triggered via React Query's onMutate context, and retrieve it in onSuccess via the context parameter to prevent race conditions during rapid upload successions.
- Replace fetch uploads with XMLHttpRequest to capture upload progress
- Track active upload tasks and file progress in Zustand store
- Show byte-level uploader progress in the sidebar list
- Replace plain 'Uploading' card status text with a circular percentage animation
- Replace plain document icon in file list view rows with a mini progress circle
- Change query key invalidation of folders to the generic ['folders'] key to update the sidebar tree on create/rename/delete/move
- Separate search box from custom filters reset button in search dialog
- Remove unused plus icon from Assets sidebar header
- Add dropdown menu (rename, download, delete) to subfolder tree items in the sidebar
Moves the onTimeUpdate callback back into the native low-frequency timeupdate handler to avoid parent performance degradation, and removes it from the requestAnimationFrame loop and useEffect dependency array.
Updates the progress bar currentTime and progress state on requestAnimationFrame loops when playing, while keeping timeupdate as a fallback when paused.
@@ -90,6 +90,9 @@ Add any additional context, caveats, or follow-up work.
-`bun run format`
-`bun run typecheck`
-`bun run test`
-`bun run test:e2e:app`
-`bun run test:e2e:webui`
-`bun run test:e2e:workflow`
- **Backend Testing Mandate**: Every backend feature, service method, workflow, and activity MUST be accompanied by comprehensive tests. Logic-heavy code without corresponding test coverage is considered incomplete.
@@ -139,7 +142,7 @@ We follow Bun's monorepo conventions for dependency management:
3.**Local DevDependencies**: Tools specific to a single package (e.g., `@vitejs/plugin-react` for `webui`) should be declared in that package's local `package.json`.
4.**Workspace Imports**: Use `workspace:*` for internal package dependencies.
5.**Clean Root**: The root `package.json` must not contain runtime `dependencies`. It is reserved for shared `devDependencies` and workspace-wide `scripts`.
6.**Installation**: Always run`bun install` from the root. Use the `--filter` flag if you only want to update a specific package (e.g., `bun install --filter @shumai/webui`).
6.**Adding Packages**: Never edit `package.json` manually to add new dependencies. Always use`bun add` (e.g., `bun add <package> --filter <workspace>`) from the root without specifying an explicit version, letting Bun resolve the correct version.
## TypeScript Configuration
@@ -196,10 +199,15 @@ The frontend is built with:
- **Prefer System Colors**: Always prefer using semantic system color design tokens (e.g., `bg-background`, `text-foreground`, `text-sidebar-primary`, `text-muted-foreground`, etc.) over hardcoded color utilities (e.g., `bg-blue-50`, `text-emerald-600`, `bg-emerald-500`). This ensures seamless compatibility with light and dark themes.
### Route Code-Splitting
To optimize initial bundle size, all routes must be code-split into `[route].tsx` (lightweight route definition and loaders) and `[route].lazy.tsx` (UI component and heavy imports).
### Development
- **Run Dev**: `bun run dev` (Runs on port 3000)
## Testing
We use **Vitest** for testing (via `bun run test`). **Do not use `bun test`** as it uses Bun's native test runner which is not compatible with this project's test suite.
@@ -244,7 +252,7 @@ The `bunfig.toml` is configured to run `setup-tests.ts` automatically as a prelo
### API Tests
- Located in `packages/api/src/api/**/*.test.ts`.
- Located in `packages/api/src/**/*.test.ts`.
- Mock the Service layer using `vi.spyOn(service, 'method')` or `vi.mock('@shumai/core/src/team/team')`.
- Verify that the API calls the Service methods correctly.
@@ -267,6 +275,21 @@ describe('Team API', () => {
})
```
### Workflow E2E Tests
- Located in `packages/e2e/workflow/**/*.test.ts`.
- Run via `bun run test:e2e:workflow` (which executes tests against both `local` and `temporal` executors dynamically).
- **Mandatory test coverage**: When creating or updating workflows or activities, you MUST create or update the corresponding E2E tests.
- **Mocking**: Mock ONLY AI API calls. Do not mock S3, databases, or media/transcode extraction services.
- Mock the AI response by spying on `AgentHarness.prototype.prompt`:
### Web App E2E Tests
- Located in `apps/web/e2e/**/*.spec.ts`, organized by domain under `tests/<domain>/` (e.g. `auth`, `project`).
- Run via `bun run test:e2e:app`.
- **Fixtures**: Use the `owner`, `project`, and `file` fixtures for setup — they seed data through API calls/DB (no slow UI setup). You can add more fixtures if necessary.
- **Documentation**: Every test case MUST be documented in [`apps/web/e2e/README.md`](apps/web/e2e/README.md). Add or update the corresponding entry in the README whenever you add, rename, or remove a spec.
## Prisma Configuration & Migrations
- Schema: `packages/db/prisma/schema.prisma`
@@ -299,6 +322,9 @@ We use `prisma-json-types-generator` to enforce strict type-safety for Prisma `J
- Development: Use `bun --bun run prisma migrate dev` to create and apply migrations during development.
- Production: Use `bun --bun run prisma migrate deploy` to apply pending migrations in production environments.
- **No Manual DDL Creation**: Never hand-write **schema (DDL) migration** files or directories. Always generate them with Prisma CLI (`bun --bun run prisma migrate dev --create-only` to create a template, then edit it if needed, or `bun --bun run prisma migrate dev` to apply) so Prisma correctly tracks migration metadata and checksums.
- **Data Migrations Are Allowed**: Pure **data migration** SQL (no schema change) may be written by hand, but it MUST live inside a migration generated via `bun --bun run prisma migrate dev --create-only` so it is tracked and applied automatically.
- **No Automatic Dev DB Reset**: Do not run `prisma migrate reset --force` or commands that force-reset the database automatically. If migrations become out of sync or a reset is required, stop executing, report the situation to the user, and present suggested manual cleanup/reset steps for the user to execute.
### Commands
@@ -396,6 +422,7 @@ To prevent Temporal from indefinitely retrying fatal, expected business validati
## Radix UI / Shadcn UI Patterns
- **Modal Overlays**: When triggering a dialog (e.g., `Dialog`, `AlertDialog`) from inside a `DropdownMenu`, you must set `modal={false}` on the `<DropdownMenu>` component. Failing to do so will cause the UI to freeze due to conflicting focus management between the two modal components.
- **Scrollable Content**: Always use the Shadcn `ScrollArea` component (`@/ui/components/ui/scroll-area`) for scrollable content containers to ensure consistent custom scrollbar UI across the application.
## Logging
@@ -407,3 +434,51 @@ We use **pino** for logging. Always use **structured logging** to ensure logs ar
- **Levels**: Use appropriate levels: `debug`, `info`, `warn`, `error`.
## Internationalization (i18n)
All user-facing text in the WebUI (`packages/webui`) MUST be internationalized using **Paraglide JS**. Hardcoded text in UI components is strictly forbidden.
### Message Definition
1. Define your English message strings in [packages/webui/messages/en.json](packages/webui/messages/en.json).
2. Define the corresponding Chinese translation strings in [packages/webui/messages/zh.json](packages/webui/messages/zh.json).
3. Keys should use `snake_case`.
Example (`en.json`):
```json
{
"my_key": "My English Text"
}
```
### Compilation
When you add or update translation keys, you must compile them to regenerate the type-safe functions:
```bash
# Compile once (generates type-safe TS/JS functions on disk)
bun run i18n:compile
# Watch and recompile on change during development
bun run i18n:watch
```
### Usage in Components
Import the compiled message namespace `m` and call it as a function:
```typescript
import { m } from '@/ui/paraglide/messages.js'
export function MyComponent() {
return (
<div>
<h1>{m.my_key()}</h1>
</div>
)
}
```
Always use the absolute workspace alias path `@/ui/paraglide/messages.js` (or `runtime.js` for runtime features) when importing Paraglide outputs.
Shumai is an open-source Frame.io alternative built for creative teams to collaborate and share media with ease:
* **S3-Compatible & Local Storage**: Store and serve creative assets securely using the local filesystem or any S3-compatible storage service, including AWS S3, Cloudflare R2, and MinIO.
* **Frame-by-Frame Annotations & Comments**: Leave precise feedback with frame-specific drawing tools and timestamped comments on both video and image assets.
* **Secure Sharing & Collections**: Create private share links and curated media collections for clients, stakeholders, and collaborators.
* **Granular Access Control**: Manage workspace permissions with team-level and project-level role-based access control.
* **Distributed Transcoding via Temporal**: Offload resource-intensive video transcoding to a background worker pool orchestrated by Temporal.
* **Custom Asset Metadata**: Define dynamic metadata fields tailored to your production pipeline.
* **Share Link Watermarking**: Protect shared media with customizable text and image watermarks, and save watermark as reusable templates.
* **Wide File Format Support**: Preview common creative file formats, including images (with PSD support), videos, audio, and documents, while allowing any file type to be uploaded and stored.
* **Office & Document PDF Proxying via Gotenberg**: Convert Word, PowerPoint, Excel, Markdown, HTML, and CSV documents to high-quality PDF proxies for in-browser previewing using [Gotenberg](https://docs.shumai.one/using-shumai/gotenberg).
But Shumai goes beyond being a Frame.io alternative. With modern models like GPT-Image-2 and Seedance 2.5, AI can already handle a surprising amount of creative work. That means you can create or update videos and images from a simple prompt without having to jump back into Photoshop, Premiere or Houdini. That is why Shumai includes a powerful agent system:
* **Agent as a Team Member**: Mention a context-aware AI agent with `@mentions` directly in asset comments, just like a real teammate.
* **Agent as a One-to-One Assistant**: Chat privately with the agent and ask it to help with tasks.
* **Custom Skills & Tools**: Extend the agent by registering custom scripts, tools, and automation skills.
* **Model Context Protocol (MCP)**: Connect agents to external tools and APIs through remote MCP servers, with OAuth authentication and per-agent MCP access.
* **Isolated Sandbox Execution**: Run agent-submitted scripts safely in a sandboxed environment.
* **AI Metadata Autofill**: Automatically fill in custom metadata for new assets.
* **Semantic Search**: Find assets by visual or conceptual meaning using vector embeddings (Gemini Embedding 2).
* **Resource Quotas**: Set usage limits on AI tokens, costs, and tool execution per member, role, or team-wide, with real-time tracking.
Here is a short demo video showing what the Shumai agent can do:
Shumai also brings project management into the creative workflow. A built-in Kanban board helps teams organize tasks, track progress, and collaborate in one place.
The ultimate goal is an **agentic Kanban**, where agents and team members can work together on tasks. This is still a **work in progress**: you can already create tasks for AI agents, but agents cannot act on those tasks yet.
> **Media Processing & Storage:** When uploading media, Shumai always preserves and stores your original raw file. Note that the file size shown in the system represents the original file size and does not include the size of any transcoded files. For all media types (both images and videos), Shumai creates a 300p WebP poster image to serve as the preview in the file list view. Additionally:
> * **For Images:** Shumai generates a high-quality WebP proxy at the exact same resolution as your original file for optimized viewing.
> * **For Videos:** It generates at least one proxy video (depending on your transcoding settings), alongside a sprite image and a small preview video (capped at 1600 frames) to enable fast timeline scrubbing and lightning-quick previews.
---
## Installation
Below is a quickstart guide for running Shumai with local storage. For advanced configuration options (including S3-compatible storage and Temporal workflow orchestration), see our [Documentation](https://docs.shumai.one/getting-started/overview).
### Option 1: Docker Compose
Docker Compose is the fastest way to get Shumai running. You do not need to clone the repository or install packages manually. Ensure you have [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) installed, then follow these steps:
1. Create and navigate to a new directory for your configuration and data volumes:
* `SHUMAI_SERVER_PORT` controls the port the Shumai server listens on. The default is `3000`.
* By default, Shumai uses the bundled local storage service in this Docker Compose setup. To use an external S3-compatible storage instead, configure the environment variables for your S3 provider. Here are examples for AWS S3 and Cloudflare R2:
* By default, `AWS_ENDPOINT_URL_S3` is set to: `http://localhost:{SHUMAI_SERVER_PORT}`, if you use local storage and expose Shumai on a custom host/port combination, set `AWS_ENDPOINT_URL_S3` to the external URL that browsers will use to upload files.
For example, if you change the port mapping in `docker-compose.yaml` from `3000:3000` to `12345:3000` and deploy on a server with IP address `12.34.56.78`, set:
```
AWS_ENDPOINT_URL_S3: http://12.34.56.78:12345
```
This value must be reachable from client browsers and should include the externally exposed port.
5. Start the services in detached mode:
```bash
docker compose up -d
```
6. Open your browser and access Shumai at `http://localhost:3000` (or `http://<your-server-ip>:3000` for remote deployments).
### Option 2: Install via NPM / Package Manager
Shumai is published as `@shumai-one/shumai` on NPM. This option allows you to run Shumai globally or locally.
#### Step 1: Start PostgreSQL with pgvector
Shumai requires PostgreSQL with the `pgvector` extension. Start a pre-configured database container using Docker:
```bash
docker run --name shumai_postgres \
-e POSTGRES_USER=shumai \
-e POSTGRES_PASSWORD=shumai_password \
-e POSTGRES_DB=shumai_db \
-p 5432:5432 \
-d pgvector/pgvector:pg18
```
#### Step 2: Create a workspace folder
Create a dedicated directory to store your environment configuration and media files (which are saved in a `./data` directory by default):
Shumai requires the following system packages on Linux:
- **`ffmpeg`** – Used for media transcoding and metadata extraction.
- **`poppler`** (`poppler-utils`) – Used for PDF page image extraction and PDF sprite preview generation (`pdftoppm`).
- **`imagemagick`** – Used for PSD format transcoding and color profile conversion to sRGB.
- **`bubblewrap`**, **`socat`**, and **`ripgrep`** – Required by the AI agent sandbox (`anthropic-experimental/sandbox-runtime`) for process isolation, networking, and workspace search.
> **Ubuntu 24.04+** restricts unprivileged user namespaces by default. This prevents `anthropic-experimental/sandbox-runtime` from using `bubblewrap` for sandbox isolation.
> Alternatively, configure an AppArmor profile that grants the required `userns` permission to the relevant binaries.
---
#### macOS
Shumai requires:
- **`ffmpeg`** – Used for media transcoding and metadata extraction.
- **`poppler`** – Used for PDF page image extraction and PDF sprite preview generation (`pdftoppm`).
- **`imagemagick`** – Used for PSD format transcoding and color profile conversion to sRGB.
- **`ripgrep`** – Required by the AI agent sandbox (`anthropic-experimental/sandbox-runtime`).
Install the required packages with Homebrew:
```bash
brew install ffmpeg poppler imagemagick ripgrep
```
---
#### Windows (alpha)
> [!WARNING]
> Windows support in anthropic-experimental/sandbox-runtime is **alpha and the design is in flux**.
>
> The current implementation provides:
> - Network egress filtering via the **Windows Filtering Platform (WFP)**.
> - File read/write denial via **ACL stamping**.
>
> However, it is **not a security boundary against a deliberately adversarial sandboxed process**.
>
> The sandbox implementation will be substantially revised in a future release, where sandboxed processes will run under a dedicated sandbox user account.
Shumai requires:
- **`ffmpeg`** – Used for media transcoding and metadata extraction.
- **`imagemagick`** – Used for PSD format transcoding and color profile conversion to sRGB.
Install required packages using your preferred package manager:
# Optional: base URL for MCP OAuth redirects (defaults to BETTER_AUTH_URL)
MCP_OAUTH_REDIRECT_BASE_URL=http://localhost:3000
```
> [!NOTE]
> `SHUMAI_SERVER_PORT` sets the port the server starts on, while `AWS_ENDPOINT_URL_S3` is used by the browser to build file upload URLs.
> If deploying on a remote server (e.g. `http://123.456.7.8`) with a mapped port (e.g. `12345:3000` in docker-compose), set `AWS_ENDPOINT_URL_S3` to `http://123.456.7.8:12345`.
> `MCP_OAUTH_REDIRECT_BASE_URL` is only needed when MCP servers use OAuth: the authorization redirect URI defaults to `{BETTER_AUTH_URL}/api/mcp/oauth/callback`; set it to an externally reachable URL if the app is served behind a proxy.
#### Step 6: Run Shumai
> [!IMPORTANT]
> **For Bun Users:** Since the published package binaries contain a `#!/usr/bin/env node` shebang, running the global command directly (e.g., `shumai`) will execute under Node.js. If you installed with Bun and want to run under the Bun runtime, you must prefix all commands with `bun run --bun` (e.g., `bun run --bun shumai`, `bun run --bun shumai -d`, `bun run --bun shumai stop`).
Start the application from your workspace folder:
```bash
shumai
```
Alternatively, you can run and manage Shumai in daemon mode:
- **Start in daemon mode**:
```bash
shumai -d
```
- **Stop Shumai**:
```bash
shumai stop
```
- **Restart Shumai**:
```bash
shumai restart
```
- **Show/tail logs**:
```bash
shumai logs
```
On startup, Shumai will automatically run database migrations and start the web server at `http://localhost:3000`.
---
### Option 3: Run from Source (Development)
To set up Shumai locally for development:
1. Clone the repository and install dependencies:
```bash
git clone https://github.com/shumaiOne/shumai.git
cd shumai
bun install
```
2. Start the `pgvector` database container (as described in **Option 2, Step 1**).
3. Create a `.env` file at the root of the workspace using the configuration from **Option 2, Step 5**.
4. Generate i18n translation files:
```bash
bun run i18n:compile
```
> [!TIP]
> During frontend development, you can run `bun run i18n:watch` in a separate terminal to automatically recompile translation files whenever you edit strings in `packages/webui/messages/`.
5. Apply the database schema migrations:
```bash
bun run db:migrate
```
6. Start the local development server:
```bash
bun run dev
```
---
## Command Line Interface (CLI)
Shumai provides a Command Line Interface (CLI) tool to manage projects, folders, and assets, upload files/folders, and create new versions directly from your terminal.
For more details on installation and usage, see the [CLI Readme](apps/cli/README.md).
> **Bun 用户特别提示:** 由于发布的包二进制文件包含 `#!/usr/bin/env node` 头部声明(Shebang),直接在终端执行全局命令(例如 `shumai`)默认会通过 Node.js 运行。如果您是用 Bun 安装的并且希望调用 Bun 运行时,请在所有命令前加上 `bun run --bun` 前缀(例如:`bun run --bun shumai`、`bun run --bun shumai -d`、`bun run --bun shumai stop`)。
在您的工作区目录下执行以下命令启动应用:
```bash
shumai
```
您也可以让 Shumai 在后台以守护进程(Daemon)模式运行和管理:
* **后台模式启动**:
```bash
shumai -d
```
* **停止服务**:
```bash
shumai stop
```
* **重启服务**:
```bash
shumai restart
```
* **查看并跟踪实时日志**:
```bash
shumai logs
```
启动时,Shumai 会自动执行数据库迁移(Migration)并在 `http://localhost:3000` 拉起 Web 服务。
A Command Line Interface (CLI) tool for interacting with your Shumai instance.
The CLI allows you to view projects, list folder contents, create folders, upload files and directories, and create new versions of existing files directly from the terminal.
---
## Installation
The CLI package is bundled inside the Shumai workspace.
### Environment Setup
Before using the CLI, you must set the following environment variables to authenticate with your Shumai instance:
```bash
# The endpoint of your Shumai API server (e.g. http://localhost:3000)
exportSHUMAI_API_SERVER="http://localhost:3000"
# Your personal API Token (Generate this in the WebUI via Settings -> Developer tab)
exportSHUMAI_API_KEY="your-developer-api-token"
# Optional: Enable asset deletion capability (disabled by default for safety)
exportSHUMAI_ALLOW_DELETE="true"
```
---
## Usage
> [!IMPORTANT]
> **For Bun Users:** Since the package binary contains a `#!/usr/bin/env node` shebang, running `shumai-cli` directly will run under Node.js. If you wish to run it using Bun, prefix the command with `bun run --bun` (e.g., `bun run --bun shumai-cli project ls`).
Run `shumai-cli` or query specific commands:
```bash
shumai-cli <command> [arguments][options]
```
### Commands
#### 1. List Projects
List all projects you have access to. Returns the project ID, name, and its root folder ID.
```bash
shumai-cli project ls
```
#### 2. List Directory Contents
List direct children (folders and files) of a parent folder.
```bash
shumai-cli ls -p <parentId>
```
- **Options:**
-`-p`, `--parent <id>`: The parent folder ID (required).
#### 3. Create a New Folder
Create a new folder under a parent folder.
```bash
shumai-cli mkdir <name> -p <parentId>
```
- **Options:**
-`-p`, `--parent <id>`: The parent folder ID (required).
#### 4. Upload Files or Folders
Upload a local file or folder to a target parent folder. File sizes and paths are mapped, and uploads are streamed chunk-by-chunk with a progress bar.
```bash
shumai-cli upload <localPath> -p <parentId>
```
- **Options:**
-`-p`, `--parent <id>`: The parent folder ID (required).
#### 5. Create a New Version of a File
Upload a local file as a new version of an existing asset (which must be a file or a version stack). This wraps the assets into a version stack dynamically on the server.
-`-p`, `--parent <id>`: The parent asset ID (file or version stack) to version (required).
#### 6. Rename an Asset
Rename an existing file or folder in Shumai.
```bash
shumai-cli rename <assetId> <newName>
```
#### 7. Move Assets
Move one or more assets (files or folders) to a destination parent folder. Note that all assets to be moved must belong to the same current parent folder.
```bash
shumai-cli move <assetId...> -p <parentId>
# or using the short alias:
shumai-cli mv <assetId...> -p <parentId>
```
- **Options:**
-`-p`, `--parent <id>`: The destination parent folder ID (required).
#### 8. Delete an Asset
Delete a single file or folder by moving it to the Trash bin. For safety, this command only deletes one asset at a time and requires the `SHUMAI_ALLOW_DELETE="true"` environment variable to be set in your environment.
```bash
shumai-cli delete <assetId>
# or using the short alias:
shumai-cli rm <assetId>
```
---
## Development
To run the CLI in development mode inside the repository source code:
| `binary` (default) | `test-file-<ts>` | never transcoded, no proxy |
| `text` | `test-file-<ts>.txt` | gets a PDF proxy in real usage |
| `image` | `test-file-<ts>.png` | |
| `video` | `test-file-<ts>.mp4` | |
| `pdf` | `test-file-<ts>.pdf` | |
| `audio` | `test-file-<ts>.wav` | |
## Test Cases
### auth
| Test | File |
|---|---|
| Sign up as the first user and become the team owner | `tests/auth/signup.spec.ts` |
| Log in with existing credentials | `tests/auth/login.spec.ts` |
| Show error when accessing reset-password page without token | `tests/auth/reset-password.spec.ts` |
| Reset password with valid token and allow logging in with new password | `tests/auth/reset-password.spec.ts` |
### team
| Test | File |
|---|---|
| Owner invites a member via invite link and the invitee joins | `tests/team/invite-member.spec.ts` |
| Owner removes a member via the members dialog; the membership is deleted from the DB | `tests/team/remove-member.spec.ts` |
| Owner changes a team member role in the members dialog and permissions update | `tests/team/change-member-role.spec.ts` |
| Owner manages resource quotas in settings and resets usage from the dashboard | `tests/team/quotas-settings.spec.ts` |
### project
| Test | File |
|---|---|
| Owner creates a project with a cover image | `tests/project/create-project.spec.ts` |
| Owner uploads a file or an unsupported file via the "Upload File" context menu action | `tests/project/upload-file.spec.ts` |
| Owner deletes a file, finds it in recently deleted, and restores it | `tests/project/delete-restore.spec.ts` |
| Owner creates a share link from a file context menu and lands on the share page | `tests/project/create-share-link.spec.ts` |
| Owner protects a share with a password; guests must enter it to view | `tests/project/share-password.spec.ts` |
| Owner creates a collection from the sidebar; the collection shows project files | `tests/project/create-collection.spec.ts` |
| Owner invites a member to a project; the member only sees the invited project | `tests/project/invite-member.spec.ts` |
| Owner changes a project member role in the members dialog and permissions update | `tests/project/change-member-role.spec.ts` |
| Owner adds team member to project with role override and permissions enforce project role | `tests/project/role-override.spec.ts` |
| Owner deletes a member from project via the members dialog and project access is revoked | `tests/project/remove-member.spec.ts` |
| Owner drags a file into a folder and a folder into another folder | `tests/project/move-folder-file.spec.ts` |
| Owner drags a file onto a file to create a version stack | `tests/project/version-stack-create.spec.ts` |
| Owner drags a file onto a version stack to add a new version | `tests/project/version-stack-add-version.spec.ts` |
| Owner manages version stack: removes version from 3-version stack | `tests/project/version-stack-manage.spec.ts` |
| Owner removes version from 2-version stack which dissolves into standalone files | `tests/project/version-stack-manage.spec.ts` |
| Dissolving a shared version stack renders the remaining standalone file name on the share page | `tests/project/version-stack-manage.spec.ts` |
| Moving a shared file into a shared version stack leaves only 1 stack card on the share page | `tests/project/version-stack-manage.spec.ts` |
| Owner searches the file list by an English name keyword | `tests/project/search-file-list.spec.ts` |
| Owner searches the file list by Chinese, Japanese, and Korean name keywords | `tests/project/search-file-list.spec.ts` |
| Owner searches the file list with a keyword and a filter condition | `tests/project/search-file-list.spec.ts` |
| Owner saves a search result as a collection | `tests/project/save-search-collection.spec.ts` |
| Image share with watermark + download disabled: no download buttons on public list/detail pages, watermarked proxy served, download-url rejected | `tests/project/share-watermark-no-download.spec.ts` |
| Video share with watermark + download disabled: no download buttons on public list/detail pages, watermarked proxy served, download-url rejected | `tests/project/share-watermark-no-download.spec.ts` |
| Owner views a file and it appears in recents | `tests/project/recents.spec.ts` |
### file
| Test | File |
|---|---|
| Create a comment on a binary file | `tests/file/binary-comment.spec.ts` |
| Upload a txt file, wait for transcode, and create a comment with draw | `tests/file/txt-draw-comment.spec.ts` |
| Upload an image file, wait for transcode, and create a comment with draw | `tests/file/image-draw-comment.spec.ts` |
| Upload a video file, wait for transcode, and create a comment with draw at a non-zero timestamp | `tests/file/video-draw-comment.spec.ts` |
| Upload an audio file, wait for transcode, and create a comment at a non-zero timestamp | `tests/file/audio-timestamp-comment.spec.ts` |
## Conventions
- Seed data via fixtures/API/DB; use the UI only for the flow under test.
- Each test truncates all tables via the auto `prisma` fixture, so fixtures must
create their data per-test.
- App projects run with `workers: 1` because all tests share a single database.
- Every new test case MUST be documented in this README.
description: Comprehensive configuration reference for Shumai.
---
Shumai is configured via environment variables. You can define these variables in a `.env` file at the root of your application workspace, or pass them directly to your Docker containers.
| `DATABASE_URL` | PostgreSQL connection URL. Must target a database with the `pgvector` extension installed. | `postgresql://user:pass@localhost:5432/db?schema=public` |
| `BETTER_AUTH_SECRET` | A secure, random 32-byte secret used to sign session cookies and JWTs. | Generate with `openssl rand -base64 32` |
| `BETTER_AUTH_URL` | The public URL of your Hono backend API. Used for redirect flows. | `http://localhost:3000` (respects `SHUMAI_SERVER_PORT` fallback) |
---
## Storage Config
Shumai supports storing files locally or on S3-compatible endpoints.
| `STORAGE_BACKEND` | Storage type. Use `local` for local file system storage, or `s3` for S3-compatible cloud storage. | `local` (default) or `s3` |
| `AWS_ENDPOINT_URL_S3` | Endpoint for S3 storage or local storage. Used by the browser to build file upload URLs. Must include port if deploying remotely or using custom ports. | `http://localhost:3000` (respects `SHUMAI_SERVER_PORT` fallback if unset) |
| `PRESIGNED_URL_EXPIRES_IN` | Number of hours for which generated presigned download/upload links are valid. | `5` (default) |
### Local Storage Port Configuration
When `STORAGE_BACKEND` is set to `local` (default):
- `SHUMAI_SERVER_PORT` configures the port on which the Shumai web server starts and listens.
- `AWS_ENDPOINT_URL_S3` configures the public/external endpoint used by client browsers to upload and download files. It must include the scheme and port.
For example, if you deploy Shumai on a remote server at `http://123.456.7.8` and configure port mapping `12345:3000` (exposing port 3000 internally as port 12345 externally) in Docker Compose:
- Set `SHUMAI_SERVER_PORT` to `3000`.
- Set `AWS_ENDPOINT_URL_S3` to `http://123.456.7.8:12345`.
---
## Workflow Config
Configures whether background transcoding and AI agents are executed locally or offloaded to a Temporal cluster.
| `WORKFLOW_EXECUTOR` | Engine choice. Use `local` for in-process database polling, or `temporal` for distributed workers. | `local` (default) or `temporal` |
| `TEMPORAL_ADDRESS` | Host and port of the Temporal cluster gRPC service. | `localhost:7233` (default) |
| `EMBEDDING_CHUNK_DURATION` | The duration in seconds for each video chunk when generating video embeddings. | `60.0` (default) |
| `EMBEDDING_CHUNK_OVERLAP` | The duration in seconds to overlap video chunks when generating video embeddings. | `5.0` (default) |
description: Configure a high-scale production deployment using Temporal and S3-compatible storage.
---
For production environments or workloads with high-volume video processing and heavy AI agent usage, running background tasks on a single server can cause bottlenecks. Shumai resolves this by offloading task queue management, retries, and background executions to **Temporal**.
This allows you to decouple workloads and run specialized worker nodes on separate, dedicated machines.
---
## How it Works
When configured for Temporal, the core Shumai web application does not process background jobs itself. Instead, it publishes tasks to a Temporal Cluster queue.
```mermaid
graph TD
UserUpload[User Uploads Video] --> WebApp[Shumai Web UI / API]
Temporal -->|Queue: AI Agents| AgentWorker[Agent Worker Machine]
```
- **Transcoding Workers**: Can be run on GPU-accelerated or high-CPU compute instances.
- **AI Agent Workers**: Can be run on isolated, sandboxed hosts to safely execute automation scripts.
> [!IMPORTANT]
> **Storage Requirement**: Because the core application and the background workers run on separate machines in a distributed setup, **you must configure an S3-compatible storage backend** (such as AWS S3, Cloudflare R2, or MinIO). Local storage (`STORAGE_BACKEND=local`) is not supported in this mode because the workers and the web application do not share a common filesystem.
---
## Step 1: Deploy the Core Stack
On your primary host machine, spin up the database, Temporal cluster services, and the main Shumai web application:
Open the `docker-compose.yaml` file and set your S3 credentials and endpoint environment variables under the `shumai` service.
3. **Start the Services**:
```bash
docker compose up -d
```
This exposes the web application on port `3000`, the database on `5432`, the Temporal gRPC service on `7233`, and the Temporal management dashboard on `8080`.
---
## Step 2: Configure & Run Workers
On your specialized worker machines, you will run the dedicated background worker services. Each worker requires a `.env` file referencing the database, S3 storage, and the Temporal cluster:
Run this on your media processing nodes (requires system installations of `ffmpeg` and `ffprobe`):
```bash
npm install -g @shumai-one/shumai-transcode
shumai-transcode
```
### AI Agent Worker
Run this on your sandboxed agent execution nodes (requires sandbox dependencies like `bubblewrap` or `sandbox-exec`):
```bash
npm install -g @shumai-one/shumai-agent
shumai-agent
```
> [!IMPORTANT]
> **For Bun Users:** Since the worker package binaries contain a `#!/usr/bin/env node` shebang, running `shumai-transcode` or `shumai-agent` directly will run under Node.js. If you wish to run them using Bun, prefix the commands with `bun run --bun` (e.g., `bun run --bun shumai-transcode` or `bun run --bun shumai-agent`).
description: Learn how to spin up the Shumai stack using Docker Compose.
---
We support two Docker Compose setups depending on your environment and use case:
1. **Local Pre-built Stack**: A lightweight setup to run the core API and frontend using pre-built images.
2. **Temporal Stack**: A production-like environment with full background worker orchestration (Agent and Transcode workers) and Temporal services, built from your local source code.
---
## Prerequisites
Before starting, ensure you have the following installed on your machine:
- **Docker**: [Install Docker Desktop](https://docs.docker.com/get-docker/) or Docker Engine.
- **Docker Compose**: Included in Docker Desktop, or [installed separately](https://docs.docker.com/compose/install/).
- **Git**: To clone and manage the repository.
---
## 1. Local Pre-built Stack
This setup runs a PostgreSQL database (with `pgvector`) and the Shumai application container.
### Step 1: Navigate to the local docker-compose folder
From the root of your Shumai workspace, run:
```bash
cd docker-compose/local
```
### Step 2: Start the services
Spin up the containers in detached mode:
```bash
docker compose up -d
```
### Step 3: Access the application
Once all containers are running and healthy:
- Open your browser and navigate to **`http://localhost:3000`** to access the Shumai Web UI.
### Step 4: Tear down
To stop and remove the containers, run:
```bash
docker compose down
```
---
## 2. Temporal Stack (Recommended for Development)
This setup builds the containers from your local workspace source code and starts the API, workers, and Temporal scheduler.
### Step 1: Navigate to the temporal docker-compose folder
From the root of your Shumai workspace, run:
```bash
cd docker-compose/temporal
```
### Step 2: Start the services
Build the images and run the full stack:
```bash
docker compose up -d --build
```
### Step 3: Access the services
- **Web UI**: Access the Shumai application at **`http://localhost:3000`**.
- **Temporal Dashboard**: View active workflows and orchestration logs at **`http://localhost:8080`**.
### Step 4: Tear down
To stop the services and clean up volumes, run:
```bash
docker compose down -v
```
---
## Troubleshooting
### Port Conflicts
If you receive port conflict errors, ensure that port `3000` (Web UI), `5432` (PostgreSQL), `7233` (Temporal), or `8080` (Temporal UI) are not currently being used by other processes on your machine.
description: Learn about Shumai, a modern collaboration and media review platform.
---
## What is Shumai?
**Shumai** is a self-hosted media collaboration platform designed for creative teams, video editors, designers, and clients. It provides a centralized workspace where you can upload, organize, and review assets (video, audio, images, and documents) in real-time. With built-in AI capabilities and background transcoding, Shumai helps streamline asset organization and feedback workflows.

---
## Core Features
- **Secure Storage**: Host your creative assets locally on your server or connect directly to any S3-compatible cloud storage provider (like AWS S3, Cloudflare R2, or MinIO).
- **Media Commenting & Drawing**: Speed up review cycles with frame-level comments and drawing tools that let you sketch directly onto video frames or images.
- **Version Control**: Stack multiple versions of the same file into a single entry to keep your workspace organized and track revision history.
- **AI-Powered Tags & Search**: Automatically tag your files on upload using Google Gemini, and find assets based on visual or conceptual meaning.
- **Custom Automations**: Build tailored background workflows using custom scripts to automate repetitive team actions.
- **Proxy Transcoding**: Automatically convert heavy camera and design files in the background into web-friendly streaming formats for smooth playback.
---
## Deployment Scales
Depending on your team size and production volume, Shumai can be configured to run in one of two modes:
### 1. Single-Server Deployment (Default)
Ideal for small teams, local development, or low-volume workflows. In this setup, the main web server, database, and background tasks (like transcoding and AI tagging) all run on a single machine.
### 2. High-Scale Distributed Deployment
Recommended for production environments with high-volume video uploads. In this setup, background tasks are managed via a centralized queue (Temporal). This allows you to offload heavy workloads to specialized worker machines (for example, putting video transcoding tasks on high-CPU/GPU machines and AI agents on separate sandboxed environments).
_Note: S3-compatible cloud storage is required for distributed deployments so that all workers can access the uploaded files._
Open your browser and navigate to **`http://localhost:3000`** to access the Shumai Web UI.
### Production Deployments (Temporal)
If you are deploying Shumai for a production team with high-volume video uploads, we recommend setting up a distributed deployment using **Temporal** and an S3-compatible storage backend. This offloads resource-heavy video transcoding and AI agents to separate, dedicated worker nodes to keep your main dashboard fast and responsive. For setup instructions, see the [Deploy with Temporal](/configuration/temporal) guide.
---
## Option 2: Install via NPM
You can run Shumai as a global service using the Node Package Manager (NPM).
### System Prerequisites
To handle media processing and security isolation, the host machine must have the following system utilities installed:
- **FFmpeg & FFprobe**: Required for video transcoding and metadata extraction.
> `AWS_ENDPOINT_URL_S3` must include the scheme and port (e.g. `http://localhost:3000`) where Shumai is publicly accessible. This URL is used directly by client browsers for file uploads.
4. **Start Shumai**:
> [!IMPORTANT]
> **For Bun Users:** Since the published package binaries contain a `#!/usr/bin/env node` shebang, running `shumai` directly will execute under Node.js. If you installed using Bun and want to run it under the Bun runtime, prefix the command with `bun run --bun` (e.g., `bun run --bun shumai`).
```bash
shumai
```
Open your browser to `http://localhost:3000`.
---
## Option 3: Run from Source (For Developers)
If you wish to run Shumai locally to customize the platform or contribute:
1. **Clone the Repository**:
```bash
git clone https://github.com/shumaiOne/shumai.git
cd shumai
```
2. **Install Dependencies**:
```bash
bun install
```
3. **Start a Database**:
Launch a local database container (as described in the NPM section above).
4. **Run Migrations**:
Set up your `.env` file at the root, then apply database migrations:
```bash
bun run db:migrate
```
5. **Start the Dev Server**:
```bash
bun run dev
```
This spins up the web server on port `3000` with hot-reloading active.
description: Configure AWS S3 or any S3-compatible storage backend (Cloudflare R2, MinIO, Google Cloud Storage, etc.) for Shumai.
---
Shumai uses Bun's high-performance native S3 client to store and serve creative assets. By utilizing standard S3 APIs, Shumai can connect to any S3-compatible storage provider, including AWS S3, Cloudflare R2, MinIO, Google Cloud Storage, and more.
Using external S3 storage is highly recommended for production teams. It allows you to separate storage from the application server, scale storage capacity independently, and support distributed media transcoding.
---
## Configuration Variables
To enable S3-compatible storage, update the following environment variables in your `.env` file or Docker Compose configuration:
| `STORAGE_BACKEND` | Set to `s3` to enable S3 compatible storage. | Yes | `s3` |
| `S3_BUCKET` | The name of the S3 bucket where files will be stored. | Yes | `shumai-bucket` |
| `S3_REGION` | The region of your S3 bucket. | No | `auto` |
| `S3_ACCESS_KEY_ID` | The access key ID for your storage service. | Yes | `your-access-key-id` |
| `S3_SECRET_ACCESS_KEY` | The secret access key for your storage service. | Yes | `your-secret-access-key` |
| `AWS_ENDPOINT_URL_S3` | The endpoint URL of your S3 compatible storage provider. | Yes | `https://s3.us-east-1.amazonaws.com` |
| `PRESIGNED_URL_EXPIRES_IN` | Duration in hours before generated presigned download/upload URLs expire. | No | `5` |
---
## Provider Examples
Below are configuration templates for popular S3 providers. Add these variables to your `.env` file or under the `environment` section in your `docker-compose.yaml`.
### AWS S3
For standard AWS S3, configure the corresponding endpoint URL:
To connect to Supabase Storage via S3, ensure you have enabled S3 connection support in the Supabase Dashboard (`Settings > Storage > Enable connection via S3 protocol`). Copy the endpoint URL and region details from the same settings page:
Since Shumai uploads files directly from the user's browser using presigned URLs, you must configure **Cross-Origin Resource Sharing (CORS)** on your storage bucket. Without CORS configuration, uploads will fail with network or origin errors.
Ensure your bucket CORS policy allows the following rules from your Shumai application URL (e.g., `http://localhost:3000` or `https://app.shumai.one`):
description: Welcome to Shumai, a modern AI agent and media processing platform.
---
## What is Shumai?
**Shumai** is a highly decoupled, modular platform built with a layered monorepo architecture. It integrates a React-based frontend, a fast Hono API backend, business logic services, and background workflow engines (supporting both local and Temporal-based execution).
### Key Features
- **Decoupled Monorepo Structure**: Managed via Bun Workspaces, ensuring each domain or layer is its own self-contained package.
- **Robust API & Core Service Layers**: Built on Bun and Hono, featuring strict end-to-end type safety with Zod schemas and DTOs.
- **Workflow & Activity Engine**: An orchestration engine capable of polling-based execution locally and enterprise-grade execution via Temporal.
- **Modern UI**: A responsive, clean React interface powered by Tailwind CSS and Shadcn UI.
- **Database Resilience**: Prisma ORM with PostgreSQL and `pgvector` for vector similarity and search capabilities.
## Explore the Documentation
Start by setting up Shumai on your local machine using our step-by-step guides.
<CardGroup cols={2}>
<Card title="Get Started with Docker" icon="docker" href="/getting-started/docker">
Learn how to build and spin up the Shumai stack quickly using Docker Compose.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.