Compare commits

...
182 Commits
Author SHA1 Message Date
yiling 52330869a2 Release v0.4.2 2026-09-03 21:49:37 +08:00
Yiling-J 0c561e9b39 refactor(webui): use semantic system colors in notification settings (#379) 2026-09-03 21:36:28 +08:00
Yiling-J a236cd3a97 feat(provider): add provider and model sync feature with vendored generator (#378)
* feat(provider): vendor and adapt pi model generator

- Vendor generate-models.ts, model-data.ts, and models-dev-reasoning-options.ts from @earendil-works/pi tag v0.82.1
- Adapt imports to use @earendil-works/pi-ai and expose generateModels programmatic function
- Add compare.test.ts to verify vendored generator against original built-in catalog
- Add packages/core/src/provider/generator to eslint ignores

* feat(provider): add provider and model sync api and ui

* feat(webui): add checkbox for existing providers and clean dialog description

* feat(webui): add expand all and collapse all to sync providers dialog

* refactor(provider): remove compare.test.ts and cli overhead from generate-models

* fix(provider): add 30s timeout and strict error handling to sync catalog fetching

* fix(e2e): eliminate drag-and-drop race condition on webkit

Allow requestAnimationFrame collision updates to settle in dragCard helper before releasing pointer, and scope fileCard locator via data-testid to avoid matching sidebar tree items.
2026-09-03 21:01:14 +08:00
Yiling-J c3ef769f51 feat(provider): optimize provider and model configuration management (#377)
* feat(provider): optimize provider and model configuration management

* style(provider): use default tabs variant in provider settings dialog

* style(provider): make tabs list full width

* feat(provider): make model card row clickable and remove pencil edit icon

* feat(settings): persist active settings tab in url hash

* feat(agent): add search box and alphabetical sorting for provider and model selectors

* refactor(providers): remove frontend pagination and display all models in scrollable list

* fix(providers): address code review findings for model editing, i18n, and api protocol propagation
2026-09-03 15:16:16 +08:00
Yiling-J e58dfaa71a feat(agent): consolidate media tools into read_asset and add S3 key support to download_asset (#376)
* feat(agent): consolidate media tools into read_asset and add S3 key support to download_asset

* fix(agent): refine parameter descriptions and remove download URL from read_asset text content

* fix(agent): resolve owning asset and enforce RBAC permissions when downloading by S3 key
2026-09-02 21:51:07 +08:00
Yiling-J 35b57cc0ab feat(agent): support dynamic annotationId resolution across chat and comments (#375)
* feat(agent): improve timestamp precision and proxy selection for video screenshots

* feat(agent): support dynamic annotationId resolution across chat and comments

* fix(agent): scope annotation resolution to asset and format has_markup consistently
2026-09-02 15:17:56 +08:00
Yiling-J 5a754bcb94 feat(agent): improve timestamp precision and proxy selection for video screenshots (#374) 2026-09-01 21:15:28 +08:00
Yiling-J 64b68e7d63 feat(agent): move thread metadata to structured context XML (#373)
* feat(agent): move thread metadata to structured context XML

* feat(core): dynamically inject thread metadata in getSessionEntries
2026-09-01 17:42:56 +08:00
Yiling-J c2f477a391 fix(agent): hide main comment sessions in dashboard sessions list (#372)
* fix(agent): hide main comment sessions in dashboard sessions list

* feat(webui): auto mention comment creator when replying
2026-09-01 16:20:58 +08:00
Yiling-J 9263971653 更新 README_CN.md 以包含 Docker Compose 部署指南
添加了使用 Docker Compose 部署 Shumai 的详细说明和提示词。
2026-09-01 10:25:57 +08:00
Yiling-J 79a479c87c fix(webui): strip styles and preserve multiline text on paste in chat input (#371)
* fix(webui): strip styles and preserve multiline text on paste in chat input

* fix(db): remove custom log wait strategy in test global setup to prevent postgres startup race condition
2026-08-31 21:29:00 +08:00
Yiling-J 134eb8b7fa fix(agent): set empty details in asset tool results to avoid data duplication (#370) 2026-08-31 20:43:05 +08:00
Yiling-J e06478df88 feat(chat): support markups, attachments, and timestamps in agent 1-to-1 chat (#368)
* feat(chat): support markups, attachments, and timestamps in agent 1-to-1 chat

* feat(chat): improve message card layout with inline badges and integrated assets

* fix(chat): enable image attachment thumbnails in 1-to-1 chat cards

* fix(chat): address code review feedback on input state, mentions, i18n, and session deletion

* fix(chat): replace ring with border color on selected message card
2026-08-31 20:11:27 +08:00
yiling 6e9cf8dfb6 Release v0.4.1 2026-08-30 20:26:57 +08:00
Yiling-J d2620f6911 fix(webui): prevent empty folder flash during initial loading (#367) 2026-08-30 18:43:23 +08:00
Yiling-J 4537baa6a2 feat(webui): redesign comment attachment display into compact rows (#366)
* feat(webui): redesign comment attachment display into compact rows

* feat(webui): support pure frontend image extension matching and self-contained preview dialog

* fix(webui): size image preview dialog to min(image size, 80% screen size)

* fix(webui): prevent horizontal overflow of comment attachment rows in scroll areas

* fix(webui): centralize attachment image preview ownership and localize download title
2026-08-30 17:16:20 +08:00
yiling 810e0c4cf5 fix(agent): only include duration_seconds for time-based media (video/audio) 2026-08-30 15:51:23 +08:00
yiling 07cb648183 fix(agent): use total_frames for video and total_pages for pdf in asset context 2026-08-30 15:44:59 +08:00
yiling c6602c9f1c docs(readme): add docker pulls, release, and license badges 2026-08-30 15:32:27 +08:00
Yiling-J 86032539ed feat(agent): refactor message context into unified shumai_message (#365)
* feat(agent): refactor message context into unified shumai_message

* fix(agent): persist custom_message entry type in DatabaseSessionStorage and handle message mapping defensively

* fix(agent): pass clean prompt to harness and replace optimistic messages on shumai_message stream

* fix(agent): align asset context DTO contract, sync comment attachments, and fix session stats
2026-08-30 15:25:13 +08:00
Yiling-J b2ec012e10 feat(webui): add manage versions option to file card context menu (#364) 2026-08-29 19:19:49 +08:00
Yiling-J 7fd8623684 fix(asset,webui): disallow dropping version stacks onto files and version stacks (#363)
* fix(asset,webui): disallow dropping version stacks onto files and version stacks

* fix(webui): derive isSingleFile from total drag selection and remove unused hasVersionStacks
2026-08-29 16:09:36 +08:00
Yiling-J 083753a96d refactor(agent): remove executeAgentToolWorkflow and invoke core services directly (#362)
* refactor(agent): remove executeAgentToolWorkflow and invoke core services directly

* fix(core): ensure creation atomicity, strict assetType filtering, and id-desc listing
2026-08-29 15:19:08 +08:00
yiling 4b62c31f29 Release v0.4.0 2026-08-28 20:40:33 +08:00
Yiling-J 2208d4e0e0 fix(webui): align mobile comment cards and thread vertical indicator (#361) 2026-08-28 20:26:26 +08:00
Yiling-J df7f4c3c2d Update README_CN with project management details
Added information about project management and media handling in Shumai.
2026-08-28 20:25:46 +08:00
Yiling-J 12323e1416 Enhance README with project management and media details
Added information about project management features and media processing in Shumai.
2026-08-28 20:23:38 +08:00
yiling 2edc5e1071 fix(e2e): enable ryuk container reaper for app e2e tests 2026-08-28 16:22:29 +08:00
Yiling-J 2162ed9ebe fix(webui): fix breadcrumb file dropdown trigger stretching and alignment (#360) 2026-08-28 16:07:27 +08:00
Yiling-J 6acebb73ec feat(webui): add mobile friendly file detail and public share view (#359)
* feat(webui): add mobile friendly file detail and public share view

* fix(webui): address review feedback for mobile file detail and public share view

* fix(webui): use dynamic viewport and flex column layout for mobile file viewer

* fix(webui): support iOS WebKit video fullscreen and guard requestFullscreen

* fix(webui): compact page indicator and layout in mobile PDF control bar
2026-08-28 15:11:33 +08:00
EICCIO Advisors 7a1d6c0ab5 docs: add SECURITY.md with a private disclosure path (#357)
The project has no security policy, so a researcher with a finding today has
only a public issue or Discord, both of which disclose before a fix exists.

Routes reports to GitHub private vulnerability reporting. Names deployment
manifests as in scope, since the documented quickstart is what most operators
actually run. Adds operator guidance for the three things a self-hoster owns:
the session secret, not publishing the database port, and treating the agent
as a code-execution surface.

Response timelines are left for the maintainer to set.
2026-08-28 09:21:21 +08:00
Yiling-J d16e31c0e5 feat(webui): add simplified mobile-friendly layout for file and share lists (#356)
* feat(webui): add simplified mobile-friendly layout for file and share lists

* refactor(webui): switch mobile file list layout to 1-column grid

* feat(webui): show foldertree by default in child sidebar and add close button

* feat(webui): add collapsible section headers and fix folder card full-width on mobile

* feat(webui): improve mobile breadcrumb bar with middle truncation and dual ellipsis

* fix(webui): unify breadcrumb DOM structure across desktop and mobile

* fix(webui): remove unused breadcrumbs variable in BreadcrumbNav

* fix(webui): complete and confirm file uploads on mobile and handle errors properly

* fix(webui): remove active scale animation on mobile FAB to stabilize dropdown menu anchor

* refactor(webui): use PresignedUrl type from dtos for uploadFiles and eliminate any cast

* fix(webui): scope folder card max width to desktop with responsive md:max-w-[340px]
2026-08-27 20:44:08 +08:00
Yiling-J d9a8b6a191 feat(webui): truncate long file names with 100ms hover tooltip (#355) 2026-08-27 10:54:35 +08:00
yiling 5ec4cd7fcd docs: update documentation link in README and README_CN 2026-08-26 21:44:50 +08:00
Yiling-J a7925e3c79 fix(share): hide and soft-delete symlinks when target assets are trashed (#354) 2026-08-26 21:04:38 +08:00
Yiling-J 472b420620 fix(transcode): apply team hardware acceleration and source bitrate capping to watermark transcoding (#353) 2026-08-26 17:00:11 +08:00
Yiling-J 8f55ba1d37 feat(transcode): add hardware-accelerated video transcoding support (#352)
* feat(transcode): add hardware-accelerated video transcoding support

* feat(transcode): use encoder quality modes and source-capped maxrate for video transcode

* fix(transcode): scale max bitrate for downsampled preview framerates

* fix(transcode): propagate videoBitRate in getMediaInfoActivity and update auto description
2026-08-26 16:28:14 +08:00
yiling 7b1428bf51 Release v0.3.11 2026-08-25 22:05:47 +08:00
yiling d5b3b1868e fix(webui): reload video player and reset state when switching videos in viewer 2026-08-25 21:52:15 +08:00
yiling 49f3fc0be5 fix(webui): restore full width bottom bar spanning across carousel in file viewer 2026-08-25 21:27:20 +08:00
Yiling-J 59e641ccb3 fix(webui): prevent left carousel remounting and scroll jump on file selection (#351) 2026-08-25 21:01:51 +08:00
Yiling-J 1d9268b224 feat(project): add recent viewed files section to project folder tree (#348)
* feat(project): add recent viewed files section to project folder tree

* feat(webui): disable toolbar buttons and context menu in recents page

* test(core): optimize recents cap test execution time

* feat(webui): show hint text in recents page top toolbar

* feat(webui): rename recents section to Recently Viewed

* refactor(core): consolidate recents methods into AssetService

* fix(api): forward pagination parameters in folder children endpoint (#349)

* fix(project-recents): scope views and disable recents mutations

* fix(api): allow recent views in demo mode

* fix(api): limit demo recent view exception to post
2026-08-25 18:51:24 +08:00
Yiling-J 470a4bd6f7 feat(auth): add admin password reset CLI command and reset password web UI (#347)
* feat(auth): add admin password reset CLI command and reset password web UI

* fix(auth): invalidate previous reset tokens and use AsyncLocalStorage for link capture
2026-08-25 14:35:33 +08:00
yiling 8801d59b2e docs: symlink CLAUDE.md to AGENTS.md 2026-08-25 09:50:54 +08:00
yiling 6a20e06982 Release v0.3.10 2026-08-23 21:19:45 +08:00
Yiling-J b17abce2c2 fix(kanban): improve drag-drop to column bottom, remove empty drop line, and fix timeline alignment (#346) 2026-08-23 21:11:33 +08:00
Yiling-J f67e334016 feat(kanban): update task status permissions, reporter defaults, and participant access (#345)
* feat(kanban): allow editors to change task status and default reporter to creator

* fix(kanban): hide create task button for reviewers and eliminate duplicate error toast
2026-08-23 20:38:56 +08:00
yiling c1de66520e update readme image 2026-08-23 16:53:44 +08:00
Yiling-J 37e086d822 fix(webui): sidebar selection styling, kanban active state, pointer cursors, and owner dashboard (#344)
* fix(webui): update sidebar selection styling, kanban active state, and pointer cursors

* feat(webui): move dashboard to left sidebar and prevent mount animation

* fix(e2e): update create-project test to click Home button in sidebar
2026-08-23 16:39:56 +08:00
Yiling-J d21f8ca8de fix(kanban): use true thumbnail and poster for task asset previews (#343) 2026-08-23 14:36:46 +08:00
Yiling-J 6af4e7c2ec feat(kanban): resolve latest version for linked version stack assets and update demo login text (#342) 2026-08-23 11:19:48 +08:00
Yiling-J 1f2bdc8c9e feat(agentic kanban): phase 1 not-agentic kanban board (#330)
* feat(kanban): implement AI-native kanban database models, services, and REST APIs

* feat(webui): implement team kanban board and task management

* style(webui): increase kanban dialog widths and form container

* fix(webui): ensure kanban dialogs override sm:max-w-lg base class

* style(webui): set edit task dialog max width to 5xl

* chore: untrack kanban plan markdown files

* feat(webui): update kanban page layout to 2 columns and refine task activity UI

* fix(webui): constrain kanban column and board heights for vertical scrolling

* feat(kanban): unify task status transition into updateTask API and allow unconstrained human task transitions

* fix(webui): add optimistic updates to kanban drag-and-drop to eliminate flash

* refactor(webui): remove client-side search box from kanban header

* feat(kanban): redesign task dialog comments area and add attachment support

* refactor(webui): remove top status action bar from task info dialog

* fix(webui): narrow query cache predicate and add defensive guards for kanban drag optimistic update

* feat(kanban): add drag-and-drop task reordering with fractional indexing (#331)

* feat(kanban): add drag-and-drop task reordering with fractional indexing

* fix(kanban): improve card drop zones, snap indicators, and column highlight

* fix(kanban): enable card droppables only during active drag

* fix(kanban): refine hover styles and drop indicators for column boundaries

* fix(kanban): unify bottom drop indicator and fix top indicator clipping

* fix(kanban): ensure full column highlight and full coverage drop target

* minor change

* feat(kanban): allow agent tasks free drag-and-drop and enforce status change permissions (#332)

* refactor(kanban): rename task_type_agentic translation keys to task_type_agent

* feat(kanban): refactor task type enum to isAgentTask boolean (#333)

* refactor(kanban): remove phase 2 agent execution logic and streamline phase 1 kanban

* feat(kanban): add notifications and audit logging for kanban (#334)

* feat(kanban): redesign config area layout and add parent tasks multi-selector (#335)

* feat(webui): redesign kanban task dialog config area

* feat(kanban): simplify task dependencies with parent tasks multi-selector

* refactor(kanban): remove unused links routes and legacy dependency methods

* feat(kanban): connect assets and kanban tasks with many-to-many relationship (#336)

* feat(kanban): connect assets and kanban tasks with many-to-many relationship

* feat(kanban): improve asset picker layout, infinite scrolling, and breadcrumb icon styling

* fix(kanban): ensure asset picker scrollarea is scrollable with min-h-0 and shrink-0 flex styling

* fix(kanban): hide recently deleted in folder tree picker and adjust header padding

* fix(kanban): use lucide files icon instead of paperclip for linked assets

* feat(kanban): improve task deletion, goal sidebar interaction, and card styling (#337)

* feat(kanban): improve task deletion, goal sidebar interaction, and card styling

* feat(kanban): move agent task badge after priority and update to orange style

* feat(kanban): add status Compare-And-Swap (CAS) to prevent drag-and-drop race conditions (#338)

* feat(kanban): add status Compare-And-Swap (CAS) to prevent drag-and-drop race conditions

* style(kanban): rotate kanban icon 180 degrees in sidebar and top nav

* style(kanban): remove rotate-180 from SquareKanban icons

* feat(kanban): add unassigned goal record and fix all tasks count (#339)

* feat(kanban): add unassigned goal record and fix all tasks count

* feat(kanban): toggle task count with three-dot action button on goal card hover

* fix(ci): pin bun version to 1.3.14 across CI and Docker (#340)

* fix(webui): truncate goal name to max 2 lines in kanban sidebar (#341)
2026-08-23 10:28:56 +08:00
yiling 4c0fd29b7f Release v0.3.9 2026-08-18 18:42:59 +08:00
Yiling-J ffef67be8f feat(quota): move usage monitoring to team dashboard (#327) 2026-08-18 15:57:36 +08:00
Yiling-J 4d100bf542 docs(team-settings): add resource quotas documentation and update readme features (#326)
* docs(team-settings): add resource quotas documentation and update readme features

* fix(webui): correct sandbox filesystem description
2026-08-18 14:48:12 +08:00
Yiling-J 8a75d015a1 fix(quota): exclude agents from quota rules and usage records (#325)
* fix(quota): exclude agents from quota rules and usage records

* refactor(quota): remove agent_skill_call_count resource type and related logic

* chore(db): replace manual migration with prisma-generated migration
2026-08-18 11:56:03 +08:00
Yiling-J f806272c06 feat(settings): reorganize settings into personal, team, and AI sections (#324) 2026-08-18 08:35:23 +08:00
Yiling-J 4935eeff28 fix(webui): preserve selected chatbot agent preference on reload (#323) 2026-08-17 20:47:57 +08:00
Yiling-J eed0264b9c feat(quota): add 2-level database-backed resource quota system (#318)
* feat(quota): add database-backed resource quota system

* feat(webui): add resource quota management in team settings (#319)

* feat(webui): add resource quota settings in team management

* refactor(webui): update quotas list to full row layout and restrict skill/mcp to select list

* feat(quota): refactor quota system to 2-level rules and records

* fix(webui): fix overflow and scroll area in quota usage records dialog

* fix(webui): ensure quota records scroll area has explicit max height

* feat(webui): improve quota cards interaction and restrict editable fields in edit mode

* fix(quota): serialize initial records, validate merged updates, and reset on dimension changes (#320)

* feat(quota): replace network quota with agent tools quota (#321)

* feat(webui): show remaining time until next refresh in quota usage records

* fix(webui): rename bash quota resource to Agent Bash Calls

* refactor(quota): remove unused reserved field from quota records

* refactor(quota): remove quota rule cache (#322)
2026-08-17 20:24:05 +08:00
Yiling-J dd82daf65c feat(webui): use project members for asset comment mentions (#315)
* feat(webui): use project members for asset comment mentions

* refactor(project): remove legacy unused bots endpoint and props
2026-08-16 16:44:50 +08:00
Yiling-J 55d9596835 feat(authz): enforce project-aware roles for agent/skill/mcp permissions (#314)
* feat(authz): enforce project-aware roles for agent/skill/mcp permissions

Introduce a shared resolveEffectiveRole helper that mirrors AuthzService
precedence rules (projectMember.role override -> team role for team-scoped
members -> restricted), and route all agent/skill/mcp runtime permission
checks through it. Previously these checks used the raw teamMember.role,
which let project-only members execute team-level agents/skills/MCP tools
and ignored per-project role overrides.

Changes:
- authz: add resolveEffectiveRole(teamId, projectId, userId, db) and
  refactor hasPermission to use it (behavior-preserving)
- chat/asset/agent-activity: gate agent usage by the effective project role
- read_skill: enforce skill permission via effective role (projectId
  threaded from the agent session)
- mcp-service: checkPermission and buildAgentTools filter by effective
  role; un-permitted servers' tools are no longer advertised to the model
- agent activity: only advertise skills the user's effective role permits
- listProjectMembers: filter included agent bots by the requester's
  effective project role
- new GET /projects/:projectId/chat-agents API returning chat agents
  scoped to the project and the user's effective role; WebUI chat selector
  now uses it (settings keeps the team-level API)
- tests: effective-role resolver, chat/mention/MCP/skill project-role
  matrices, listProjectChatAgents, listProjectMembers filter, API route

* fix(agent): forward projectId to getAgentChatContextActivity in agentChat workflow

The agentChat workflow called getAgentChatContextActivity without passing
projectId from the task payload, so project-scoped members (scope:
'project') resolved a null effective role in the activity and were denied
with 'Insufficient role to use agent' even though the chat selector and
chat.ts both authorized them. Forward payload.projectId so the activity
enforces the user's effective project role, and assert the forwarding in
the workflow unit test (fails without the fix).
2026-08-16 15:44:45 +08:00
Yiling-J cb1c1c502b feat(agent): add permission controls for chat agents (#313)
* feat(agent): add permission controls for chat agents

* refactor(agent): extract shared agent permission helpers

Consolidate the duplicated role-hierarchy logic into
packages/core/src/agent/permissions.ts:

- getRoleLevel(): numeric level for a team role (fail closed at 0)
- getAgentRequiredLevel(): required level for an agent permission
  (defaults to reviewer/1)
- getAllowedAgentRoles(): inclusive role set for filtering

The typed Record over TeamMemberRole makes the mapping exhaustive,
so adding a new role fails compilation until all sites are updated.
Used by ChatService.startOrContinueChat, AssetService comment
mentions, getAgentChatContextActivity, AgentService.listAgents, and
TeamService.getTeamMembers, replacing four inline copies of the
hierarchy and the allowed-roles conditionals.

Also fix the API permission-denial test to mock HTTPException(403)
and assert a 403 response instead of a generic Error/500, matching
real authzService behavior.

* test(agent): fix flaky comment DAG tests by spacing ULID id generation

ULID ids have 1ms timestamp granularity; back-to-back comment creates
frequently land in the same millisecond, in which case Prisma ULIDs sort
by their random suffix instead of creation order. The DAG spine in
initializeAgentSessionActivity chains top-level comments by id, so a
shuffled order produces paths that omit preceding comments, making the
path-to-root assertions flaky (~80% of full-file runs locally).

Add a 2ms sleep between consecutive comment creates in the affected
tests, guaranteeing strictly increasing ULID timestamps. A 2ms delay is
sufficient because the next create always fires at least 2ms after the
previous ULID was generated. Production is unaffected: real comments
arrive seconds apart via separate requests.
2026-08-16 11:16:14 +08:00
yiling 6d4f62d6f6 Release v0.3.8 2026-08-15 20:35:30 +08:00
Yiling-J 43808216e6 fix(core): rename latest version child record on version stack rename (#312) 2026-08-15 16:18:11 +08:00
Yiling-J f66bb7af73 feat(version-stack): add manage versions dialog and version removal (#311)
* feat(version-stack): add manage versions dialog and version removal

* fix(webui): remove ellipsis and add pointer cursor to manage versions button

* fix(version-stack): handle symlink deduplication and name sync for share links

* refactor(version-stack): extract shared symlink dedup helper

The 'repoint first symlink per share folder, delete duplicates with
fileCount decrement' logic was triplicated across createVersionStack,
reparentAssets, and reparentFileToFile with subtle variations. Extract
it into dedupeSymlinksToTarget() in packages/core/src/asset/symlink.ts
and call it from all three sites (behavior unchanged), with a dedicated
test suite covering the edge cases.
2026-08-15 15:37:17 +08:00
Yiling-J ae86d56a7d feat(webui): improve AGENTS.md editor placeholder and read-only notice (#310)
* feat(webui): improve AGENTS.md editor placeholder and read-only notice

* docs: add AGENTS.md documentation under Shumai Agent Settings
2026-08-15 11:00:30 +08:00
Yiling-J 1cee103b1c feat(agent): add nested agents.md support across backend and webui (#309)
* feat(agent): add nested agents.md support across backend and webui

* fix(webui): update agentsmd route paths and unnest folder agentsmd route

* fix(agent): flush agents.md autosave on close and address review feedback

- Flush pending debounced AGENTS.md autosave on close/unmount so edits
  made within 750ms of navigating away are no longer silently lost
- Serialize autosaves through a promise chain so an older in-flight
  request can never overwrite newer content on the server
- Internationalize the hardcoded "Failed to save" and
  "Loading AGENTS.md..." strings (en + zh)
- Validate that AGENTS.md is only stored on folder/root assets in
  AssetService, preventing bogus virtual paths for file assets
- Cap updateAgentsMd content at 100k chars via Zod validation
- Remove unused getAgentsMd/updateAgentsMd response DTO schemas
- Drop unused teamId prop from AgentsMdEditor
- Add tests: non-folder asset rejection, oversized content 400,
  and agent-chat workflow assetId passthrough

* refactor(db): isolate AGENTS.md content in dedicated 1:1 AssetAgentMd table

* style(webui): unify top bar button hover styles and add wand icon to agentsmd button

* style(webui): update agentsmd button state colors and remove presence dot

* fix(webui): fix close button overflow and stabilize debounced autosave in agentsmd editor

* fix(webui): update debounce to 1s and await pending save before closing agentsmd editor

* feat(webui): add terminal AGENTS.md breadcrumb on agentsmd editor page

* fix(webui): use ScrollArea for markdown editor scrolling
2026-08-15 10:25:46 +08:00
Yiling-J 3c3b6880e4 feat(webui): replace legacy markdown editor with new lexical wysiwyg editor (#308)
* feat(webui): replace legacy markdown editor with new lexical wysiwyg editor

* refactor(webui): organize markdown editor files into components/markdown-editor folder
2026-08-14 16:27:14 +08:00
yiling 0b84402c83 Release v0.3.7 2026-08-14 14:47:18 +08:00
Yiling-J 61a61abc7c fix(webui): fix select field height and multi-select overflow/interaction issues (#307)
* fix(webui): standardize single-select custom field height

* fix(webui): fix multi-select badge overflow calculation and simplify direct edit
2026-08-14 14:35:17 +08:00
Yiling-J fc878775c2 feat(agent): support dynamic newOption in select and selectMulti metadata fields (#306)
* feat(webui): redesign select and multi-select metadata fields with search box and dynamic option creation

* fix(webui): stabilize select popover placement and preserve field visibility in store

* refactor(webui): enforce 2-row collapsed height and whitespace-pre-wrap on LongTextField

* feat(agent): support dynamic newOption in select and selectMulti metadata fields

* fix(metadata): address code review feedback on selectMulti item validation and WebUI option deduplication
2026-08-14 10:56:18 +08:00
Yiling-J 7acf736300 feat(webui): redesign select metadata fields and improve long text field widget (#305)
* feat(webui): redesign select and multi-select metadata fields with search box and dynamic option creation

* fix(webui): stabilize select popover placement and preserve field visibility in store

* refactor(webui): enforce 2-row collapsed height and whitespace-pre-wrap on LongTextField
2026-08-13 22:46:40 +08:00
Yiling-J 9d50301a43 feat(sandbox): default Network Sandbox to off and add settings toggle switch (#304) 2026-08-13 20:50:34 +08:00
Yiling-J 7f185ca23c feat(agent): type-safe CREATION_CONTEXT metadata on create_file/create_version (#303)
* feat(agent): type-safe CREATION_CONTEXT metadata on create_file/create_version

Build the create_file and create_version tool metadata parameter from the
project's CREATION_CONTEXT fields at harness creation time, instead of the
schemaless key-value map that required the agent to call list_autofill_fields
first.

- Thread projectId from the chat workflow into createAgentSession and build
  a strict, all-required (nullable) metadata schema via fieldsToTypeBoxSchema
  (additionalProperties: false, option IDs enforced via enum).
- Remove the list_autofill_fields tool and its activity case.
- Reject metadata keys that are not CREATION_CONTEXT fields of the target
  project with a clear non-retryable error telling the agent to omit metadata.
- Skip null metadata values when persisting (create_file/create_version and
  the autofill workflow).
- Switch the touched schemas from @sinclair/typebox 0.34 to typebox v1 to
  match the harness validator.

* refactor(agent): migrate remaining tools from @sinclair/typebox 0.34 to typebox v1

All agent tool schemas now use the same typebox v1 package as the harness
validator; drop the @sinclair/typebox 0.34 dependency from @shumai/agent.

* test(agent): reproduce null coercion in harness tool validation

The pi-ai harness (<=0.82.1) coerces null values in [string, null] unions to
'' during tool argument validation for typebox v1 schemas. A valid create_file
call with path: null + data: {...} therefore arrives at execute with path: ''
and fails with 'Provide exactly one of path or data'. Null metadata values
(the 'unknown' sentinel) are likewise turned into ''.

This test reproduces both failures. It is expected to fail until
@earendil-works/pi-ai is upgraded to >=0.84.0 (#7328), which preserves values
that already match a union arm.

* fix(agent): patch pi-ai 0.82.1 to preserve null in nullable union tool args

Apply the upstream fix for earendil-works/pi#7328 (PR #7373) via a bun
patch instead of upgrading pi-ai. coerceWithUnionSchema now returns the
original value when it already validates against any union member,
instead of coercing null in [string, null] unions into "".

Fixes the create_file false "Provide exactly one of path or data" error
when the model passes path: null with data. Reproduced by
packages/agent/src/reproduce-tool-null-coercion.test.ts, which now passes.

* fix(docker): copy bun patch dir before frozen-lockfile install

The pi-ai patchedDependencies entry makes bun install --frozen-lockfile
fail in the builder stage because patches/ was not present in the image
yet. Copy it alongside package.json/bun.lock so the patch applies.

* refactor(agent): simplify metadata guidance in system prompt and error message

Shorten the create_file/create_version metadata instructions in the system
prompt and reword the invalid-metadata-key error to not reference
CREATION_CONTEXT internals. Update the matching test assertion.
2026-08-13 20:06:09 +08:00
yiling 94582c9001 try fixing aliyun docker push 2026-08-13 15:06:44 +08:00
Yiling-J 41b73feeab feat(metadata): replace aiAutofill boolean with autofillSource enum (#302)
* feat(metadata): replace aiAutofill boolean with autofillSource enum

* fix(agent): validate creation-context metadata and make writes transactional

Address code review feedback on the autofillSource migration:

- Update the agent system prompt to reference the 'metadata' parameter of
  create_file/create_version instead of the removed 'context' parameter,
  which was being silently discarded by the activity
- Write metadata inside the caller's transaction via the new
  metadataService.updateAssetMetadataInTx, keeping metadata rows atomic
  with asset creation instead of opening a separate non-atomic transaction
- Reject metadata keys that are not defined CREATION_CONTEXT fields of the
  project with a non-retryable ApplicationFailure before creating the asset
- Remove dead agentContext handling from autofillAiActivity and its tests,
  and drop the unused ai_autofill i18n keys (paraglide recompiled)
- Derive the DTO autofillSource enum from the AutofillSource const instead
  of a hardcoded z.enum to prevent drift
2026-08-13 13:41:46 +08:00
yiling 96493a5493 Release v0.3.6 2026-08-12 14:56:54 +08:00
Yiling-J 93c593545d fix(i18n): update Chinese translation for dashboard to 仪表板 (#301) 2026-08-12 14:46:46 +08:00
Yiling-J 11758c23c0 docs(mcp): add MCP support documentation to README and docs (#300)
* docs(mcp): add MCP support documentation

* docs(readme): simplify MCP feature description

* docs(mcp): fix inaccuracies in MCP servers documentation

- Clarify that discovery runs on refresh, OAuth completion, or first
  tool call, not automatically at server creation
- Note that advanced settings (transport, auth, permission, direct
  tools) are configured in the edit dialog after creation
- Correct agent assignment UI description: switch toggle, not checkbox

* docs(mcp): explain proxy vs direct tool configuration

Add a dedicated section covering the three tool states (Off / Proxy /
Direct), what the mcp proxy tool does and why it exists, and when and
why to use direct tools, including the context-token trade-off.
2026-08-12 14:26:10 +08:00
Yiling-J 1cd94a77c6 feat(webui): direct canvas PDF rendering with selectable text (#299)
* feat(webui): render pdf pages via live canvas with selectable text

Replace the rasterize-to-PNG pipeline in the PDF viewer with direct
canvas rendering that tracks the current zoom, so text stays crisp when
zoomed in, and overlay a pdf.js TextLayer that makes text natively
selectable when not in draw mode.

- pdf-viewer: render pages onto a live canvas at a zoom-proportional
  scale (debounced, memory-capped) instead of a fixed 2x PNG snapshot
- pdf-viewer: add pdf.js TextLayer overlay per page, transformed in
  lockstep with the Konva stage so spans stay aligned while panning
  and zooming
- drawing-canvas: accept an optional live canvas element and force a
  layer redraw when it is swapped
- add text-layer.css with the minimal pdf.js text layer styles
- add e2e spec verifying text selection in view mode and that draw
  mode routes drags to the canvas

* fix(webui): show pdf text selection highlight above the page canvas

The pdf.js text layer was stacked below the opaque Konva page canvas, so
the native selection highlight was painted underneath it and invisible
(selection itself worked). Move the overlay above the canvas and gate its
pointer-events on draw mode: auto when not drawing (select text), none in
draw mode so drags reach the Konva stage below (draw markup).

Verified with screenshot pixel analysis: the selected region now renders
the highlight (blue) where it was pure grayscale before.

* fix(webui): transparent selection background for pdf text layer br elements

* fix(webui): stop pdf text selection over-selecting in empty space

The text layer container was user-selectable (user-select: auto), so when
a drag's caret moved into empty space Chromium targeted the container and
extended the selection to a container-content boundary, selecting across
rows (e.g. dragging from line 1 into the gap selected lines 1-2).

Make the container user-select: none while keeping the text spans
user-select: text, so the caret clamps to the nearest selectable text and
the selection stays put when dragging into empty space.

* fix(webui): deselect pdf text when clicking empty space

With user-select: none on the text layer container, Chromium no longer
clears the selection when clicking non-selectable areas. Clear it manually
on mousedown over the container or <br> (not a text span) so clicking
empty space between lines or outside text deselects, matching native
behavior.
2026-08-12 13:58:02 +08:00
Yiling-J 5ad3fdbf69 fix(webui): content-only pinch-zoom and two-finger pan in media viewers (#298)
* fix(webui): add content-only pinch-zoom and two-finger pan to media viewers

Intercept trackpad pinch (ctrl+wheel), touchscreen pinch (two touches),
and two-finger move inside the image, video, and PDF viewers so they
transform only the media content instead of triggering browser page
zoom (matching Frame.io behavior).

- New shared pan-zoom math (zoomAtPoint, fitScale, isZoomed) and a
  usePanZoomGestures hook using native non-passive wheel/touch listeners
  with touch-action: none on the media container.
- Plain wheel / two-finger scroll pans the content, but only once the
  user has zoomed in; drag-pan is gated the same way.
- Image and PDF viewers no longer reset zoom to fit on container resizes
  while the user has manually zoomed (matches the video viewer).
- DrawingCanvas ignores multi-touch moves so single-pointer pan/draw
  doesn't fight the pinch handler; the video viewer keeps play/pause on
  the container div to avoid double-toggling when the overlay is active.

Includes unit tests for the pan-zoom math.

* fix(webui): disable click-drag pan for pdf and video viewers

When zoomed, PDF and video previously panned on left-button (or
single-finger touch) drag via the shared DrawingCanvas. That conflicts
with left-click semantics: PDF wants the drag for text selection and
video reserves the click for play/pause.

Remove the gated onPan from both viewers so the Konva overlay stays
pointer-events:none when not drawing - clicks pass straight through
(video click-to-play now works cleanly at any zoom) - while panning
remains available via mouse wheel and two-finger gestures. Image keeps
drag-pan.
2026-08-12 10:54:42 +08:00
Yiling-J 851ae49c84 feat(mcp): support tool-level directTools array and 3-state UI configuration (#297)
* feat(mcp): support tool-level directTools array and 3-state UI configuration

* refactor(mcp): refine tool hint explanation and separate server instructions block

* style(mcp): display tool count in header title and split option hints into 3 lines

* style(mcp): update tools description paragraph and place bulk buttons on a separate row

* fix(mcp): enforce excludeTools in proxy path and align direct-tool counts

- Filter the per-agent tool registry by excludeTools at build time (DB cache
  and lazy-connect paths), so excluded tools are unreachable through the mcp
  proxy (call, search, list, describe) — matching pi-mcp-adapter where
  isToolAllowed gates both direct and gateway exposure.
- Add shared isDirectTool predicate (matches original or prefixed name) used
  by both buildDirectTools and the proxy description counts, so the
  advertised direct/proxy counts can no longer drift from actual
  registration; toolCount now reflects allowed tools only.
- Remove the legacy boolean directTools branch (no legacy data exists).
- Drop the now-unused mcp_excluded_tools_hint i18n key and regenerate
  Paraglide messages.
- Extend unit tests: excluded tools are not counted, listed, searched, or
  callable via the proxy; prefixed-name directTools configs count correctly.
2026-08-11 15:43:27 +08:00
Yiling-J 82aee9a291 fix(agent): resolve version stack name and size in list_assets (#296)
* fix(agent): resolve version stack name and size in list_assets

* fix(agent): resolve version stack to latest version in download_asset, analyze_image, and read_pdf_pages
2026-08-11 12:09:29 +08:00
Yiling-J 1b5cbf936a feat(mcp): add MCP server support with proxy tool and OAuth flow (#289)
* feat(mcp): add MCP server support with proxy tool and OAuth flow

Add URL-based MCP (Model Context Protocol) server support to the backend:

- DB: McpServer / AgentMcpServer / McpServerCredential models with
  typed JSON columns (auth config, cached tools, stored OAuth tokens,
  pending auth state) and a migration.
- Core (@shumai/core/src/mcp): DB-backed credential store, OAuth
  client provider (authorization_code + client_credentials) with a
  Hono-hosted redirect callback, connection lifecycle manager
  (Streamable HTTP with SSE fallback, lazy connect, 401 -> needs_auth),
  JSON Schema -> AgentTool parameter conversion, in-memory tool registry
  with ranked search, the single 'mcp' proxy AgentTool (search /
  describe / list / call / connect / status / auth-start), per-server
  opt-in direct tools ({server}_{tool}), and McpService (CRUD,
  discovery, OAuth, per-agent assignment via AgentMcpServer, and
  permission gating on tool execution mirroring skills).
- Agent: chat agents receive the 'mcp' proxy tool (plus direct tools
  for opted-in servers) built from their assigned, enabled servers.
- API: /teams/:teamId/mcp/servers CRUD, tools refresh/list, auth
  start/complete/status/remove, test, permission, and the public
  OAuth callback route; Admin authz via new ResourceType.McpServer;
  audit logging.
- DTOs: zod schemas + types for servers, tools, auth, and per-agent
  mcpServerIds.
- Tests: DB store, schema conversion, registry/search, service
  integration against a real in-process MCP server (v2
  @modelcontextprotocol/server), permission gating, rename
  invalidation, direct tools, per-agent assignment, API route tests,
  and workflow E2E (local + temporal) for proxy and direct modes.

* fix(mcp): persist OAuth discovery snapshot and stamp SEP-2352 issuer

The callback leg of the OAuth flow failed with
AuthorizationServerMismatchError ('discoveryState was not available on
the callback leg') because McpDbStore.savePendingAuth overwrote the
entire pendingAuth JSON column, clobbering the discoverySnapshot that
the provider's saveDiscoveryState() had stored inside it.

- McpDbStore.savePendingAuth now preserves an in-flight discovery
  snapshot so the callback leg can satisfy the SEP-2352 authorization-
  server issuer check.
- McpOauthProvider now round-trips the issuer stamp derived from the
  discovery snapshot: saveClientInformation/saveTokens persist it,
  clientInformation()/tokens() return it, and tokens() backfills a
  missing stamp for pre-existing credentials (removes the
  'stored OAuth credential has no issuer stamp' warning).
- Regression tests: savePendingAuth keeps the snapshot; the provider
  stamps/round-trips the issuer on both legs.

* fix(mcp): backfill SEP-2352 issuer stamp on stale OAuth client info

clientInformation() returned stored client info without an issuer field
when it had been dynamically registered before the stamping fix, so the
SDK logged 'stored OAuth credential has no issuer stamp' on every read.
Now the stored branch backfills the issuer from the discovery snapshot
(persisting it) exactly like the tokens() backfill.

* feat(webui): add MCP server configuration and agent assignment UI (#290)

* feat(webui): add MCP server configuration and agent assignment UI

* fix(webui): auto-detect auth for unconfigured MCP servers and show connect button

* fix(mcp): auto-generate state in mcp-oauth-provider when missing in DB

* fix(mcp): persist OAuth discovery snapshot and stamp SEP-2352 issuer

The callback leg of the OAuth flow failed with
AuthorizationServerMismatchError ('discoveryState was not available on
the callback leg') because McpDbStore.savePendingAuth overwrote the
entire pendingAuth JSON column, clobbering the discoverySnapshot that
the provider's saveDiscoveryState() had stored inside it.

- McpDbStore.savePendingAuth now preserves an in-flight discovery
  snapshot so the callback leg can satisfy the SEP-2352 authorization-
  server issuer check.
- McpOauthProvider now round-trips the issuer stamp derived from the
  discovery snapshot: saveClientInformation/saveTokens persist it,
  clientInformation()/tokens() return it, and tokens() backfills a
  missing stamp for pre-existing credentials (removes the
  'stored OAuth credential has no issuer stamp' warning).
- Regression tests: savePendingAuth keeps the snapshot; the provider
  stamps/round-trips the issuer on both legs.

* fix(mcp): backfill SEP-2352 issuer stamp on stale OAuth client info

clientInformation() returned stored client info without an issuer field
when it had been dynamically registered before the stamping fix, so the
SDK logged 'stored OAuth credential has no issuer stamp' on every read.
Now the stored branch backfills the issuer from the discovery snapshot
(persisting it) exactly like the tokens() backfill.

* fix(mcp): attach McpOauthProvider for auto-detected OAuth servers and discover tools on auth completion

* fix(webui): fix flex min-h-0 and max-h scrolling on MCP tools dialog

* fix(webui): set explicit flex height on McpToolsDialog for active scrolling

* feat(webui): make MCP server cards clickable to edit and widen server config dialog

* feat(mcp): agent-level server enablement and auto-detected server metadata (#291)

* feat(mcp): agent-level server enablement and auto-detected server metadata

- Remove the server-level `enabled` column from mcp_servers; enable/disable
  now happens per-agent via the AgentMcpServer join table (mirrors skills).
- Drop the unique (teamId, name) constraint so duplicate server names are allowed.
- Add a `description` column, auto-filled from client.getServerVersion() on
  connect; the server name is auto-filled from the reported title (fallback name)
  with a URL-derived placeholder until first connect.
- Remove manual name input: the create dialog is URL-only, and the edit dialog
  shows the auto-detected name/description read-only.
- Replace the include/exclude raw-text inputs with the tool list in the edit
  dialog, managing only excludeTools; remove includeTools support and the
  separate McpToolsDialog component.
- Sanitize server-reported names into safe direct-tool prefixes (spaces/punctuation
  become underscores) and use the same sanitization in the proxy resolver.

* fix(mcp): give the edit-dialog tool list a definite height

The Shadcn ScrollArea viewport is h-full, which requires a definite height
on the root. Using max-h-64 left the viewport at auto height, so the root
clipped the list at 256px without engaging its own scrollbar — tools past
the cutoff were unreachable. Use a fixed h-64 container (same pattern as
move-copy-dialog) so the inner scroll area scrolls independently of the
dialog's outer scroll.

* refactor(mcp): fixed header/footer with scrollable content in server config dialog

Mirror the skill config dialog layout: the edit dialog now keeps the header
and footer fixed and scrolls only the content area (ScrollArea flex-1
min-h-0 inside an h-[85vh] flex column). The tools list no longer has its
own nested ScrollArea — it flows inside the content-level scroll, so all
tools are reachable without a nested-scroll conflict. The create dialog
stays compact (URL-only) like the skill add dialog.

Verified headlessly that max-height-based ScrollArea roots do not engage
scrolling (viewport height resolves to auto), which is why the previous
nested list clipped; the flex-based definite-height pattern scrolls.

* feat(mcp): make the endpoint URL immutable after creation

Editing a server's URL effectively points it at a different server, and the
create flow is URL-only anyway — so the edit dialog no longer allows changing
it (delete + re-add covers endpoint changes). The URL is displayed read-only
next to the auto-detected name/description, and `url` was removed from the
update DTO and service; auth/transport changes still invalidate credentials
and cached tools.

* refactor(mcp): match MCP server cards to the skill card layout

The settings list cards now share the skill card structure: row 1 is the
server name (with direct-mode/transport badges) plus the three-dot menu,
row 2 is the description (falling back to 'No description provided') with
the endpoint URL and last error as small mono lines, and row 3 holds the
connection-status badge and updated date on the left with the permission
selector and the connect button at the right end.

* fix(mcp): stop card menu clicks from opening the edit dialog

The three-dot menu is portaled, so its item clicks bubble through React's
synthetic events to the card's onClick, which opened the edit dialog before
the delete confirmation appeared. Add e.stopPropagation() to every menu
item (matching the skill card pattern).

* fix(mcp): stop Refresh Tools from submitting/closing the config dialog

The Refresh Tools button lived inside the <form> and, like all buttons
without an explicit type, defaulted to type="submit" — clicking it ran the
form submit (save + close) instead of just refreshing. Set type="button".

Also per UX: while refreshing, the button is disabled with a spinning icon,
the per-tool enable/disable switches are disabled, and the success toast is
removed — the button reverting to its idle state is the success signal
(keep the error toast for failures).

* test(mcp): provide explicit redirectUri in McpOauthProvider test helper

* fix(mcp): scope tool registry to agent-assigned servers

The MCP tool registry was a process-global singleton keyed by server id,
so a chat agent could search, describe, and even call tools belonging to
MCP servers that were not assigned to it (e.g. disabled or unassigned
servers), as long as their metadata had been discovered at some point in
the process. This mirrors the disabled-skill bug: the read path never
checked per-agent enablement.

Replace the singleton with a fresh McpToolRegistry built per
buildAgentTools call and warmed from the persisted McpServer.tools DB
cache. Because the registry only ever contains the agent's assigned
servers, unassigned/disabled servers can neither be suggested, searched,
described, nor executed. Live discovery (lazy connect / connect action)
writes into the per-agent registry via ensureConnected(serverId,
registry?).

Adds test-first coverage: suggestions, search, describe and tool-call
leaks across an unassigned same-team server, staleness after
unassignment, and direct-tools scoping.

* feat(mcp): capture and surface server instructions + server-level refresh (#293)

* feat(mcp): capture and surface server instructions

- Add McpServer.instructions column, populated from getInstructions() at
  connect time (legacy initialize + modern discover results)
- Surface instructions at three levels like pi-mcp-adapter: truncated
  snippet in the mcp proxy tool description, preview in server listings,
  and full text via mcp({ instructions: "name" })
- Fix misleading mcp({ server, connect }) usage line to mcp({ connect })
- Share one renderServerListing helper across list and connect modes
- Expose instructions in McpServerInfo DTO and WebUI config card
- Test server supports instructions; service + API tests updated

* refactor(mcp): server-level refresh endpoint, drop agent connect mode

- Replace POST /mcp/servers/:id/tools/refresh with POST /mcp/servers/:id/refresh
  returning the refreshed McpServerInfo (server-level, not tools-level)
- Add McpService.refreshServer(): force-close the connection, re-discover and
  persist tools, instructions, name/description and status; used by the refresh
  endpoint, Test Connection, and post-OAuth completion (replacing discoverTools)
- Remove discoverTools entirely - refreshServer is the single discovery entry
  point (testServer and completeAuth route through it)
- Remove refreshTools from UpdateMcpServerRequest/PATCH (dead param)
- Remove mcp({ connect: "name" }) from the agent proxy tool: lazy tool calls
  still auto-connect, so agents are unaffected
- WebUI: remove the dialog 'Refresh Tools' button and create-mode background
  auto-refresh; add a server-level Refresh button at the left end of the edit
  dialog footer; i18n mcp_refresh_server (+ success message)

* feat(mcp): add in-flight request tracking and idle connection cleanup (#294)

* feat(mcp): add in-flight request tracking and idle connection cleanup

* feat(mcp): add background idle cleanup timer to McpService

* feat(mcp): update default background idle cleanup check interval to 30s

* feat(webui): improve MCP server card footer, instructions, and refresh dialog freeze (#295)

* feat(webui): improve MCP server list card, instructions dialog, and refresh freeze UI

* feat(webui): refine MCP card action menu items

* docs(webui): update direct tools mode description in i18n messages

* style(webui): refine Direct Tools Mode layout in MCP server config dialog

* feat(webui): remove tool search box from MCP server config dialog

* fix(mcp): make in-flight idle tracking test deterministic

The 'tracks in-flight requests and safely prunes idle connections' test
relied on a wall-clock sleep (10ms) to guess when the tool handler had
started executing on the server. On slow/loaded CI runners the HTTP round
trip through the real MCP client/server stack can exceed that window, so
resolveSlowTool() fired the no-op placeholder, the gated handler never
completed, and callPromise hung until the 5000ms test timeout.

The handler now signals via a deferred promise when it actually starts,
and the test awaits that signal (racing it against the call promise to
surface early failures) before asserting in-flight state and releasing
the gate. Also gives the test an explicit 10s timeout.
2026-08-11 11:01:38 +08:00
Yiling-J 1daeaf51ef fix(agent): prevent agents from accessing disabled skills (#292)
Disabling a skill in the agent config dialog only removed it from the
agent_skills join table, but the chat flow still advertised ALL team
skills to the model in the <available_skills> system prompt section and
the read_skill tool never checked per-agent enablement.

- getAgentChatContextActivity / getAgentAutofillContextActivity now only
  return skills explicitly enabled for the agent (agent_skills), and
  expose enabledSkillIds in the execution context.
- createReadSkillTool accepts enabledSkillIds and rejects read_skill
  calls for skills not enabled for the agent, even if the model knows
  the skill id from earlier conversation history.
- Added unit tests for context filtering (chat + autofill), tool-level
  enforcement, and the zero-enabled-skills case.
2026-08-10 17:33:40 +08:00
yiling 49ef48a74a Release v0.3.5 2026-08-09 20:58:35 +08:00
Yiling-J a680fccb12 fix(webui): enable rename and delete actions in file detail top bar breadcrumb (#288)
* fix(webui): enable rename and delete actions in file detail top bar breadcrumb

* fix(webui): select only base file name when renaming from file detail breadcrumb
2026-08-09 20:49:55 +08:00
Yiling-J e67552d8bd fix(webui): batch download no longer flashes a tab per file (#287)
startDownload() used <a target="_blank" download> anchors, which made
Chrome open a new tab for every file that then auto-closed as soon as the
download started (Content-Disposition: attachment). Replace with hidden
iframes: downloads start in the background without opening tabs, and a
failed/expired link renders inside the hidden iframe instead of
navigating the SPA tab away.
2026-08-09 17:09:01 +08:00
Yiling-J 8080acfb2e fix: use renamed asset name in download Content-Disposition (#286)
* fix(core): use renamed asset name in download Content-Disposition

Downloads of renamed assets (e.g. test.png renamed to foo.png) previously
saved under the storage-key basename because the presigned URLs / local
file handler only set Content-Disposition: attachment without a filename,
and the HTML download attribute is ignored for cross-origin URLs.

- Add buildContentDisposition() helper that emits a sanitized
  'attachment; filename=...; filename*=UTF-8...' header (RFC 5987) so
  non-ASCII names survive.
- Thread an optional filename through S3Service.presign():
  * S3 backend passes it via contentDisposition (response-content-disposition)
  * Local backend carries it as a ?filename= query param consumed by the
    /files serveStatic onFound handler
- getDownloadLinks / getDownloadUrl now pass the renamed asset name from
  the DB, so all download flows (file browser, viewers, breadcrumb, share)
  save files under the current asset name.

* fix(core): keep Content-Disposition header ASCII for non-ASCII names

Bun's Headers API rejects header values containing non-ASCII bytes, so a
download of a renamed file with a non-ASCII name (e.g. 幻境边界.pptx) threw
'TypeError: Header Content-Disposition has invalid value' on the local
storage backend.

Per RFC 5987, non-ASCII names belong only in the percent-encoded
filename* parameter. buildContentDisposition now emits the quoted
filename="..." fallback only when the name is pure ASCII; otherwise the
header carries just 'attachment; filename*=UTF-8''...' (all-ASCII, accepted
by Bun/Node Headers and all modern browsers).
2026-08-09 16:39:15 +08:00
Yiling-J dbd30a815f fix(webui): exclude file extension when selecting file name during inline rename (#285) 2026-08-09 16:06:44 +08:00
Yiling-J 442568de00 fix(webui): resolve upload card freeze for unsupported files (#284) 2026-08-09 15:16:41 +08:00
yiling d15fc5c6be update doc 2026-08-09 14:49:07 +08:00
Yiling-J aa6d5ed626 feat(transcode): add Gotenberg support for Office, HTML, CSV, and Markdown PDF proxies (#283)
* feat(transcode): add Gotenberg support for Office, HTML, CSV, and Markdown PDF proxies

* fix(transcode): use Gotenberg chromium markdown endpoint with index.html wrapper for markdown conversion

* test(transcode): add complete unit test coverage for HTML, CSV landscape/portrait, TXT, and Gotenberg fallback branches

* fix(gotenberg): force .md extension for Gotenberg markdown conversion endpoint compatibility

* refactor(gotenberg): standardize basic auth environment variables on GOTENBERG_BASIC_AUTH_USERNAME and GOTENBERG_BASIC_AUTH_PASSWORD

* test(e2e): add real Gotenberg testcontainers workflow E2E test for docx, pptx, and xlsx

* docs: add Gotenberg document PDF conversion documentation and update features in README

* docs(gotenberg): add Docker quickstart commands and remove obsolete GOTENBERG_USERNAME note
2026-08-09 14:44:13 +08:00
yiling 283d28bed9 Release v0.3.4 2026-08-08 20:15:25 +08:00
Yiling-J 315cacd474 Enhance README with new features and capabilities
Added new features including customizable watermark for shared links and support for various file formats.
2026-08-08 16:25:48 +08:00
Yiling-J 44ab1e7023 Add wide file format support to README 2026-08-08 16:19:09 +08:00
yiling beef4e9963 update readme 2026-08-08 16:06:25 +08:00
Yiling-J 990f68fb3b docs(agent): clarify create_file/create_version context param guidance (#282) 2026-08-08 15:41:22 +08:00
Yiling-J ee4933a4f8 feat(transcode): update preview thumbnail resolution to 300p with max length constraint (#281)
* feat(transcode): update preview thumbnail resolution to 300p with max length constraint

* fix(transcode): address code review feedback on preview thumbnail transcoding
2026-08-08 15:17:51 +08:00
Yiling-J c91867b8eb chore: split e2e scripts into app and webui targets (#280)
Split the monolithic test:e2e script into:

- test:e2e:app: fullstack web app e2e (playwright --project=app-chromium)
- test:e2e:webui: harness UI integration tests
  (E2E_SKIP_APP_SERVER=true --project=harness-chromium)

Remove the now-unused test:e2e:ui and test:e2e:report scripts, and update
the AGENTS.md submission rules and e2e README to reference the new
commands.
2026-08-08 13:20:52 +08:00
Yiling-J 8608ee027a test(e2e): stabilize watermark share spec on Firefox (#279)
Two issues made the watermarked share-link spec flaky on Firefox (and
silently mis-typed on every browser):

1. test.use inside a for loop leaks the last iteration's value to all
   tests: both the 'image' and 'video' cases were actually running the
   video flow. Scope each test.use inside a test.describe block.

2. Selecting the file by clicking the card body raced with the preview
   area re-rendering (video sprite scrubber hover) and the Radix menu
   exit animation, which could swallow the click on Firefox. Select via
   the card checkbox instead, and wait for the menu to fully unmount
   after Escape before the next interaction.

Verified stable: 6/6 on Firefox, 2/2 on Chromium, 2/2 on WebKit.
2026-08-08 13:11:44 +08:00
Yiling-J 394ddc8db4 docs(share-links): document allow download and lazy watermark settings (#278)
Document the two new share link security settings:

- Allow Download: hides all download affordances on the public pages and
  rejects the download endpoint with 403 while keeping previews available.
- Watermark: per-share-link lazy watermarking — transcodes are only
  triggered when the watermark is enabled on a share link (and again for
  files added later), so Shumai never stores watermark copies for files
  that are not shared through a watermarked link, saving storage.
2026-08-08 11:51:39 +08:00
Yiling-J 83455aeaea test(e2e): cover watermarked download-disabled share links for image and video (#277)
Add web app e2e coverage for the frequently-combined watermark +
download-disabled share link configuration. For both watermarkable media
types (image and video) the test asserts:

- no download affordance on the public share list page (card menu,
  selection bar) and the public file detail page (viewer control bar,
  top-nav file dropdown; video also has no resolution submenu)
- the proxy loaded in the detail page is the watermarked version
  (-watermark- S3 keys) and no original transcode key is requested
- the download-url endpoint rejects guest download attempts with 403

Extends the e2e share helpers with apiUpdateShare, apiEnableShareWatermark
and apiPublicDownloadUrl, and documents the new cases in the e2e README.
2026-08-08 11:45:06 +08:00
Yiling-J 707c091c9b fix(webui): persist project sorting and prevent full page flash (#276) 2026-08-08 10:37:45 +08:00
Yiling-J d3f5892dd2 feat(share): add allow download config to share links (#275)
* feat(share): add allow download config to share links

* fix(share): hide download in public share list page when disabled
2026-08-08 10:25:02 +08:00
Yiling-J 70b6c33204 fix(share): resolve symlinks for download URLs and enable card download action (#274) 2026-08-07 23:24:24 +08:00
Yiling-J 624a3a6a54 feat(webui): remove original-file display fallbacks, use proxies only (#273)
* feat(webui): remove original-file display fallbacks, use proxies only

WebUI now never uses the raw original file for display. All viewers
(image, video, compare, pdf) render exclusively from transcoded proxies;
when no proxy exists the viewer shows an unavailable placeholder instead
of falling back to media.original.

- getBestTranscode: drop the isRaw fallback candidates
- video-viewer / compare-video-pane: remove the synthetic 'Original'
  resolution entry and the original.downloadUrl hasMedia gate
- image-viewer / compare-image-pane: remove original.downloadUrl default
- pdf-viewer: require pdfTranscode.url (no original fallback)
- download menus (video-control-bar, breadcrumb-nav): use
  media.original.key for the 'Original' download option instead of the
  isRaw transcode marker entry

Backend:
- transcode-video workflow no longer pushes the isRaw:true marker entry
  into videoTranscodes (it pointed at the original file key)
- remove media.original.downloadUrl from the DTO, asset service mapping,
  MediaInfo DB type, and transcode workflows/activities (it was only
  consumed by the removed display fallbacks; downloads already use key
  via the download-url endpoints)

Tests/fixtures updated; docs updated to state the proxy-only display
policy.

* refactor: remove isRaw field via data migration, clean legacy media JSON

Add a Prisma data migration that cleans legacy media JSONB on both
assets and watermark_files:
- drop the raw-original marker entries (isRaw: true) from
  videoTranscodes / imageTranscodes — they pointed at the original file
  key and were only used by the removed display fallbacks / download
  menus, which now use media.original.key directly
- strip the isRaw field from remaining transcode entries
- drop original.downloadUrl (removed from the AssetInfo DTO)

The migration is generated via prisma migrate dev --create-only and
applied automatically by migrate deploy in production. It is guarded by
a test (packages/db/src/clean-media-migration.test.ts) that seeds legacy
media JSON, runs the migration SQL, and asserts the transform on both
tables.

With the data cleaned, remove the isRaw field entirely from the codebase:
- DTO schemas + ImageTranscode/VideoTranscode interfaces
- PrismaJson MediaInfo transcode types
- watermark activity isRaw skip (and its test), watermark proxy mapping,
  agent-embedding transcode selection, webui transcode filters, and all
  test fixtures

AGENTS.md: clarify that the no-manual-migration rule applies to DDL;
data migrations (pure SQL transforms, no schema change) are allowed
inside a --create-only generated migration and must be tested.
2026-08-07 22:40:06 +08:00
Yiling-J 03aea2cb36 feat(sharelink): add sharelink watermark support (#270)
* feat(sharelink): add sharelink watermark support

* fix(sharelink): harden watermark feature against races, crashes, and tenant leaks

Addresses review findings on the sharelink watermark feature:

- fix(watermark): read back sharelink via transaction client so the
  updateShareLinkWatermark response reflects the just-committed state
  (global-client reads on a pooled connection miss uncommitted changes)
- fix(watermark): use prisma upsert on the config hash instead of
  find-then-create to avoid P2002 races on concurrent enables
- fix(watermark): resolve nested symlinks when walking the share tree and
  batch the walk level-by-level instead of one query per node
- fix(watermark): keep watermark tasks in flight from being purged by
  orphan-config GC (avoids FK violations in completing activities)
- fix(transcode): re-transcode stale 'processing' watermark rows instead
  of silently skipping (crash recovery); downscale embedded block images;
  use pino structured logging; drop unused any casts
- fix(api): scope watermark-template list/create to the caller's teams,
  reject teamless template creation, and make teamless templates
  read-only; validate watermark config required when enabling (400);
  map sharelink-not-found to 404
- fix(api): resolve symlink target ids once in public share children
  endpoint instead of twice per item
- refactor(db): dedicated WatermarkFileStatus enum (migration included)
- test: activity unit tests (init crash recovery, image + video transcode
  paths), concurrent upsert, nested-symlink resolution, API scoping and
  validation, e2e video watermark path

* refactor(transcode): move sharp usage out of watermark activity into transcodeService

The transcode package does not declare sharp as a dependency (only
@shumai/core does), so importing it directly in the watermark activity
broke CI typecheck for the transcode package.

- add transcodeService.renderSvgToPng, downscaleImageToPng and
  compositeOverlayToWebpFile encapsulating the sharp logic
- watermark activity now calls these methods instead of importing sharp
- activity tests mock the new methods (no sharp/real fixture needed)
- add unit tests for the new transcodeService helpers

* feat(watermark): restrict watermark transcode to existing image and video transcodes

* refactor(transcode): reuse transcodeVideo in watermark activity

* refactor(watermark): use Bun.hash instead of node crypto for watermark config hash

* refactor(watermark): move computeWatermarkConfigHash from dtos to core watermark-hash module

* refactor(api): delegate watermark DB queries from public-share route to watermarkService

* refactor(share): replace dynamic watermarkService import with static import

* feat(watermark): add waitForWatermarkFileActivity for in-flight watermark transcoding tasks

* refactor(asset): replace dynamic watermarkService import with static import

* test(watermark): harden purgeOrphanWatermarkConfigs coverage

Replace the single happy-path GC test with a dedicated suite covering:

- orphan config with proxies: purged, watermarkFiles removed, all four
  proxy keys deleted from S3 while the original key is preserved
- config still referenced by a share link is not purged
- config with pending/processing transcode_watermark tasks is not purged
  (in-flight guard), but completed tasks do not block purging
- pending tasks of other types (transcode_video) do not protect a config
- an S3 delete failure is non-fatal: remaining keys still deleted and the
  config is still purged
- mixed run: only the orphan config is purged while a referenced one
  survives (exact count assertion)

* feat(webui): add sharelink watermark editor and sidebar controls

* style(webui): enlarge watermark editor dialog and allocate 2/3 width to preview column

* fix(webui): override sm:max-w-lg in DialogContent to allow full 1400px width

* fix(webui): stop block click propagation and remove preview mode toggles

* fix(webui): store watermark block size as fraction of canvas width

The editor stored block.size in pixels (slider 10-120px, preview
fontSize: `${block.size}px`) while the SVG generator
(generateWatermarkSvg) interprets size as a fraction (0..1) of canvas
width. A 36px block therefore rendered as a ~36x-canvas glyph covering
the entire image.

- Keep backend fraction semantics (canvasWidth * size) so watermarks
  scale proportionally with output resolution across proxies
- Editor: defaults (text 5%, image 15%), slider ranges (1-12% / 2-50%)
  and number input now work in percent, storing a fraction
- Preview: scales blocks by the measured canvas width so WYSIWYG
- DTO schema: size constrained to (0..1) to enforce the contract
- Update DTO/editor tests to fraction sizes

* feat(webui): add single-row aspect ratio selector and 5 predefined background colors to watermark editor preview

* feat(webui): update aspect ratio to multi-toggle group and enable dot-grid pattern for all background colors

* fix(webui): update background swatches to distinct colors and share PRESET_BACKGROUND_OPTIONS array

* fix(webui): hide redundant absolute close button on watermark editor dialog header

* fix(transcode): skip raw video transcode when generating watermark proxies

The raw marker entry (isRaw: true) in videoTranscodes points at the
original source file and is only used to offer the 'Original' playback
option. It is not a transcoded proxy, so watermarking it produced a
redundant full-resolution transcode (the most expensive pass) plus an
unneeded S3 upload.

Skip entries with isRaw: true in transcodeWatermarkMediaActivity, and
cover the behavior with a unit regression test plus an e2e assertion
that exactly one watermark proxy is produced for a video asset.

* feat(webui): replace watermark image asset ID with 1MB file uploader

* fix(webui): correct watermark editor preview block sizing

The preview canvas width was measured in a useEffect([open]) that ran
before the Radix Dialog had mounted its content, so canvasRef.current was
still null, canvasWidthPx stayed 0, and every block silently fell back to
the hardcoded 640px width (block.size * (canvasWidthPx || 640)). A 100%
size therefore rendered at ~640px on an ~876px canvas (~72%), making the
preview smaller than the real watermarked output for both image and text
blocks.

- Measure the canvas via a ref callback (fires exactly when the node
  mounts, regardless of dialog mount timing) and keep a ResizeObserver
  attached for aspect-ratio/window changes.
- Use clientWidth (padding-box width) instead of getBoundingClientRect(),
  which includes the dialog's zoom-in-95 animation transform.
- Drop max-w-full/max-h-full on the preview <img> so logos scale up to
  the block size like the backend SVG rendering does, instead of being
  capped at their natural pixel size.

* fix(webui): keep watermark editor preview aspect ratio and restore config scroll

- Compute the preview canvas size with a JS contain-fit (ResizeObserver on the
  frame) instead of relying on CSS aspect-ratio + w-full/max-h-full, which
  broke the selected ratio on wide-and-short (or tall-and-narrow) windows.
- Add min-h-0 to the config ScrollArea so it shrinks to the column's remaining
  height instead of growing to its content height, restoring the scroll range.

* fix(webui): remove toast notification on template load

* fix(share): serve per-file watermark proxies in public share APIs

Stop gating watermark replacement on the share-link-wide watermarkStatus.
Public share endpoints now resolve each asset's own completed watermark
proxy and:

- replace videoTranscodes/imageTranscodes with the watermarked entries
  when the watermark file is completed;
- empty those arrays when no completed proxy exists yet (still
  transcoding or failed) so original, unwatermarked media is never
  exposed while a newly added file is being processed;
- leave non-video/image media (pdf, audio, ...) untouched, since they
  never receive watermark transcodes.

This removes the previous behavior where adding a file to a ready
watermark share flipped the whole share link to processing and served
original media for every file until the new transcode finished.

Adds API tests for detail/children endpoints (ready, in-flight, pdf, and
watermark-disabled cases) and a workflow E2E test for adding an asset to
an already-ready watermark sharelink.
2026-08-07 21:36:20 +08:00
yiling 0a31659588 Release v0.3.3 2026-08-05 19:41:02 +08:00
Yiling-J 39a66dc569 Merge pull request #269 from shumaiOne/disable-project-member-team-role-dropdown
feat(webui): disable role dropdown for project-scoped members in team view
2026-08-05 19:06:28 +08:00
yiling 2a27e19b15 style(webui): expand members dialog max width to sm:max-w-xl 2026-08-05 18:54:51 +08:00
yiling f306229b21 style(webui): increase members dialog max width 2026-08-05 18:53:58 +08:00
yiling f5786f742b feat(webui): disable role dropdown for project-scoped members in team view 2026-08-05 18:50:55 +08:00
Yiling-J fe7e541585 Merge pull request #268 from shumaiOne/feat/e2e-member-permissions
test(e2e): add member permission change tests
2026-08-05 18:39:09 +08:00
yiling 47ecbdff6a fix(e2e): resolve lint rule for Playwright baseURL option 2026-08-05 18:33:10 +08:00
yiling 2dc0f08725 chore(e2e): default to chromium locally, run all browsers on CI 2026-08-05 18:28:46 +08:00
yiling e76ed19013 test(e2e): add member permission change tests 2026-08-05 17:51:27 +08:00
Yiling-J b7d3e32fb9 Merge pull request #267 from shumaiOne/feat/project-members-team-scope
feat(project): include team-scoped members in project member list and add 3-section dialog layout
2026-08-05 17:23:09 +08:00
yiling 94bda76c56 refactor(project): remove unnecessary explicit any cast in updateMemberRole 2026-08-05 17:14:00 +08:00
yiling 7586b8a44a feat(project): display flat member list for non-owners in members dialog 2026-08-05 16:52:16 +08:00
yiling d9a6734527 feat(project): include team-scoped members in project member list and add 3-section dialog layout 2026-08-05 16:39:19 +08:00
yiling 72aad56560 feat(project): include team-scoped members in project member list and refine dialog copy 2026-08-05 16:15:27 +08:00
Yiling-J ffeccaa514 Merge pull request #266 from shumaiOne/fix/field-reorder-dnd-kit
fix(webui): persist field order on drag and drop
2026-08-05 16:01:55 +08:00
yiling d73f7f81a1 fix(webui): persist field order on drag and drop
The handleDragEnd handlers in the Manage Fields dialog and Fields
dropdown guarded reordering on source.id !== target.id. With dnd-kit
react's OptimisticSortingPlugin (enabled by default), the drop target
always resolves to the dragged item itself, so the guard was never true
and the order API was never called.

Detect moves via the sortable-specific index/initialIndex properties on
the source instead, and reorder the rendered (grouped/search-filtered)
subset before merging back into the full field list. All field groups are
now reorderable in the dialog.
2026-08-05 15:52:58 +08:00
Yiling-J 1104ec36cc Merge pull request #265 from shumaiOne/fix/custom-field-options-and-placeholder
feat(webui): render select options in field creation mode and pick random option color
2026-08-05 15:28:56 +08:00
yiling ac5970c03e docs(webui): update field description placeholder text 2026-08-05 14:50:52 +08:00
yiling b568eac77d feat(webui): render select options in field creation mode and pick random option color 2026-08-05 14:44:30 +08:00
yiling bb9c53adad feat(agent): clarify create_file tool usage and context hint instructions 2026-08-05 12:01:04 +08:00
Yiling-J 29161f3a1a Merge pull request #264 from shumaiOne/feat/autofill-context
feat(agent): propagate autofill context from chat to autofill workflow
2026-08-05 11:25:04 +08:00
yiling b806b0e2ba refactor(agent): drop field/option ids from list_autofill_fields results 2026-08-05 11:02:25 +08:00
yiling 033652cde3 feat(agent): propagate autofill context from chat to autofill workflow
Add an optional 'context' parameter to the create_file and create_version
agent tools so the chat agent can pass generation/source context (e.g.
'Generated using gemini') to the AI metadata autofill workflow.

- Add Asset.autofillContext column, persisted in the same transaction as
  asset creation when context is provided
- Add new list_autofill_fields agent tool exposing autofillable metadata
  fields (name, type, description, options) per project
- Autofill workflow reads the stored context and includes it in the
  autofill AI prompt
- Add short chat system-prompt guidance to use list_autofill_fields and
  pass <=50 word context when creating files/versions
2026-08-05 10:33:02 +08:00
Yiling-J 30021b07d0 Merge pull request #263 from shumaiOne/feat/agent-create-file-from-content
feat(agent): allow create_file tool to create files directly from content
2026-08-04 21:50:19 +08:00
yiling 07ad0d2f65 feat(agent): allow create_file tool to create files directly from content
Add an optional 'data' field ({name, content}) to the create_file agent tool
so agents can create files without a local file (e.g. when bash is not
available). The existing 'path' upload mode is preserved; exactly one of
'path' or 'data' must be provided.

Extract mime detection helpers (getFileMimeType, readFileMimeType) into a
new file-mime module, keeping utils/mime.ts free of fs/path imports so it
stays compatible with the Temporal workflow bundle (agent-autofill imports
getProxyType from it). Refactor create_version to reuse the extracted
helpers.

Content uploads reuse s3Service.putObject; mime type is derived from the
filename extension with a text/plain fallback, and the name is sanitized
for the S3 key.
2026-08-04 21:16:33 +08:00
Yiling-J 5ef763fbae Merge pull request #262 from shumaiOne/feat/agent-bash-restriction-for-non-owners
feat(agent): restrict bash access for non-owner users
2026-08-04 18:21:22 +08:00
yiling e6fde25446 fix(agent): activate bash tool after skill load and fix chat stream
Two bugs found while testing skill loading as a non-owner:

1. The injected bash tool was never actually usable: AgentHarness.setTools
   only updates the tool registry unless activeToolNames is passed, so bash
   was missing from the active tool set sent to the model on the next turn.
   Pass the full post-update tool list as activeToolNames.

2. The active_tools_change session entry (written by setTools) has no chat
   representation, so mapEntryToMessage fell back to fabricating an empty
   user message that appeared as a stray empty bubble in the chat stream.
   Render active_tools_change as a typed custom message and skip entry
   types that have no chat representation instead of emitting empty text.
2026-08-04 17:01:47 +08:00
yiling dd2f8e68d9 fix(agent): fail closed when user role cannot be resolved
A userId that is not a team member previously resolved to isOwner=true
(role === undefined || role === 'owner'), bypassing bash restrictions.
Treat only flows without a user context (autofill etc.) and explicit
owner memberships as unrestricted; unknown users are now restricted.
2026-08-04 16:26:16 +08:00
yiling c938e84ccc feat(agent): restrict bash access for non-owner users
Editor/reviewer users can no longer ask the agent to run arbitrary bash
commands. Bash execution for them is gated behind approved team skills:

- Add required source param (enum user/skill) to the bash tool; commands
  with source=user are blocked when the user is not a team owner.
- Append a Restricted User Context instruction to the system prompt for
  non-owner users: never run user-requested bash, only skill-required
  commands with source=skill.
- Delay bash tool injection for editor/reviewer: it is only added after
  a skill is successfully loaded via read_skill (or restored when a
  skill was loaded earlier in the session). Team owners are unaffected.
2026-08-04 16:08:04 +08:00
Yiling-J 9745985f57 Merge pull request #261 from shumaiOne/feat/e2e-search-file-list
test(e2e): add search in file list and save-search-as-collection cases
2026-08-04 15:11:37 +08:00
yiling b645a29608 test(e2e): add search in file list and save-search-as-collection cases 2026-08-04 14:52:24 +08:00
Yiling-J deeb02351f Merge pull request #260 from shumaiOne/feat/e2e-file-domain-comments
feat(e2e): add file domain comment and draw tests
2026-08-04 11:25:01 +08:00
yiling c53919b891 ci: add system dependencies installation to app E2E CI job 2026-08-04 11:17:16 +08:00
yiling edf1f6a327 feat(e2e): add file domain comment and draw tests 2026-08-04 10:59:31 +08:00
Yiling-J 1afdbb671b Merge pull request #259 from shumaiOne/feat/project-e2e-tests
test(e2e): add project-level invite and file-move e2e tests
2026-08-03 18:33:48 +08:00
yiling 134d19d907 test(e2e): add project-level invite and file-move e2e tests
Add four project-level webapp e2e tests:
- invite a member to a project; the invitee only sees the invited project
- drag a file into a folder and a folder into another folder
- drag a file onto a file to create a version stack
- drag a file onto a version stack to add a new version

Supporting changes:
- expose rootFolderId on the project fixture
- add seedFolder/seedFile helpers and a cross-browser dragCard helper
  (WebKit requires pointer moves in awaited batches for dnd-kit activation)
- add project-members-trigger testid to the file-browser toolbar
- document the new test cases in the e2e README
2026-08-03 18:04:19 +08:00
yiling 043cde9fcc update agentsmd 2026-08-03 17:09:46 +08:00
Yiling-J 472ffddc99 Merge pull request #258 from shumaiOne/feat/web-e2e-file-share-collection
test(e2e): add file, share, and collection web app e2e tests
2026-08-03 17:07:08 +08:00
yiling 78eb840bf5 test(e2e): upload a 5px png instead of txt to avoid slow transcode 2026-08-03 16:45:38 +08:00
yiling 077536de98 test(e2e): move invite-member spec to the team domain 2026-08-03 16:39:51 +08:00
yiling 3789366d35 test(e2e): add file, share, and collection web app e2e tests
Add six domain-level web app e2e tests covering team member removal,
file upload/delete/restore, share link creation and password protection,
and collection creation. Introduce a parametrized 'file' fixture with
configurable media types (text/binary/image/video/pdf) for future
type-scoped tests, plus helpers for share APIs and team member seeding.

Also document all web app e2e test cases in apps/web/e2e/README.md and
require this documentation in AGENTS.md.
2026-08-03 16:34:37 +08:00
Yiling-J 5647231d7f Merge pull request #257 from shumaiOne/feat/e2e-domain-tests
test(e2e): add domain-based auth and project web app e2e tests
2026-08-03 15:42:32 +08:00
yiling b2a11c5ce9 docs: add Web App E2E Tests section to developer guide 2026-08-03 15:32:38 +08:00
yiling 4532deb753 test(e2e): add domain-based auth and project web app e2e tests 2026-08-03 15:24:21 +08:00
Yiling-J 6efa634f54 Merge pull request #256 from shumaiOne/fix/mention-popup-avatar-and-scroll
fix(webui): mention popup avatars, keyboard auto-scroll, and non-sticky section headers
2026-08-03 14:56:20 +08:00
yiling d696c39622 fix(webui): stop mention popup section headers from sticking and overlapping
Both Agents and Members header buttons had 'sticky top-0', so while
scrolling the 256px mention list the Agents header stayed pinned and the
Members header snapped to the same top position, covering it exactly.
Remove 'sticky top-0' so headers scroll away naturally with their
sections.
2026-08-03 14:47:43 +08:00
yiling 83d22d47a2 fix(webui): show member avatars and auto-scroll in mention popup
- Render real avatar images (with initials fallback) for users in the
  @-mention popup instead of a colored letter circle; bots keep the
  colored fallback since BotInfo has no image field.
- Auto-scroll the highlighted mention row into view when navigating
  the list with ArrowUp/ArrowDown keys.
2026-08-03 14:38:40 +08:00
Yiling-J 4436c3304d Merge pull request #255 from shumaiOne/refactor/use-pi-builtin-providers
refactor(core): replace generated provider files with pi-ai dynamic methods
2026-08-03 11:43:38 +08:00
yiling 2fee708863 feat(agent): implement DatabaseSessionStorage getSessionName and getSessionStats methods 2026-08-03 10:40:33 +08:00
yiling dbe3420b6c feat(agent): update pi packages to 0.82.1 and implement new SessionStorage interface methods 2026-08-03 10:25:10 +08:00
yiling 16d136b45a refactor(core): replace generated provider files with pi-ai dynamic methods 2026-08-03 10:00:21 +08:00
Yiling-J 2656a42c86 Merge pull request #254 from shumaiOne/feat/agent-preference-localstorage
feat(webui): persist agent preference in localStorage
2026-08-02 15:31:00 +08:00
yiling 15910a0bf4 feat(webui): persist agent preference in localStorage 2026-08-02 15:06:51 +08:00
Yiling-J 5458189c1f Merge pull request #253 from shumaiOne/refactor/replace-deprecated-get-model
refactor(agent): load models directly from database without pi fallback
2026-08-02 14:36:54 +08:00
yiling d3aef5f0c1 fix(agent): add required api field to test mock provider config 2026-08-02 14:30:28 +08:00
yiling 5946a679ea refactor(agent): load models directly from database without pi fallback 2026-08-02 14:26:03 +08:00
yiling a54c04e15a update providers 2026-08-01 21:38:42 +08:00
Yiling-J 0bccd13399 Merge pull request #252 from shumaiOne/test/add-missing-core-service-tests
test(core): add missing tests for methods introduced in 97069042
2026-08-01 20:58:15 +08:00
yiling 3b86d9b6d8 test(core): add missing tests for methods introduced in 97069042
Add comprehensive test coverage for 5 core service methods that were
added in the layered architecture refactor but lacked tests:

- AgentService.getAgent: found/not-found cases
- ProjectService.getUserProjects: team scope, project scope,
  multi-team aggregation, limit, and empty result cases
- SkillService.getSkill: full field mapping and not-found cases
- TeamService.hasWritableRoleInAnyTeam: owner, editor, reviewer,
  and no-membership cases
- UserService (new file): getUserById and createGuestUser
2026-08-01 20:44:34 +08:00
Yiling-J 1600949e4a Merge pull request #251 from shumaiOne/feat/audit-log-support
feat(audit): add audit log support for APIs and WebUI dashboard
2026-08-01 19:19:20 +08:00
yiling 2f5b968935 fix(api): fix eslint any type assertion in file.test.ts 2026-08-01 16:35:57 +08:00
yiling 97069042b2 refactor(api): enforce layered architecture by moving db queries to core services 2026-08-01 16:25:01 +08:00
yiling fb66166b49 feat(core): wrap audit log creation in try catch to prevent blocking APIs 2026-08-01 15:50:35 +08:00
yiling b0d8ecb589 refactor(dtos): define AuditAction enum in dtos package to decouple from db 2026-08-01 15:28:23 +08:00
yiling 2d61fae699 refactor(db): consolidate audit log migration into single add_audit_log migration 2026-08-01 15:19:53 +08:00
yiling 5ef7716c9c feat(audit): change audit log action to string at db level and split file/folder actions 2026-08-01 14:49:08 +08:00
yiling 6d80f06c22 refactor(core): update getAssetContext to throw error on missing asset 2026-08-01 11:21:56 +08:00
yiling 0324b95c7e feat(api): add missing audit log calls to API endpoints 2026-08-01 11:03:50 +08:00
yiling 7dfb2c7f2f refactor(dtos): use @shumai/db/enums across DTO definitions 2026-08-01 10:47:41 +08:00
yiling 11fc76e56b refactor(dtos): import AuditAction from @shumai/db/enums subpath 2026-08-01 10:45:10 +08:00
yiling 52f85fcd15 fix(dtos): decouple AuditAction from @shumai/db to prevent leaking DB dependencies to WebUI 2026-08-01 10:41:57 +08:00
yiling 118107ebc9 feat(audit): add audit log support for APIs and WebUI dashboard 2026-08-01 10:39:31 +08:00
Yiling-J 7a83f7d406 Merge pull request #250 from shumaiOne/feat/agent-harness-retry
feat(agent): enable LLM API call retries in AgentHarness
2026-08-01 09:57:22 +08:00
yiling c2d25bd2f1 feat(agent): enable LLM API call retries in AgentHarness 2026-08-01 09:49:43 +08:00
1543 changed files with 135457 additions and 27646 deletions
+15 -10
View File
@@ -22,9 +22,9 @@ jobs:
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
bun-version: 1.3.14
- name: Install dependencies
run: bun install
@@ -76,9 +76,9 @@ jobs:
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
bun-version: 1.3.14
- name: Install dependencies
run: bun install
@@ -89,7 +89,7 @@ jobs:
- name: Run video player harness tests
env:
E2E_SKIP_APP_SERVER: true
run: bun run test:e2e --project=harness-${{ matrix.project }}
run: bunx playwright test --project=harness-${{ matrix.project }}
- name: Upload Playwright report
if: ${{ !cancelled() }}
@@ -118,18 +118,23 @@ jobs:
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
bun-version: 1.3.14
- name: Install dependencies
run: bun install
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y ripgrep bubblewrap socat ffmpeg poppler-utils imagemagick
- name: Install Playwright browser (${{ matrix.project }})
run: bunx playwright install --with-deps ${{ matrix.project }}
- name: Run app E2E tests
run: bun run test:e2e --project=app-${{ matrix.project }} --no-deps
run: bunx playwright test --project=app-${{ matrix.project }} --no-deps
- name: Upload Playwright report
if: ${{ !cancelled() }}
@@ -147,9 +152,9 @@ jobs:
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
bun-version: 1.3.14
- name: Install dependencies
run: bun install
+1 -1
View File
@@ -36,7 +36,7 @@ jobs:
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
bun-version: 1.3.14
- name: Setup Node.js
uses: actions/setup-node@v6
+1
View File
@@ -54,5 +54,6 @@ jobs:
platforms: linux/amd64,linux/arm64
context: .
push: true
provenance: false
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
+1
View File
@@ -11,6 +11,7 @@ data/
*.db
*.diff
*.orig
code_quality.md
.agents
# Playwright
+1
View File
@@ -3,6 +3,7 @@ dist
.bun
src/generated
packages/db/src/generated
packages/core/src/generated
src/ui/components/ui
packages/webui/paraglide
*.md
+11 -2
View File
@@ -90,7 +90,8 @@ Add any additional context, caveats, or follow-up work.
- `bun run format`
- `bun run typecheck`
- `bun run test`
- `bun run test:e2e`
- `bun run test:e2e:app`
- `bun run test:e2e:webui`
- `bun run test:e2e:workflow`
- **Backend Testing Mandate**: Every backend feature, service method, workflow, and activity MUST be accompanied by comprehensive tests. Logic-heavy code without corresponding test coverage is considered incomplete.
@@ -282,6 +283,13 @@ describe('Team API', () => {
- **Mocking**: Mock ONLY AI API calls. Do not mock S3, databases, or media/transcode extraction services.
- Mock the AI response by spying on `AgentHarness.prototype.prompt`:
### Web App E2E Tests
- Located in `apps/web/e2e/**/*.spec.ts`, organized by domain under `tests/<domain>/` (e.g. `auth`, `project`).
- Run via `bun run test:e2e:app`.
- **Fixtures**: Use the `owner`, `project`, and `file` fixtures for setup — they seed data through API calls/DB (no slow UI setup). You can add more fixtures if necessary.
- **Documentation**: Every test case MUST be documented in [`apps/web/e2e/README.md`](apps/web/e2e/README.md). Add or update the corresponding entry in the README whenever you add, rename, or remove a spec.
## Prisma Configuration & Migrations
- Schema: `packages/db/prisma/schema.prisma`
@@ -314,7 +322,8 @@ We use `prisma-json-types-generator` to enforce strict type-safety for Prisma `J
- Development: Use `bun --bun run prisma migrate dev` to create and apply migrations during development.
- Production: Use `bun --bun run prisma migrate deploy` to apply pending migrations in production environments.
- **No Manual Migration Creation**: Never create migration SQL files or directories manually by hand. Always use Prisma CLI commands (e.g. `bun --bun run prisma migrate dev --create-only` to generate a migration template, or `bun --bun run prisma migrate dev`) so Prisma correctly tracks migration metadata and checksums.
- **No Manual DDL Creation**: Never hand-write **schema (DDL) migration** files or directories. Always generate them with Prisma CLI (`bun --bun run prisma migrate dev --create-only` to create a template, then edit it if needed, or `bun --bun run prisma migrate dev` to apply) so Prisma correctly tracks migration metadata and checksums.
- **Data Migrations Are Allowed**: Pure **data migration** SQL (no schema change) may be written by hand, but it MUST live inside a migration generated via `bun --bun run prisma migrate dev --create-only` so it is tracked and applied automatically.
- **No Automatic Dev DB Reset**: Do not run `prisma migrate reset --force` or commands that force-reset the database automatically. If migrations become out of sync or a reset is required, stop executing, report the situation to the user, and present suggested manual cleanup/reset steps for the user to execute.
### Commands
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+8 -2
View File
@@ -1,6 +1,6 @@
#syntax=docker/dockerfile:1.7-labs
# Stage 1: Builder
FROM oven/bun:1 AS builder
FROM oven/bun:1.3.14 AS builder
WORKDIR /app
@@ -8,6 +8,7 @@ WORKDIR /app
COPY package.json bun.lock ./
COPY --parents packages/*/package.json ./
COPY --parents apps/*/package.json ./
COPY patches/ ./patches/
# Install dependencies (including devDependencies for build tools)
RUN bun install --frozen-lockfile
@@ -24,7 +25,7 @@ RUN bun run build.ts
RUN bun run scripts/extract-runtime-deps.ts
# Stage 2: Runner
FROM oven/bun:1 AS runner
FROM oven/bun:1.3.14 AS runner
# Install ffmpeg/ffprobe
COPY --from=mwader/static-ffmpeg:8.1 /ffmpeg /usr/local/bin/
@@ -79,6 +80,11 @@ USER root
COPY scripts/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
# Expose "bun index.js" as "shumai" CLI command (and alias "shuami")
RUN printf '#!/bin/sh\nexec bun /app/index.js "$@"\n' > /usr/local/bin/shumai \
&& chmod +x /usr/local/bin/shumai \
&& ln -s /usr/local/bin/shumai /usr/local/bin/shuami
ENTRYPOINT ["docker-entrypoint.sh"]
# Default command: run migrations and start API server
+22 -2
View File
@@ -9,7 +9,13 @@
</p>
<p align="center">
<a target="_blank" href="https://shumai.one/">Website</a> | <a target="_blank" href="https://docs.shumai.one/introduction">Docs</a> | <a target="_blank" href="https://staging.shumai.one">Demo</a> | <a target="_blank" href="https://discord.gg/sUzhujK6DD">Discord</a> | <a target="_blank" href="README_CN.md">中文</a>
<a href="https://hub.docker.com/r/shumaione/shumai"><img alt="Docker pulls" src="https://img.shields.io/docker/pulls/shumaione/shumai" /></a>
<a href="https://github.com/shumaiOne/shumai/releases"><img alt="GitHub latest release" src="https://img.shields.io/github/v/release/shumaiOne/shumai" /></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-MIT-blue" /></a>
</p>
<p align="center">
<a target="_blank" href="https://shumai.one/">Website</a> | <a target="_blank" href="https://docs.shumai.one/getting-started/overview">Docs</a> | <a target="_blank" href="https://staging.shumai.one">Demo</a> | <a target="_blank" href="https://discord.gg/sUzhujK6DD">Discord</a> | <a target="_blank" href="README_CN.md">中文</a>
</p>
![Shumai App Screenshot](docs/screenshot.webp)
@@ -24,15 +30,20 @@ Shumai is an open-source Frame.io alternative built for creative teams to collab
* **Granular Access Control**: Manage workspace permissions with team-level and project-level role-based access control.
* **Distributed Transcoding via Temporal**: Offload resource-intensive video transcoding to a background worker pool orchestrated by Temporal.
* **Custom Asset Metadata**: Define dynamic metadata fields tailored to your production pipeline.
* **Share Link Watermarking**: Protect shared media with customizable text and image watermarks, and save watermark as reusable templates.
* **Wide File Format Support**: Preview common creative file formats, including images (with PSD support), videos, audio, and documents, while allowing any file type to be uploaded and stored.
* **Office & Document PDF Proxying via Gotenberg**: Convert Word, PowerPoint, Excel, Markdown, HTML, and CSV documents to high-quality PDF proxies for in-browser previewing using [Gotenberg](https://docs.shumai.one/using-shumai/gotenberg).
But Shumai goes beyond being a Frame.io alternative. With modern models like GPT-Image-2 and Seedance 2.5, AI can already handle a surprising amount of creative work. That means you can create or update videos and images from a simple prompt without having to jump back into Photoshop, Premiere or Houdini. That is why Shumai includes a powerful agent system:
* **Agent as a Team Member**: Mention a context-aware AI agent with `@mentions` directly in asset comments, just like a real teammate.
* **Agent as a One-to-One Assistant**: Chat privately with the agent and ask it to help with tasks.
* **Custom Skills & Tools**: Extend the agent by registering custom scripts, tools, and automation skills.
* **Model Context Protocol (MCP)**: Connect agents to external tools and APIs through remote MCP servers, with OAuth authentication and per-agent MCP access.
* **Isolated Sandbox Execution**: Run agent-submitted scripts safely in a sandboxed environment.
* **AI Metadata Autofill**: Automatically fill in custom metadata for new assets.
* **Semantic Search**: Find assets by visual or conceptual meaning using vector embeddings (Gemini Embedding 2).
* **Resource Quotas**: Set usage limits on AI tokens, costs, and tool execution per member, role, or team-wide, with real-time tracking.
Here is a short demo video showing what the Shumai agent can do:
@@ -40,10 +51,16 @@ Here is a short demo video showing what the Shumai agent can do:
https://github.com/user-attachments/assets/1186067a-72c0-41b9-836f-987438f2332a
Shumai also brings project management into the creative workflow. A built-in Kanban board helps teams organize tasks, track progress, and collaborate in one place.
The ultimate goal is an **agentic Kanban**, where agents and team members can work together on tasks. This is still a **work in progress**: you can already create tasks for AI agents, but agents cannot act on those tasks yet.
<img width="2880" height="2160" alt="localhost_3000_teams_01M0HQR3EEBS0B46K4FK39X2C8_kanban(1080p-2) (1)" src="https://github.com/user-attachments/assets/cd0648d0-c69a-4398-958b-8d15a33378b0" />
> [!NOTE]
> **Media Processing & Storage:** When uploading media, Shumai always preserves and stores your original raw file. Note that the file size shown in the system represents the original file size and does not include the size of any transcoded files. For all media types (both images and videos), Shumai creates a 480p WebP poster image to serve as the preview in the file list view. Additionally:
> **Media Processing & Storage:** When uploading media, Shumai always preserves and stores your original raw file. Note that the file size shown in the system represents the original file size and does not include the size of any transcoded files. For all media types (both images and videos), Shumai creates a 300p WebP poster image to serve as the preview in the file list view. Additionally:
> * **For Images:** Shumai generates a high-quality WebP proxy at the exact same resolution as your original file for optimized viewing.
> * **For Videos:** It generates at least one proxy video (depending on your transcoding settings), alongside a sprite image and a small preview video (capped at 1600 frames) to enable fast timeline scrubbing and lightning-quick previews.
@@ -242,11 +259,14 @@ BETTER_AUTH_SECRET=ySxs7DxzHDZBbeeHNPEwBuspYwipBqz5Gk5XdBjNhWw=
STORAGE_BACKEND=local
SHUMAI_SERVER_PORT=3000
AWS_ENDPOINT_URL_S3=http://localhost:3000
# Optional: base URL for MCP OAuth redirects (defaults to BETTER_AUTH_URL)
MCP_OAUTH_REDIRECT_BASE_URL=http://localhost:3000
```
> [!NOTE]
> `SHUMAI_SERVER_PORT` sets the port the server starts on, while `AWS_ENDPOINT_URL_S3` is used by the browser to build file upload URLs.
> If deploying on a remote server (e.g. `http://123.456.7.8`) with a mapped port (e.g. `12345:3000` in docker-compose), set `AWS_ENDPOINT_URL_S3` to `http://123.456.7.8:12345`.
> `MCP_OAUTH_REDIRECT_BASE_URL` is only needed when MCP servers use OAuth: the authorization redirect URI defaults to `{BETTER_AUTH_URL}/api/mcp/oauth/callback`; set it to an externally reachable URL if the app is served behind a proxy.
#### Step 6: Run Shumai
+37 -2
View File
@@ -9,8 +9,13 @@
</p>
<p align="center">
<a target="_blank" href="https://shumai.one/">官方网站</a> | <a target="_blank" href="https://docs.shumai.one/introduction">文档</a> | <a target="_blank" href="https://staging.shumai.one">在线演示</a> | <a target="_blank" href="README.md">English</a>
<a href="https://hub.docker.com/r/shumaione/shumai"><img alt="Docker pulls" src="https://img.shields.io/docker/pulls/shumaione/shumai" /></a>
<a href="https://github.com/shumaiOne/shumai/releases"><img alt="GitHub latest release" src="https://img.shields.io/github/v/release/shumaiOne/shumai" /></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-MIT-blue" /></a>
</p>
<p align="center">
<a target="_blank" href="https://shumai.one/">官方网站</a> | <a target="_blank" href="https://docs.shumai.one/getting-started/overview">文档</a> | <a target="_blank" href="https://staging.shumai.one">在线演示</a> | <a target="_blank" href="README.md">English</a>
</p>
![Shumai 应用截图](docs/screenshot.webp)
@@ -25,22 +30,34 @@ Shumai 是一个开源的 Frame.io 替代方案,专为创意团队打造,让
* **细粒度权限控制**:支持按团队、项目设置角色权限,方便管理不同成员的访问范围。
* **基于 Temporal 的分布式转码**:把吃资源的视频转码任务交给后台 worker 池处理,由 Temporal 统一调度。
* **自定义素材元数据**:可以按你的制作流程,自由定义动态元数据字段。
* **分享链接水印**:通过可自定义的文字和图片水印保护共享媒体,并将水印配置保存为模板以便重复使用。
* **丰富的文件格式支持**:支持预览常见创意文件格式,包括图片(支持 PSD)、视频、音频和文档,同时可上传和存储任意类型的文件。
* **基于 Gotenberg 的文档 PDF 转码代理**:通过集成 [Gotenberg](https://docs.shumai.one/using-shumai/gotenberg) 支持将 Word、PowerPoint、Excel、Markdown、HTML 和 CSV 文档高质量转换为 PDF 代理文件进行在线预览。
但 Shumai 不仅仅只满足于 Frame.io 的开源替代。随着 GPT-Image-2、Seedance 2.5 这类强大模型的问世,AI已经生成产品级的图片/视频。很多时候,你只要给一句提示词,就能直接生成或修改图片和视频,而不用再打开 Photoshop、Premiere 或者 Houdini。也正因为这样,Shumai 才把 AI Agent 做成了核心能力:
* **Agent 就像团队成员**:你可以在素材评论区里直接用 `@` 提到 AI Agent,像拉一个真实同事进来一起看稿一样。
* **Agent 也能当你的私聊助手**:你可以单独和它聊天,让它帮你查东西、做任务。
* **自定义技能和工具**:你可以给 Agent 接上自己的脚本、工具和自动化能力。
* **Model Context Protocol (MCP) 支持**:通过远程 MCP 服务端为 Agent 接入外部工具与 API,支持 OAuth 鉴权与按 Agent 独立分配访问权限。
* **沙箱隔离执行**:Agent 生成的脚本会在隔离环境里运行,更安全。
* **AI 自动补全元数据**:新素材上传后,系统可以自动帮你补一些元数据。
* **语义搜索**:支持按画面内容或语义找素材,底层用的是向量 embedding(Gemini Embedding 2)。
* **资源配额**:支持按成员、角色或全团队对 AI Token、成本和工具调用设置使用上限与实时用量监控。
下面是一个简短的 demo 视频,展示了 Shumai Agent 能做什么:
https://github.com/user-attachments/assets/1186067a-72c0-41b9-836f-987438f2332a
Shumai 也将项目管理融入了创意工作流程。 内置的看板可以帮助团队整理任务以及跟踪进度。
最终目标是打造一个 Agentic看板,让 Agent 和团队成员能够围绕同一个目标协同工作。目前这一功能仍在开发中: 你已经可以在看板中为 Agent 创建任务,但 Agent 目前还无法自动执行这些任务。
<img width="2880" height="2160" alt="localhost_3000_teams_01M0HQR3EEBS0B46K4FK39X2C8_kanban(1080p-2) (1)" src="https://github.com/user-attachments/assets/29ff1db1-2d65-4e3c-b467-754b777b406f" />
> [!NOTE]
> **媒体处理与存储机制:** 上传媒体文件后,Shumai 始终会保留并存储原始文件。请注意,系统中显示的文件大小为原始文件的大小,不包含转码后生成的代理文件大小。无论图片还是视频,都会额外生成一张 480p 的 WebP 封面图,作为文件列表中的预览图。此外:
> **媒体处理与存储机制:** 上传媒体文件后,Shumai 始终会保留并存储原始文件。请注意,系统中显示的文件大小为原始文件的大小,不包含转码后生成的代理文件大小。无论图片还是视频,都会额外生成一张 300p 的 WebP 封面图,作为文件列表中的预览图。此外:
>
> * **图片:** 会生成一份与原图分辨率完全一致的高质量 WebP 转码版本,在保证画质的同时提升浏览性能。
> * **视频:** 会根据转码配置生成至少一个代理视频,同时生成雪碧图(Sprite)和一段最长 1600 帧的低分辨率转码视频,用于实现快速拖动时间轴和即时视频预览。
@@ -52,6 +69,24 @@ https://github.com/user-attachments/assets/1186067a-72c0-41b9-836f-987438f2332a
以下是使用本地存储运行 Shumai 的快速上手指南。如需了解高级配置(包括 S3 兼容存储配置、Temporal 工作流编排等),请参阅我们的[官方文档](https://docs.shumai.one/getting-started/overview)。
### 方案一:使用 Docker Compose(推荐)
<details>
<summary>复制这个提示词,让agent帮你安装</summary>
```
请帮我在本机使用 Docker Compose 一键部署 Shumai。
请严格参考官方文档:
https://gh-proxy.com/https://raw.githubusercontent.com/shumaiOne/shumai/main/README_CN.md
要求:
1. 只能使用 Docker Compose local方式部署,不要使用其他部署方式。
2. 部署前请先检查本机是否已安装并可正常使用 Docker 和 Docker Compose。
3. 如果本机未安装 Docker,请立即提醒用户先安装 Docker,并停止后续部署操作。不要自行安装 Docker,也不要采用其他方式部署。
4. 获取 `docker-compose-aliyun.yaml` 文件内容时,必须通过 `https://gh-proxy.com` 加速访问,不要直接访问 GitHub Raw 地址。
```
</details>
使用 Docker Compose 是运行 Shumai 最快的方式。您无需克隆仓库,也不需要手动安装依赖包。请确保电脑上已安装 [Docker](https://docs.docker.com/get-docker/) 和 [Docker Compose](https://docs.docker.com/compose/install/),然后按照以下步骤操作:
+58
View File
@@ -0,0 +1,58 @@
# Security Policy
## Reporting a Vulnerability
Please report security vulnerabilities **privately**. Do not open a public issue,
pull request or Discord message for a suspected vulnerability, as that discloses it
to everyone before a fix exists.
Use **[GitHub private vulnerability reporting](https://github.com/shumaiOne/shumai/security/advisories/new)**
(the "Report a vulnerability" button on the Security tab). If that is unavailable,
contact the maintainer directly rather than in a public channel.
Please include:
- What the issue is, and the impact you believe it has
- The version or commit you tested (`git rev-parse HEAD`, or the image digest)
- Steps to reproduce, ideally minimal
- Your deployment shape: storage backend, whether the agent feature is enabled,
and whether the instance is internet-reachable
## Supported Versions
Shumai is pre-1.0 and moves quickly. Security fixes are made against the **latest
release**. Users are encouraged to track releases rather than pin indefinitely.
## Scope
**In scope** — the application, its API, the shipped Docker images, and the
deployment manifests under `docker-compose/`.
Deployment manifests are explicitly in scope. A default that is unsafe when
followed as documented is a real finding, because the documented quickstart is what
most operators actually run.
**Out of scope**
- Vulnerabilities in third-party dependencies, unless Shumai's use of them is what
creates the exposure. Report those upstream.
- Findings that require an already-compromised host or database.
- Configurations that deviate from the documented deployment.
## Operator guidance
Shumai is self-hosted, so several controls belong to the operator rather than to the
project:
- **Set `BETTER_AUTH_SECRET` per deployment.** It signs sessions and, in local
storage mode, upload URLs. Never reuse a value from documentation or an example.
- **Do not expose the instance to the public internet without a reverse proxy**
terminating TLS and applying your own access policy.
- **Do not publish the database port.** Set a real database password.
- **Treat the agent feature as a code-execution surface.** If you enable it, run it
where a compromise is survivable, and review the sandbox network policy.
## Disclosure
Please allow a reasonable period for a fix before public disclosure. The maintainer
will acknowledge reports and keep reporters informed of progress.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent-app",
"version": "0.3.2",
"version": "0.4.2",
"private": true,
"type": "module",
"bin": {
-1
View File
@@ -1,5 +1,4 @@
export * from '@shumai/agent/src/workflows/agent-autofill'
export * from '@shumai/agent/src/workflows/agent-chat'
export * from '@shumai/agent/src/workflows/agent-embedding'
export * from '@shumai/agent/src/workflows/agent-tool-call'
export * from '@shumai/agent/src/workflows/query-embedding-for-search'
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/cli",
"version": "0.3.2",
"version": "0.4.2",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/transcode-app",
"version": "0.3.2",
"version": "0.4.2",
"private": true,
"type": "module",
"bin": {
+114
View File
@@ -0,0 +1,114 @@
# Web App E2E Tests
Fullstack end-to-end tests for the Shumai web app. They run against a real backend
(pgvector container + local S3 storage) served on `http://localhost:5200` and are
executed with Playwright via `bun run test:e2e:app`.
## Structure
```
apps/web/e2e/
├── fixtures.ts # prisma / owner / project / file fixtures
├── serve.ts # starts the test DB container + web app
├── global-teardown.ts # stops the container and cleans up storage
├── helpers/
│ ├── auth.ts # signup / auth-state injection helpers
│ ├── project.ts # project creation helpers
│ ├── files.ts # share link API helpers
│ ├── team.ts # team member seeding helpers
│ └── ui.ts # common UI flows (signup, login, members dialog)
└── tests/<domain>/ # domain-scoped specs (auth, team, project)
```
## Fixtures
- `prisma` — auto fixture that truncates all tables before each test. Also the
direct way to read/write DB state for assertions and seeding.
- `owner` — an authenticated team owner (seeded through the API), with the session
cookie and auth state injected into the browser context, opened on the team page.
- `project` — `owner` plus a project created through the API (`projectId`,
`projectName`).
- `file` — `project` plus a seeded processed file asset in the project root
(`fileId`, `fileName`). The media kind is configurable per test via
`fileOptions` (default `binary`):
```ts
// default binary file (no extension, no proxy)
test('...', async ({ file }) => {})
// type-scoped file for future viewer/transcode tests
test('...', { fileOptions: { mediaType: 'image' } }, async ({ file }) => {})
```
| mediaType | filename | notes |
|---|---|---|
| `binary` (default) | `test-file-<ts>` | never transcoded, no proxy |
| `text` | `test-file-<ts>.txt` | gets a PDF proxy in real usage |
| `image` | `test-file-<ts>.png` | |
| `video` | `test-file-<ts>.mp4` | |
| `pdf` | `test-file-<ts>.pdf` | |
| `audio` | `test-file-<ts>.wav` | |
## Test Cases
### auth
| Test | File |
|---|---|
| Sign up as the first user and become the team owner | `tests/auth/signup.spec.ts` |
| Log in with existing credentials | `tests/auth/login.spec.ts` |
| Show error when accessing reset-password page without token | `tests/auth/reset-password.spec.ts` |
| Reset password with valid token and allow logging in with new password | `tests/auth/reset-password.spec.ts` |
### team
| Test | File |
|---|---|
| Owner invites a member via invite link and the invitee joins | `tests/team/invite-member.spec.ts` |
| Owner removes a member via the members dialog; the membership is deleted from the DB | `tests/team/remove-member.spec.ts` |
| Owner changes a team member role in the members dialog and permissions update | `tests/team/change-member-role.spec.ts` |
| Owner manages resource quotas in settings and resets usage from the dashboard | `tests/team/quotas-settings.spec.ts` |
### project
| Test | File |
|---|---|
| Owner creates a project with a cover image | `tests/project/create-project.spec.ts` |
| Owner uploads a file or an unsupported file via the "Upload File" context menu action | `tests/project/upload-file.spec.ts` |
| Owner deletes a file, finds it in recently deleted, and restores it | `tests/project/delete-restore.spec.ts` |
| Owner creates a share link from a file context menu and lands on the share page | `tests/project/create-share-link.spec.ts` |
| Owner protects a share with a password; guests must enter it to view | `tests/project/share-password.spec.ts` |
| Owner creates a collection from the sidebar; the collection shows project files | `tests/project/create-collection.spec.ts` |
| Owner invites a member to a project; the member only sees the invited project | `tests/project/invite-member.spec.ts` |
| Owner changes a project member role in the members dialog and permissions update | `tests/project/change-member-role.spec.ts` |
| Owner adds team member to project with role override and permissions enforce project role | `tests/project/role-override.spec.ts` |
| Owner deletes a member from project via the members dialog and project access is revoked | `tests/project/remove-member.spec.ts` |
| Owner drags a file into a folder and a folder into another folder | `tests/project/move-folder-file.spec.ts` |
| Owner drags a file onto a file to create a version stack | `tests/project/version-stack-create.spec.ts` |
| Owner drags a file onto a version stack to add a new version | `tests/project/version-stack-add-version.spec.ts` |
| Owner manages version stack: removes version from 3-version stack | `tests/project/version-stack-manage.spec.ts` |
| Owner removes version from 2-version stack which dissolves into standalone files | `tests/project/version-stack-manage.spec.ts` |
| Dissolving a shared version stack renders the remaining standalone file name on the share page | `tests/project/version-stack-manage.spec.ts` |
| Moving a shared file into a shared version stack leaves only 1 stack card on the share page | `tests/project/version-stack-manage.spec.ts` |
| Owner searches the file list by an English name keyword | `tests/project/search-file-list.spec.ts` |
| Owner searches the file list by Chinese, Japanese, and Korean name keywords | `tests/project/search-file-list.spec.ts` |
| Owner searches the file list with a keyword and a filter condition | `tests/project/search-file-list.spec.ts` |
| Owner saves a search result as a collection | `tests/project/save-search-collection.spec.ts` |
| Image share with watermark + download disabled: no download buttons on public list/detail pages, watermarked proxy served, download-url rejected | `tests/project/share-watermark-no-download.spec.ts` |
| Video share with watermark + download disabled: no download buttons on public list/detail pages, watermarked proxy served, download-url rejected | `tests/project/share-watermark-no-download.spec.ts` |
| Owner views a file and it appears in recents | `tests/project/recents.spec.ts` |
### file
| Test | File |
|---|---|
| Create a comment on a binary file | `tests/file/binary-comment.spec.ts` |
| Upload a txt file, wait for transcode, and create a comment with draw | `tests/file/txt-draw-comment.spec.ts` |
| Upload an image file, wait for transcode, and create a comment with draw | `tests/file/image-draw-comment.spec.ts` |
| Upload a video file, wait for transcode, and create a comment with draw at a non-zero timestamp | `tests/file/video-draw-comment.spec.ts` |
| Upload an audio file, wait for transcode, and create a comment at a non-zero timestamp | `tests/file/audio-timestamp-comment.spec.ts` |
## Conventions
- Seed data via fixtures/API/DB; use the UI only for the flow under test.
- Each test truncates all tables via the auto `prisma` fixture, so fixtures must
create their data per-test.
- App projects run with `workers: 1` because all tests share a single database.
- Every new test case MUST be documented in this README.
+153 -12
View File
@@ -1,31 +1,114 @@
import { test as base } from '@playwright/test'
import type { BrowserContext, Page } from '@playwright/test'
import { PrismaClient } from '../../../packages/db/src/generated/prisma/client'
import { PrismaPg } from '@prisma/adapter-pg'
import { Pool } from 'pg'
import * as fs from 'fs'
import * as path from 'path'
import { fileURLToPath } from 'url'
import {
E2E_APP_URL,
E2E_PASSWORD,
apiSignup,
injectAuthState,
resolveTeamId,
uniqueEmail,
} from './helpers/auth'
import { apiCreateProject, uniqueProjectName } from './helpers/project'
import { apiUploadFile } from './helpers/files'
const currentDir = path.dirname(fileURLToPath(import.meta.url))
const envPath = path.resolve(currentDir, '.env.e2e')
const fixturesDir = path.resolve(currentDir, '../../../packages/e2e/fixtures')
// Load E2E environment variables if present
if (fs.existsSync(envPath)) {
const content = fs.readFileSync(envPath, 'utf8')
content.split('\n').forEach((line) => {
const parts = line.split('=')
if (parts.length >= 2) {
const key = parts[0].trim()
const value = parts.slice(1).join('=').trim()
process.env[key] = value
}
})
function loadEnv() {
if (fs.existsSync(envPath)) {
const content = fs.readFileSync(envPath, 'utf8')
content.split('\n').forEach((line) => {
const parts = line.split('=')
if (parts.length >= 2) {
const key = parts[0].trim()
const value = parts.slice(1).join('=').trim()
process.env[key] = value
}
})
}
}
export const test = base.extend<{ prisma: PrismaClient }>({
// Load E2E environment variables if present
loadEnv()
export interface OwnerFixture {
/** Page opened in a browser context logged in as the team owner. */
page: Page
context: BrowserContext
teamId: string
email: string
password: string
}
export interface ProjectFixture extends OwnerFixture {
projectId: string
projectName: string
/** The project's root folder asset id. */
rootFolderId: string
}
/** The media kind of a seeded test file. */
export type FileMediaType = 'text' | 'binary' | 'image' | 'video' | 'pdf' | 'audio'
export interface FileFixtureOptions {
/** Defaults to `binary` (no extension, never transcoded, no proxy). */
mediaType?: FileMediaType
}
export interface FileFixture extends ProjectFixture {
fileId: string
fileName: string
fileMediaType: FileMediaType
}
const FILE_TYPE_MAP: Record<FileMediaType, { ext: string; mime: string }> = {
text: { ext: 'txt', mime: 'text/plain' },
binary: { ext: '', mime: 'application/octet-stream' },
image: { ext: 'png', mime: 'image/png' },
video: { ext: 'mp4', mime: 'video/mp4' },
pdf: { ext: 'pdf', mime: 'application/pdf' },
audio: { ext: 'wav', mime: 'audio/wav' },
}
function getFileBuffer(mediaType: FileMediaType): Buffer {
switch (mediaType) {
case 'image':
return fs.readFileSync(path.join(fixturesDir, 'small.png'))
case 'video':
return fs.readFileSync(path.join(fixturesDir, 'small.mp4'))
case 'audio':
return fs.readFileSync(path.join(fixturesDir, 'small.wav'))
case 'pdf':
return fs.readFileSync(path.join(fixturesDir, 'test.pdf'))
case 'text':
return Buffer.from(
'Sample text content for PDF proxy transcode test.\nSecond line of text content.',
)
case 'binary':
default:
return Buffer.from('binary-file-content-sample')
}
}
export const test = base.extend<{
prisma: PrismaClient
owner: OwnerFixture
project: ProjectFixture
fileOptions: FileFixtureOptions
file: FileFixture
}>({
fileOptions: [{ mediaType: 'binary' }, { option: true }],
prisma: [
// eslint-disable-next-line no-empty-pattern
async ({}, use) => {
loadEnv()
const connectionString = process.env.DATABASE_URL
const pool = new Pool({ connectionString })
const adapter = new PrismaPg(pool)
@@ -51,6 +134,64 @@ export const test = base.extend<{ prisma: PrismaClient }>({
},
{ auto: true },
],
/**
* Sets up an authenticated team owner through the API (fast, no UI): creates
* the user, stores the session cookie in the browser context, injects the
* persisted auth state, and opens the team page.
*/
owner: async ({ browser }, use) => {
const context = await browser.newContext({
// eslint-disable-next-line @typescript-eslint/naming-convention
baseURL: E2E_APP_URL,
})
const email = uniqueEmail('owner')
const password = E2E_PASSWORD
try {
// Create the owner via the API so the session cookie lands in the context
const user = await apiSignup(context.request, email, password)
await injectAuthState(context, user)
const teamId = await resolveTeamId(context.request)
// Open the app already logged in on the team page
const page = await context.newPage()
await page.goto(`/teams/${teamId}`)
await use({ page, context, teamId, email, password })
} finally {
await context.close()
}
},
/** Sets up an authenticated team owner plus a project created via the API. */
project: async ({ owner, prisma }, use) => {
const name = uniqueProjectName()
const project = await apiCreateProject(owner.context.request, owner.teamId, name)
const projectRow = await prisma.project.findUnique({ where: { id: project.id } })
const rootFolderId = projectRow?.rootFolderId
if (!rootFolderId) throw new Error('Project has no root folder')
await use({ ...owner, projectId: project.id, projectName: name, rootFolderId })
},
/**
* Sets up an owner + project with an uploaded file asset under the
* project root folder via the API upload task endpoints. The media type is configurable per test via
* `fileOptions` (defaults to `binary`).
*/
file: async ({ project, fileOptions }, use) => {
const mediaType: FileMediaType = fileOptions.mediaType ?? 'binary'
const { ext, mime } = FILE_TYPE_MAP[mediaType]
const fileName = `test-file-${Date.now()}${ext ? `.${ext}` : ''}`
const buffer = getFileBuffer(mediaType)
const uploaded = await apiUploadFile(
project.context.request,
project.teamId,
project.rootFolderId,
fileName,
mime,
buffer,
)
await use({ ...project, fileId: uploaded.fileId, fileName, fileMediaType: mediaType })
},
})
export { expect } from '@playwright/test'
+31
View File
@@ -0,0 +1,31 @@
import type { PrismaClient } from '../../../../packages/db/src/generated/prisma/client'
/**
* Seeds a folder asset directly through the database (fast, no UI). The folder
* is placed under `parentId` (usually a project's root folder).
*/
export async function seedFolder(
prisma: PrismaClient,
projectId: string,
parentId: string,
name: string,
) {
return prisma.asset.create({
data: { name, type: 'folder', status: 'processed', projectId, parentId },
})
}
/**
* Seeds a processed file asset directly through the database (fast, no UI).
* The file is placed under `parentId` (a folder or a version stack).
*/
export async function seedFile(
prisma: PrismaClient,
projectId: string,
parentId: string,
name: string,
) {
return prisma.asset.create({
data: { name, type: 'file', status: 'processed', sizeByte: 10, projectId, parentId },
})
}
+76
View File
@@ -0,0 +1,76 @@
import type { APIRequestContext, BrowserContext } from '@playwright/test'
/** Base URL of the fullstack app under test (must match playwright.config.ts). */
export const E2E_APP_URL = process.env.E2E_APP_URL || 'http://localhost:5200'
export const E2E_PASSWORD = 'Password123!'
export function uniqueEmail(prefix = 'user'): string {
return `${prefix}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}@shumai.local`
}
export interface AuthUser {
id: string
email: string
name: string
}
interface SignupResponse {
token?: string | null
user?: AuthUser
error?: { message?: string } | null
}
/**
* Creates a user through the better-auth API.
* When passed a `context.request`, the session cookie is stored in the browser
* context and the user is logged in.
*/
export async function apiSignup(
request: APIRequestContext,
email: string,
password: string,
options: { inviteCode?: string } = {},
): Promise<AuthUser> {
const res = await request.post('/api/auth/sign-up/email', {
data: {
name: email,
email,
password,
...(options.inviteCode ? { inviteCode: options.inviteCode } : {}),
},
})
if (!res.ok()) {
throw new Error(`Signup API failed (${res.status()}): ${await res.text()}`)
}
const body = (await res.json()) as SignupResponse
if (!body.user) {
throw new Error(`Signup API returned no user: ${JSON.stringify(body)}`)
}
return body.user
}
/**
* Injects the persisted auth state (zustand `auth-storage`) so the app
* considers the session logged in. The session cookie must already be present
* in the context (created via `context.request`).
*/
export async function injectAuthState(context: BrowserContext, user: AuthUser): Promise<void> {
await context.addInitScript((u) => {
localStorage.setItem('auth-storage', JSON.stringify({ state: { user: u }, version: 0 }))
}, user)
}
/** Resolves the id of the first team for the authenticated request context. */
export async function resolveTeamId(request: APIRequestContext): Promise<string> {
const res = await request.get('/api/teams')
if (!res.ok()) {
throw new Error(`GET /api/teams failed (${res.status()}): ${await res.text()}`)
}
const body = (await res.json()) as { data: { id: string }[] }
const teamId = body.data?.[0]?.id
if (!teamId) {
throw new Error(`No team found in /api/teams response: ${JSON.stringify(body)}`)
}
return teamId
}
+55
View File
@@ -0,0 +1,55 @@
import type { Locator, Page } from '@playwright/test'
/**
* Performs a pointer drag from one card to another. dnd-kit v6 uses a
* `PointerSensor` with a 10px distance activation constraint, and WebKit only
* reliably starts the drag when pointer moves are delivered in separate,
* awaited batches (a single fast `mouse.move(steps)` can be coalesced before
* the drag activates, losing the drop). Moves are therefore chunked with
* intermediate waits so the activation and drag move handlers run between
* batches on every browser.
*/
export async function dragCard(page: Page, source: Locator, target: Locator): Promise<void> {
const sourceBox = (await source.boundingBox())!
const targetBox = (await target.boundingBox())!
if (!sourceBox || !targetBox) {
throw new Error('Drag source or target has no bounding box')
}
const fromX = sourceBox.x + sourceBox.width / 2
const fromY = sourceBox.y + sourceBox.height / 2
const toX = targetBox.x + targetBox.width / 2
const toY = targetBox.y + targetBox.height / 2
await page.mouse.move(fromX, fromY)
await page.mouse.down()
const dx = toX - fromX
const dy = toY - fromY
const dist = Math.hypot(dx, dy)
// Step past the 10px activation constraint to trigger drag activation
// and give the browser a brief moment for status to become 'dragging'
const activationRatio = dist > 0 ? Math.min(1, 15 / dist) : 1
await page.mouse.move(fromX + dx * activationRatio, fromY + dy * activationRatio)
await page.waitForTimeout(50)
// Move in several chunks so the intermediate dragmove handlers run
const chunks = 8
for (let i = 1; i <= chunks; i++) {
const x = fromX + (dx * i) / chunks
const y = fromY + (dy * i) / chunks
await page.mouse.move(x, y)
await page.waitForTimeout(20)
}
// Ensure pointer is directly over target center and allow requestAnimationFrame
// and collision microtasks to settle before triggering drop
await page.mouse.move(toX, toY)
await page.evaluate(
() => new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve))),
)
await page.waitForTimeout(100)
await page.mouse.up()
}
+183
View File
@@ -0,0 +1,183 @@
import type { APIRequestContext, Locator, Page } from '@playwright/test'
interface CreateShareResponse {
id?: string
name?: string
rootFolderId?: string
}
/**
* Locates a file/folder card by its name. Matches both read-only text nodes
* and active EditableText inputs. The returned locator points at the card
* wrapper div (right-clickable to open the card context menu).
*/
export function fileCard(page: Page, name: string): Locator {
return page
.locator('[data-testid="file-card"]')
.filter({
has: page.locator(`input[value="${name}"]`).or(page.getByText(name, { exact: true })),
})
.first()
}
/** Creates a share link through the API for the authenticated request context. */
export async function apiCreateShare(
request: APIRequestContext,
projectId: string,
name: string,
): Promise<{ id: string; name: string; rootFolderId: string }> {
const res = await request.post(`/api/projects/${projectId}/shares`, {
data: { name },
})
if (!res.ok()) {
throw new Error(`Create share API failed (${res.status()}): ${await res.text()}`)
}
const body = (await res.json()) as CreateShareResponse
if (!body.id || !body.name || !body.rootFolderId) {
throw new Error(`Create share returned an unexpected response: ${JSON.stringify(body)}`)
}
return { id: body.id, name: body.name, rootFolderId: body.rootFolderId }
}
/** Adds assets to a share link through the API. */
export async function apiAddAssetsToShare(
request: APIRequestContext,
shareId: string,
assetIds: string[],
): Promise<void> {
const res = await request.post(`/api/shares/${shareId}/assets`, {
data: { assetIds },
})
if (!res.ok()) {
throw new Error(`Add assets to share API failed (${res.status()}): ${await res.text()}`)
}
}
/** Updates share link settings (e.g. `allowDownload`) through the API. */
export async function apiUpdateShare(
request: APIRequestContext,
shareId: string,
patch: { allowDownload?: boolean },
): Promise<void> {
const res = await request.put(`/api/shares/${shareId}`, { data: patch })
if (!res.ok()) {
throw new Error(`Update share API failed (${res.status()}): ${await res.text()}`)
}
}
/**
* Enables a text watermark on the share link through the API. Uses the same
* default block shape the watermark editor produces (a "CONFIDENTIAL" text
* block), so the transcode workflow runs against a realistic config.
*/
export async function apiEnableShareWatermark(
request: APIRequestContext,
shareId: string,
): Promise<void> {
const res = await request.put(`/api/shares/${shareId}/watermark`, {
data: {
enabled: true,
config: {
blocks: [
{
id: `wm-${Date.now()}`,
type: 'text',
text: 'CONFIDENTIAL',
x: 0.5,
y: 0.5,
// size is a fraction (0..1) of the canvas width
size: 0.05,
color: '#999999',
opacity: 0.5,
rotation: -30,
},
],
},
},
})
if (!res.ok()) {
throw new Error(`Enable share watermark API failed (${res.status()}): ${await res.text()}`)
}
}
/** Requests a public download URL. Returns status + body for assertions. */
export async function apiPublicDownloadUrl(
request: APIRequestContext,
shareId: string,
fileId: string,
key: string,
): Promise<{ status: number; body: unknown }> {
const res = await request.post(`/api/shares/${shareId}/files/${fileId}/download-url`, {
data: { key },
})
return { status: res.status(), body: await res.json().catch(() => null) }
}
/** Uploads a file asset through the official API upload task endpoints. */
export async function apiUploadFile(
request: APIRequestContext,
teamId: string,
parentId: string,
fileName: string,
mimeType: string,
buffer: Buffer,
): Promise<{ fileId: string; fileName: string }> {
const tempId = `temp-${Date.now()}`
const createRes = await request.post(`/api/teams/${teamId}/upload/tasks`, {
data: {
parentId,
files: [
{
id: tempId,
name: fileName,
type: 'file',
size: buffer.length,
mediaType: mimeType,
children: [],
},
],
},
})
if (!createRes.ok()) {
throw new Error(`Create upload task failed (${createRes.status()}): ${await createRes.text()}`)
}
const taskData = (await createRes.json()) as {
taskId: string
presignedUrls: Array<{ id: string; fileId: string; url: string }>
createdAssets?: Array<{ tempId: string; assetId: string }>
}
const presigned = taskData.presignedUrls.find((p) => p.id === tempId)
const createdAssetId =
taskData.createdAssets?.find((c) => c.tempId === tempId)?.assetId || presigned?.fileId
if (!presigned || !createdAssetId) {
throw new Error('Create upload task did not return presigned URL or asset ID')
}
// Upload file buffer to presigned URL
const uploadRes = await request.put(presigned.url, {
headers: {
'Content-Type': mimeType,
},
data: buffer,
})
if (!uploadRes.ok()) {
throw new Error(
`Upload to S3 presigned URL failed (${uploadRes.status()}): ${await uploadRes.text()}`,
)
}
// Confirm upload task
const confirmRes = await request.patch(`/api/teams/${teamId}/upload/tasks/${taskData.taskId}`, {
data: {
fileId: createdAssetId,
status: 'success',
},
})
if (!confirmRes.ok()) {
throw new Error(
`Confirm upload task failed (${confirmRes.status()}): ${await confirmRes.text()}`,
)
}
return { fileId: createdAssetId, fileName }
}
+29
View File
@@ -0,0 +1,29 @@
import type { APIRequestContext } from '@playwright/test'
export function uniqueProjectName(): string {
return `Project ${Date.now()}-${Math.random().toString(36).slice(2, 6)}`
}
interface CreateProjectResponse {
id?: string
name?: string
}
/** Creates a project through the API for the authenticated request context. */
export async function apiCreateProject(
request: APIRequestContext,
teamId: string,
name: string,
): Promise<{ id: string; name: string }> {
const res = await request.post(`/api/teams/${teamId}/projects`, {
data: { name },
})
if (!res.ok()) {
throw new Error(`Create project API failed (${res.status()}): ${await res.text()}`)
}
const body = (await res.json()) as CreateProjectResponse
if (!body.id || !body.name) {
throw new Error(`Create project returned an unexpected response: ${JSON.stringify(body)}`)
}
return { id: body.id, name: body.name }
}
+121
View File
@@ -0,0 +1,121 @@
import type { Browser, BrowserContext } from '@playwright/test'
import type { PrismaClient } from '../../../../packages/db/src/generated/prisma/client'
import { E2E_APP_URL, E2E_PASSWORD, apiSignup, injectAuthState, uniqueEmail } from './auth'
export interface SeededMember {
userId: string
email: string
teamMemberId: string
context?: BrowserContext
}
/**
* Seeds a new user with a team membership of the given role.
* If `browser` and `ownerContext` are provided, generates an invite code via the owner's request context
* so `apiSignup` succeeds and session cookies/auth state are initialized.
*/
export async function seedTeamMember(
prisma: PrismaClient,
teamId: string,
role: 'owner' | 'editor' | 'reviewer',
browser?: Browser,
ownerContext?: BrowserContext,
): Promise<SeededMember> {
if (browser && ownerContext) {
const inviteRes = await ownerContext.request.post(`/api/teams/${teamId}/invite`, {
data: { role: role === 'owner' ? 'editor' : role },
})
if (!inviteRes.ok()) {
throw new Error(`Failed to generate team invite code: ${await inviteRes.text()}`)
}
const inviteBody = (await inviteRes.json()) as { code: string }
// eslint-disable-next-line @typescript-eslint/naming-convention
const context = await browser.newContext({ baseURL: E2E_APP_URL })
const email = uniqueEmail(`member-${role}`)
const user = await apiSignup(context.request, email, E2E_PASSWORD, {
inviteCode: inviteBody.code,
})
await injectAuthState(context, user)
const tm = await prisma.teamMember.findUnique({
where: { teamIdUserId: { teamId, userId: user.id } },
})
if (tm && tm.role !== role) {
await prisma.teamMember.update({
where: { id: tm.id },
data: { role, scope: 'team' },
})
}
return { userId: user.id, email, teamMemberId: tm!.id, context }
}
const email = uniqueEmail(`member-${role}`)
const user = await prisma.user.create({
data: { name: email, email, password: 'Password123!' },
})
const tm = await prisma.teamMember.create({
data: { teamId, userId: user.id, role, scope: 'team' },
})
return { userId: user.id, email, teamMemberId: tm.id }
}
/**
* Seeds a new user with a project membership of the given role.
* If `browser` and `ownerContext` are provided, generates an invite code via the owner's request context
* so `apiSignup` succeeds and session cookies/auth state are initialized.
*/
export async function seedProjectMember(
prisma: PrismaClient,
teamId: string,
projectId: string,
role: 'editor' | 'reviewer',
scope: 'team' | 'project' = 'project',
browser?: Browser,
ownerContext?: BrowserContext,
): Promise<SeededMember & { projectMemberId: string }> {
if (browser && ownerContext) {
const inviteRes = await ownerContext.request.post(`/api/projects/${projectId}/invite`, {
data: { role },
})
if (!inviteRes.ok()) {
throw new Error(`Failed to generate project invite code: ${await inviteRes.text()}`)
}
const inviteBody = (await inviteRes.json()) as { code: string }
// eslint-disable-next-line @typescript-eslint/naming-convention
const context = await browser.newContext({ baseURL: E2E_APP_URL })
const email = uniqueEmail(`proj-member-${role}`)
const user = await apiSignup(context.request, email, E2E_PASSWORD, {
inviteCode: inviteBody.code,
})
await injectAuthState(context, user)
const tm = await prisma.teamMember.findUnique({
where: { teamIdUserId: { teamId, userId: user.id } },
})
const pm = await prisma.projectMember.findFirst({
where: { projectId, teamMemberId: tm!.id },
})
if (pm && pm.role !== role) {
await prisma.projectMember.update({
where: { id: pm.id },
data: { role },
})
}
return { userId: user.id, email, teamMemberId: tm!.id, projectMemberId: pm!.id, context }
}
const email = uniqueEmail(`proj-member-${role}`)
const user = await prisma.user.create({
data: { name: email, email, password: 'Password123!' },
})
const tm = await prisma.teamMember.create({
data: { teamId, userId: user.id, role: scope === 'team' ? 'editor' : 'reviewer', scope },
})
const pm = await prisma.projectMember.create({
data: { projectId, teamMemberId: tm.id, role },
})
return { userId: user.id, email, teamMemberId: tm.id, projectMemberId: pm.id }
}
+55
View File
@@ -0,0 +1,55 @@
import { expect, type Locator, type Page } from '@playwright/test'
export function parseTeamIdFromUrl(url: string): string {
const match = url.match(/\/teams\/([^/]+)/)
if (!match) throw new Error(`Could not parse teamId from URL: ${url}`)
return match[1]
}
/** Performs the signup UI flow and waits for the redirect to the team page. */
export async function signupViaUi(page: Page, email: string, password: string): Promise<string> {
await page.goto('/signup')
await page.fill('#email', email)
await page.fill('#password', password)
await page.click('button[type="submit"]')
await page.waitForURL(/\/teams\/[^/]+/)
return parseTeamIdFromUrl(page.url())
}
/** Performs the login UI flow and waits for the redirect to the team page. */
export async function loginViaUi(page: Page, email: string, password: string): Promise<string> {
await page.goto('/login')
await page.fill('#email', email)
await page.fill('#password', password)
await page.click('button[type="submit"]')
await page.waitForURL(/\/teams\/[^/]+/)
return parseTeamIdFromUrl(page.url())
}
/** Opens the team members dialog from the team page header. */
export async function openMembersDialog(page: Page): Promise<Locator> {
await page.getByTestId('team-members-trigger').click()
const dialog = page.locator('[role="dialog"]')
await expect(dialog).toBeVisible()
return dialog
}
/** Opens the project members dialog from the project file-browser toolbar. */
export async function openProjectMembersDialog(page: Page): Promise<Locator> {
await page.getByTestId('project-members-trigger').click()
const dialog = page.locator('[role="dialog"]')
await expect(dialog).toBeVisible()
return dialog
}
/** Clicks "Generate Link" in the members dialog and returns the invite link. */
export async function generateInviteLink(dialog: Locator): Promise<string> {
await dialog.getByRole('button', { name: 'Generate Link' }).click()
const inviteInput = dialog.locator('input[readonly]')
await expect(inviteInput).toBeVisible()
const inviteLink = await inviteInput.inputValue()
if (!inviteLink.includes('inviteCode=')) {
throw new Error(`Unexpected invite link: ${inviteLink}`)
}
return inviteLink
}
+4 -1
View File
@@ -32,4 +32,7 @@ async function start() {
await import('../src/index.ts')
}
start().catch(console.error)
start().catch((err) => {
console.error('Failed to start E2E server:', err)
process.exit(1)
})
+18
View File
@@ -0,0 +1,18 @@
import { expect, test } from '../../fixtures'
import { apiSignup, E2E_PASSWORD, uniqueEmail } from '../../helpers/auth'
import { loginViaUi } from '../../helpers/ui'
test('should log in with existing credentials', async ({ page, request }) => {
const email = uniqueEmail('login')
const password = E2E_PASSWORD
// Seed an existing user through the API (setup, not the flow under test)
await apiSignup(request, email, password)
// The default page context is a fresh, logged-out session
await loginViaUi(page, email, password)
// The logged-in user lands on their team page
await expect(page).toHaveURL(/\/teams\/[^/]+/)
await expect(page.getByRole('heading', { name: 'Projects' })).toBeVisible()
})
@@ -0,0 +1,63 @@
import { expect, test } from '../../fixtures'
import { apiSignup, E2E_PASSWORD, uniqueEmail } from '../../helpers/auth'
import { loginViaUi } from '../../helpers/ui'
test('should show error when accessing reset-password page without token', async ({ page }) => {
await page.goto('/reset-password')
await expect(page.getByText('Invalid or missing reset token.')).toBeVisible()
await expect(page.getByRole('button', { name: 'Go to Login' })).toBeVisible()
})
test('should reset password with valid token and allow logging in with new password', async ({
page,
request,
prisma,
}) => {
const email = uniqueEmail('reset-pw')
const oldPassword = E2E_PASSWORD
const newPassword = 'newSecretPassword123'
// 1. Seed user
await apiSignup(request, email, oldPassword)
// 2. Request password reset via API
const resetRes = await request.post('/api/auth/request-password-reset', {
data: {
email,
redirectTo: '/reset-password',
},
})
expect(resetRes.ok()).toBe(true)
// Retrieve the generated verification token from DB
const verification = await prisma.verification.findFirst({
where: {
identifier: { startsWith: 'reset-password:' },
},
orderBy: { createdAt: 'desc' },
})
expect(verification).not.toBeNull()
const token = verification!.identifier.replace('reset-password:', '')
// 3. Open reset password page with token
await page.goto(`/reset-password?token=${token}`)
// 4. Fill in new password and confirmation
await page.locator('input#password').fill(newPassword)
await page.locator('input#confirmPassword').fill(newPassword)
// 5. Submit form
await page.getByRole('button', { name: 'Reset Password' }).click()
// 6. Verify success state
await expect(page.getByText('Password reset successfully!')).toBeVisible()
// 7. Click Go to Login
await page.getByRole('button', { name: 'Go to Login' }).click()
await expect(page).toHaveURL('/login')
// 8. Log in with the new password
await loginViaUi(page, email, newPassword)
await expect(page).toHaveURL(/\/teams\/[^/]+/)
await expect(page.getByRole('heading', { name: 'Projects' })).toBeVisible()
})
+24
View File
@@ -0,0 +1,24 @@
import { expect, test } from '../../fixtures'
import { E2E_PASSWORD, uniqueEmail } from '../../helpers/auth'
import { signupViaUi } from '../../helpers/ui'
test('should sign up as the first user and become the team owner', async ({ page, prisma }) => {
const email = uniqueEmail('signup')
const password = E2E_PASSWORD
const teamId = await signupViaUi(page, email, password)
// The first user is redirected to their single team
await expect(page).toHaveURL(/\/teams\/[^/]+/)
await expect(page.getByRole('heading', { name: 'Projects' })).toBeVisible()
// The new user exists and is the owner of the team
const user = await prisma.user.findUnique({ where: { email } })
expect(user).not.toBeNull()
const membership = await prisma.teamMember.findUnique({
where: { teamIdUserId: { teamId, userId: user!.id } },
})
expect(membership?.role).toBe('owner')
expect(membership?.scope).toBe('team')
})
@@ -0,0 +1,68 @@
import { expect, test } from '../../fixtures'
test.use({ fileOptions: { mediaType: 'audio' } })
test('upload an audio file, create a comment at a non-zero timestamp', async ({ file, prisma }) => {
const { page, projectId, fileId } = file
const commentText = `Audio timestamp comment ${Date.now()}`
// Wait for audio transcode completion
await expect
.poll(
async () => {
const asset = await prisma.asset.findUnique({ where: { id: fileId } })
return asset?.status
},
{ timeout: 30_000 },
)
.toBe('processed')
await page.goto(`/projects/${projectId}/files/${fileId}`)
// Wait for audio/video media element to be visible
const mediaEl = page.locator('audio, video').first()
await expect(mediaEl).toBeVisible({ timeout: 30_000 })
// Seek media element to non-zero timestamp (e.g. 1 second)
await page.evaluate(() => {
const el = document.querySelector('audio, video') as HTMLMediaElement | null
if (el) el.currentTime = 1
})
await page.waitForTimeout(500)
// Type comment text and send
const input = page.locator('[contenteditable="true"]').first()
await input.fill(commentText)
const sendBtn = page.locator('button:has(svg.lucide-arrow-up)').last()
await sendBtn.click()
// Verify comment card appears in sidebar
await expect(page.getByText(commentText)).toBeVisible()
// Ensure comment is persisted in DB before page reload
await expect
.poll(async () => {
return prisma.assetComment.findFirst({
where: { assetId: fileId, message: commentText },
})
})
.not.toBeNull()
// Reload page
await page.reload()
// Click the comment in right sidebar
const commentCard = page.getByText(commentText).first()
await expect(commentCard).toBeVisible()
await commentCard.click()
// Verify media element jumped to timestamp
await expect
.poll(async () => {
return page.evaluate(() => {
const el = document.querySelector('audio, video') as HTMLMediaElement | null
return el ? el.currentTime : 0
})
})
.toBeGreaterThan(0)
})
@@ -0,0 +1,32 @@
import { expect, test } from '../../fixtures'
test('create a comment on a binary file', async ({ file, prisma }) => {
const { page, projectId, fileId } = file
const commentText = `Binary comment ${Date.now()}`
await page.goto(`/projects/${projectId}/files/${fileId}`)
// Wait for file page & comments sidebar to load
const input = page.locator('[contenteditable="true"]').first()
await expect(input).toBeVisible()
// Type comment text and send
await input.fill(commentText)
const sendBtn = page.locator('button:has(svg.lucide-arrow-up)').last()
await sendBtn.click()
// Verify comment card appears in sidebar
await expect(page.getByText(commentText)).toBeVisible()
// Refresh page and click the comment
await page.reload()
const commentCard = page.getByText(commentText).first()
await expect(commentCard).toBeVisible()
await commentCard.click()
// DB verification
const commentInDb = await prisma.assetComment.findFirst({
where: { assetId: fileId, message: commentText },
})
expect(commentInDb).not.toBeNull()
})
@@ -0,0 +1,71 @@
import { expect, test } from '../../fixtures'
test.use({ fileOptions: { mediaType: 'image' } })
test('upload an image file, create a comment with draw', async ({ file, prisma }) => {
const { page, projectId, fileId } = file
const commentText = `Image draw comment ${Date.now()}`
// Wait for transcode completion
await expect
.poll(
async () => {
const asset = await prisma.asset.findUnique({ where: { id: fileId } })
return asset?.status
},
{ timeout: 30_000 },
)
.toBe('processed')
await page.goto(`/projects/${projectId}/files/${fileId}`)
// Verify image is visible
await expect(page.locator('img').first()).toBeVisible({ timeout: 30_000 })
// Toggle drawing mode
const drawToggleBtn = page.getByTitle('Toggle Annotation')
await expect(drawToggleBtn).toBeVisible()
await drawToggleBtn.click()
// Perform drawing gesture on canvas overlay
const canvas = page.locator('.konvajs-content canvas').first()
await expect(canvas).toBeVisible()
const box = await canvas.boundingBox()
if (box) {
const startX = box.x + box.width * 0.3
const startY = box.y + box.height * 0.3
const endX = box.x + box.width * 0.6
const endY = box.y + box.height * 0.6
await page.mouse.move(startX, startY)
await page.mouse.down()
await page.mouse.move(endX, endY, { steps: 5 })
await page.mouse.up()
}
// Type comment text and send
const input = page.locator('[contenteditable="true"]').first()
await input.fill(commentText)
const sendBtn = page.locator('button:has(svg.lucide-arrow-up)').last()
await sendBtn.click()
// Verify comment card appears in sidebar
await expect(page.getByText(commentText)).toBeVisible()
// Ensure comment is persisted in DB before page reload
await expect
.poll(async () => {
return prisma.assetComment.findFirst({
where: { assetId: fileId, message: commentText },
})
})
.not.toBeNull()
// Reload page and click comment in right sidebar
await page.reload()
const commentCard = page.getByText(commentText).first()
await expect(commentCard).toBeVisible()
await commentCard.click()
// Verify draw annotation is rendered on canvas
await expect(page.locator('.konvajs-content canvas').first()).toBeVisible()
})
@@ -0,0 +1,75 @@
import { expect, test } from '../../fixtures'
test.use({ fileOptions: { mediaType: 'text' } })
test('upload a txt file, create a comment with draw', async ({ file, prisma }) => {
const { page, projectId, fileId } = file
const commentText = `Txt draw comment ${Date.now()}`
// Wait for background transcode worker to complete (txt -> pdf proxy)
await expect
.poll(
async () => {
const asset = await prisma.asset.findUnique({ where: { id: fileId } })
return asset?.status
},
{ timeout: 60_000 },
)
.toBe('processed')
await page.goto(`/projects/${projectId}/files/${fileId}`)
// Verify PDF viewer container or page is loaded
await expect(page.locator('.pdf-viewer, canvas, [data-testid="pdf-viewer"]').first()).toBeVisible(
{
timeout: 30_000,
},
)
// Toggle drawing mode
const drawToggleBtn = page.getByTitle('Toggle Annotation')
await expect(drawToggleBtn).toBeVisible()
await drawToggleBtn.click()
// Perform drawing gesture on canvas overlay
const canvas = page.locator('.konvajs-content canvas').first()
await expect(canvas).toBeVisible()
const box = await canvas.boundingBox()
if (box) {
const startX = box.x + box.width * 0.3
const startY = box.y + box.height * 0.3
const endX = box.x + box.width * 0.6
const endY = box.y + box.height * 0.6
await page.mouse.move(startX, startY)
await page.mouse.down()
await page.mouse.move(endX, endY, { steps: 5 })
await page.mouse.up()
}
// Type comment text and send
const input = page.locator('[contenteditable="true"]').first()
await input.fill(commentText)
const sendBtn = page.locator('button:has(svg.lucide-arrow-up)').last()
await sendBtn.click()
// Verify comment card appears in sidebar
await expect(page.getByText(commentText)).toBeVisible()
// Ensure comment is persisted in DB before page reload
await expect
.poll(async () => {
return prisma.assetComment.findFirst({
where: { assetId: fileId, message: commentText },
})
})
.not.toBeNull()
// Reload page and click comment in right sidebar
await page.reload()
const commentCard = page.getByText(commentText).first()
await expect(commentCard).toBeVisible()
await commentCard.click()
// Verify draw annotation is rendered on canvas
await expect(page.locator('.konvajs-content canvas').first()).toBeVisible()
})
@@ -0,0 +1,93 @@
import { expect, test } from '../../fixtures'
test.use({ fileOptions: { mediaType: 'video' } })
test('upload a video file, create a comment with draw at a non-zero timestamp', async ({
file,
prisma,
}) => {
const { page, projectId, fileId } = file
const commentText = `Video draw comment ${Date.now()}`
// Wait for video transcode completion
await expect
.poll(
async () => {
const asset = await prisma.asset.findUnique({ where: { id: fileId } })
return asset?.status
},
{ timeout: 60_000 },
)
.toBe('processed')
await page.goto(`/projects/${projectId}/files/${fileId}`)
// Wait for video element to be visible
const video = page.locator('video').first()
await expect(video).toBeVisible({ timeout: 30_000 })
// Seek video to non-zero timestamp (e.g. 1 second)
await page.evaluate(() => {
const v = document.querySelector('video')
if (v) v.currentTime = 1
})
await page.waitForTimeout(500)
// Toggle drawing mode
const drawToggleBtn = page.getByTitle('Toggle Annotation')
await expect(drawToggleBtn).toBeVisible()
await drawToggleBtn.click()
// Perform drawing gesture on canvas overlay
const canvas = page.locator('.konvajs-content canvas').first()
await expect(canvas).toBeVisible()
const box = await canvas.boundingBox()
if (box) {
const startX = box.x + box.width * 0.3
const startY = box.y + box.height * 0.3
const endX = box.x + box.width * 0.6
const endY = box.y + box.height * 0.6
await page.mouse.move(startX, startY)
await page.mouse.down()
await page.mouse.move(endX, endY, { steps: 5 })
await page.mouse.up()
}
// Type comment text and send
const input = page.locator('[contenteditable="true"]').first()
await input.fill(commentText)
const sendBtn = page.locator('button:has(svg.lucide-arrow-up)').last()
await sendBtn.click()
// Verify comment card appears in sidebar
await expect(page.getByText(commentText)).toBeVisible()
// Ensure comment is persisted in DB before page reload
await expect
.poll(async () => {
return prisma.assetComment.findFirst({
where: { assetId: fileId, message: commentText },
})
})
.not.toBeNull()
// Reload page
await page.reload()
// Click the comment in right sidebar
const commentCard = page.getByText(commentText).first()
await expect(commentCard).toBeVisible()
await commentCard.click()
// Verify video jumped to timestamp and draw annotation is shown on canvas
await expect
.poll(async () => {
return page.evaluate(() => {
const v = document.querySelector('video')
return v ? v.currentTime : 0
})
})
.toBeGreaterThan(0)
await expect(page.locator('.konvajs-content canvas').first()).toBeVisible()
})
@@ -0,0 +1,64 @@
import { expect, test } from '../../fixtures'
import { openProjectMembersDialog } from '../../helpers/ui'
import { seedProjectMember } from '../../helpers/team'
test('owner changes a project member role in the members dialog and permissions update', async ({
project,
browser,
prisma,
}) => {
const { page, teamId, projectId, context } = project
const member = await seedProjectMember(
prisma,
teamId,
projectId,
'reviewer',
'project',
browser,
context,
)
// Open the project detail page and open the project members dialog
await page.goto(`/projects/${projectId}`)
const dialog = await openProjectMembersDialog(page)
const memberRow = dialog
.locator('div.flex.items-center.justify-between')
.filter({ hasText: member.email })
await expect(memberRow).toBeVisible()
// Click role dropdown trigger for this member (currently Reviewer)
await memberRow.getByRole('button', { name: /Reviewer/i }).click()
// Select "Editor" from the dropdown menu
await page.getByRole('menuitemradio', { name: 'Editor' }).click()
// Verify UI displays updated role "Editor"
await expect(memberRow.getByRole('button', { name: /Editor/i })).toBeVisible()
// DB verification: projectMember role is updated to editor
const pm = await prisma.projectMember.findUnique({
where: {
projectIdTeamMemberId: {
projectId,
teamMemberId: member.teamMemberId,
},
},
})
expect(pm?.role).toBe('editor')
// Functional verification: project member can now perform edit API operations on the project
const memberContext = member.context!
const memberPage = await memberContext.newPage()
try {
await memberPage.goto(`/teams/${teamId}`)
// Verify edit permission on project via API (PUT /api/projects/:projectId requires Edit permission)
const res = await memberContext.request.put(`/api/projects/${projectId}`, {
data: { enableNotification: true },
})
expect(res.status()).toBe(200)
} finally {
await memberContext.close()
}
})
@@ -0,0 +1,31 @@
import { expect, test } from '../../fixtures'
import { fileCard } from '../../helpers/files'
test('owner creates a collection from the sidebar and the collection shows project files', async ({
file,
prisma,
}) => {
const { page, projectId, fileName } = file
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, fileName)).toBeVisible()
// Create a collection via the sidebar "+" button
await page.getByTestId('create-collection').click()
await expect(page).toHaveURL(/\/collections\/[^/]+/)
await expect(page.getByText('Collection created')).toBeVisible()
// The collection page is a recursive files-only view of the project
await expect(fileCard(page, fileName)).toBeVisible()
const collection = await prisma.collection.findFirst({ where: { projectId } })
expect(collection).not.toBeNull()
expect(collection?.name).toBe('Untitled Collection')
const project = await prisma.project.findUnique({ where: { id: projectId } })
expect(collection?.filter).toMatchObject({
sourceFolderId: project?.rootFolderId,
searchFilter: { conditions: [], recursively: true },
})
})
@@ -1,4 +1,4 @@
import { expect, test } from './fixtures'
import { expect, test } from '../../fixtures'
import * as fs from 'fs'
import * as path from 'path'
import { fileURLToPath } from 'url'
@@ -19,57 +19,34 @@ test.afterAll(async () => {
}
})
test('should sign up and create a project with a cover image', async ({ page }) => {
const email = `test-${Date.now()}@shumai.local`
const password = 'Password123!'
// 1. Sign Up Flow
await page.goto('/signup')
await expect(page).toHaveURL(/\/signup/)
// Fill credentials
await page.fill('#email', email)
await page.fill('#password', password)
// Submit sign up
await page.click('button[type="submit"]')
// Wait for the automatic redirect to the team page (since only one team exists)
await expect(page).toHaveURL(/\/teams\/[^/]+/)
// 2. Create Project Flow
// Click the "Create Project" button on the dashboard
await page.click('button:has-text("Create Project")')
test('owner should create a project with a cover image', async ({ owner, prisma }) => {
const { page, teamId } = owner
const projectName = `My Project ${Date.now()}`
// Open the create project dialog
await page.getByRole('button', { name: 'Create Project' }).click()
const dialog = page.locator('[role="dialog"]')
await expect(dialog).toBeVisible()
// Fill project name
const projectName = `My Project ${Date.now()}`
// Fill in project details and upload a cover image
await dialog.locator('#name').fill(projectName)
// Set cover image
await dialog.locator('input[type="file"]').setInputFiles(tempImgPath)
// Wait for image upload completion and preview
await expect(dialog.locator('img[alt="Cover Preview"]')).toBeVisible({ timeout: 15_000 })
// Click submit in dialog
// Submit and land on the project page
await dialog.locator('button[type="submit"]').click()
// Wait for navigation to the project page
await expect(page).toHaveURL(/\/projects\/[^/]+/)
// Verify project name appears in the project view
await expect(page.locator(`text=${projectName}`).first()).toBeVisible()
// 3. Verification on Dashboard
// Navigate back to the dashboard via sidebar
await page.click('button[aria-label="Dashboard"]')
// Wait for dashboard URL
// Back on the home page the new project card is visible with its cover
await page.click('button[aria-label="Home"]')
await expect(page).toHaveURL(/\/teams\/[^/]+/)
// Verify the project card is visible and displays the cover image with project name alt text
await expect(page.locator(`img[alt="${projectName}"]`)).toBeVisible()
// The project was persisted for the owner's team
const project = await prisma.project.findFirst({ where: { name: projectName } })
expect(project).not.toBeNull()
expect(project?.teamId).toBe(teamId)
})
@@ -0,0 +1,36 @@
import { expect, test } from '../../fixtures'
import { fileCard } from '../../helpers/files'
test('owner creates a share link from a file context menu and lands on the share page', async ({
file,
prisma,
}) => {
const { page, projectId, fileName, fileId } = file
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, fileName)).toBeVisible()
// Create a share link for the file through the context menu
await fileCard(page, fileName).click({ button: 'right' })
await page.getByRole('menuitem', { name: 'Create Share Link' }).click()
// The toast offers a "View" action that navigates to the share page
const toast = page.locator('[data-sonner-toast]').filter({ hasText: 'Share link created' })
await expect(toast).toBeVisible()
await toast.getByRole('button', { name: 'View' }).click()
await expect(page).toHaveURL(/\/projects\/[^/]+\/shares\/[^/]+/)
// The shared file is listed in the share contents
await expect(fileCard(page, fileName)).toBeVisible()
// DB: the share exists and the file is symlinked under the share root
const share = await prisma.shareLink.findFirst({ where: { projectId } })
expect(share).not.toBeNull()
const symlink = await prisma.asset.findFirst({
where: { parentId: share?.rootFolderId, targetId: fileId },
})
expect(symlink).not.toBeNull()
expect(symlink?.type).toBe('symlink')
})
@@ -0,0 +1,45 @@
import { expect, test } from '../../fixtures'
import { fileCard } from '../../helpers/files'
test('owner deletes a file, finds it in recently deleted, and restores it', async ({
file,
prisma,
}) => {
const { page, projectId, fileName, fileId } = file
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, fileName)).toBeVisible()
// Delete through the context menu + confirmation dialog
await fileCard(page, fileName).click({ button: 'right' })
await page.getByRole('menuitem', { name: 'Delete' }).click()
const deleteDialog = page.getByRole('alertdialog')
await expect(deleteDialog.getByText('Delete Asset?')).toBeVisible()
await deleteDialog.getByRole('button', { name: 'Delete' }).click()
await expect(fileCard(page, fileName)).not.toBeVisible()
let asset = await prisma.asset.findUnique({ where: { id: fileId } })
expect(asset?.isDeleted).toBe(true)
expect(asset?.deletedAt).not.toBeNull()
// Navigate to the Recently Deleted view
await page.getByText('Recently Deleted').click()
await expect(page).toHaveURL(/\/recently-deleted/)
await expect(fileCard(page, fileName)).toBeVisible()
// Restore through the context menu
await fileCard(page, fileName).click({ button: 'right' })
await page.getByRole('menuitem', { name: 'Restore' }).click()
await expect(fileCard(page, fileName)).not.toBeVisible()
asset = await prisma.asset.findUnique({ where: { id: fileId } })
expect(asset?.isDeleted).toBe(false)
expect(asset?.deletedAt).toBeNull()
// Back at the project root the file is visible again
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, fileName)).toBeVisible()
})
@@ -0,0 +1,79 @@
import { expect, test } from '../../fixtures'
import { E2E_PASSWORD, uniqueEmail } from '../../helpers/auth'
import { apiCreateProject, uniqueProjectName } from '../../helpers/project'
import { generateInviteLink, openProjectMembersDialog } from '../../helpers/ui'
test('owner invites a member to a project; the member only sees the invited project', async ({
project,
browser,
prisma,
}) => {
const { page, context, teamId, projectId, projectName } = project
// A second project the invitee must NOT see
const secondProject = await apiCreateProject(context.request, teamId, uniqueProjectName())
// 1. Owner opens the project members dialog and generates an invite link
await page.goto(`/projects/${projectId}`)
await expect(page.getByTestId('project-members-trigger')).toBeVisible()
const dialog = await openProjectMembersDialog(page)
const inviteLink = await generateInviteLink(dialog)
const inviteCode = new URL(inviteLink).searchParams.get('inviteCode')
if (!inviteCode) {
throw new Error(`Invite link is missing the inviteCode param: ${inviteLink}`)
}
const origin = new URL(page.url()).origin
expect(inviteLink.startsWith(`${origin}/signup?inviteCode=`)).toBe(true)
// 2. The invitee signs up through the invite link in a brand-new session
const inviteeContext = await browser.newContext()
const inviteePage = await inviteeContext.newPage()
const inviteeEmail = uniqueEmail('project-invitee')
try {
await inviteePage.goto(inviteLink)
// The invite context banner is shown on the signup page
await expect(inviteePage.getByText(/invited you to join/)).toBeVisible()
await inviteePage.fill('#email', inviteeEmail)
await inviteePage.fill('#password', E2E_PASSWORD)
await inviteePage.click('button[type="submit"]')
// The invitee is redirected to the shared team page
await expect(inviteePage).toHaveURL(/\/teams\/[^/]+/)
// 3. The invitee only sees the invited project, not the second one
await expect(inviteePage.getByText(projectName, { exact: true })).toBeVisible()
await expect(inviteePage.getByText(secondProject.name, { exact: true })).not.toBeVisible()
// 4. Direct access to the other project is denied (API returns 403)
const res = await inviteeContext.request.get(`/api/projects/${secondProject.id}`)
expect(res.status()).toBe(403)
// 5. Opening the other project URL renders the router error boundary
await inviteePage.goto(`/projects/${secondProject.id}`)
await expect(inviteePage.getByText('Something went wrong!')).toBeVisible()
} finally {
await inviteeContext.close()
}
// 6. DB: the invitee is a project-scoped team member of exactly the invited project
const inviteeUser = await prisma.user.findUnique({ where: { email: inviteeEmail } })
expect(inviteeUser).not.toBeNull()
const tm = await prisma.teamMember.findUnique({
where: { teamIdUserId: { teamId, userId: inviteeUser!.id } },
})
expect(tm?.scope).toBe('project')
const projectMemberships = await prisma.projectMember.findMany({
where: { teamMemberId: tm!.id },
})
expect(projectMemberships).toHaveLength(1)
expect(projectMemberships[0].projectId).toBe(projectId)
const invite = await prisma.invite.findUnique({ where: { code: inviteCode } })
expect(invite?.used).toBe(true)
expect(invite?.projectId).toBe(projectId)
})
@@ -0,0 +1,66 @@
import { expect, test } from '../../fixtures'
import { seedFile, seedFolder } from '../../helpers/assets'
import { dragCard } from '../../helpers/dnd'
import { fileCard } from '../../helpers/files'
test('owner moves a file into a folder and a folder into another folder', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const folderA = await seedFolder(prisma, projectId, rootFolderId, `e2e-folder-a-${Date.now()}`)
const folderB = await seedFolder(prisma, projectId, rootFolderId, `e2e-folder-b-${Date.now()}`)
const file1 = await seedFile(prisma, projectId, rootFolderId, `e2e-move-file-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, file1.name)).toBeVisible()
await expect(fileCard(page, folderA.name)).toBeVisible()
await expect(fileCard(page, folderB.name)).toBeVisible()
// 1. Drag the file onto folder A -> the file moves into folder A
await dragCard(page, fileCard(page, file1.name), fileCard(page, folderA.name))
await expect
.poll(async () => {
const asset = await prisma.asset.findUnique({ where: { id: file1.id } })
return asset?.parentId
})
.toBe(folderA.id)
// The file is no longer in the project root
await expect(fileCard(page, file1.name)).not.toBeVisible()
// Opening folder A shows the moved file
await fileCard(page, folderA.name).dblclick()
await expect(page).toHaveURL(new RegExp(`/projects/${projectId}/folders/${folderA.id}`))
await expect(fileCard(page, file1.name)).toBeVisible()
// 2. Back at the root, drag folder A onto folder B -> folder A moves into folder B
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, folderA.name)).toBeVisible()
await expect(fileCard(page, folderB.name)).toBeVisible()
await dragCard(page, fileCard(page, folderA.name), fileCard(page, folderB.name))
await expect
.poll(async () => {
const asset = await prisma.asset.findUnique({ where: { id: folderA.id } })
return asset?.parentId
})
.toBe(folderB.id)
// Folder A is no longer in the project root
await expect(fileCard(page, folderA.name)).not.toBeVisible()
// Opening folder B shows folder A inside it
await fileCard(page, folderB.name).dblclick()
await expect(page).toHaveURL(new RegExp(`/projects/${projectId}/folders/${folderB.id}`))
await expect(fileCard(page, folderA.name)).toBeVisible()
// 3. DB: folder/file counters are kept in sync on the moved folders
const movedFolderA = await prisma.asset.findUnique({ where: { id: folderA.id } })
expect(movedFolderA?.fileCount).toBe(1) // file1 moved in
const movedFolderB = await prisma.asset.findUnique({ where: { id: folderB.id } })
expect(movedFolderB?.fileCount).toBe(1) // folderA moved in
})
@@ -0,0 +1,35 @@
import { expect, test } from '../../fixtures'
import { fileCard } from '../../helpers/files'
test('owner views a file and it appears in recents', async ({ file }) => {
const { page, projectId, fileName, fileId } = file
// Open the file details page to record view
await page.goto(`/projects/${projectId}/files/${fileId}`)
await expect(page.getByText(fileName).first()).toBeVisible()
// Navigate to Recents page via sidebar or direct URL
await page.goto(`/projects/${projectId}`)
await page.getByText('Recently Viewed').first().click()
await expect(page).toHaveURL(new RegExp(`/projects/${projectId}/recents`))
// The viewed file should be visible in recents
const card = fileCard(page, fileName)
await expect(card).toBeVisible()
// Toolbar should show hint text instead of action buttons
await expect(
page.getByText('This page displays the 100 most recent files you viewed in this project.'),
).toBeVisible()
await expect(page.getByRole('button', { name: 'Field' })).not.toBeVisible()
await expect(page.getByRole('button', { name: 'Search' })).not.toBeVisible()
await expect(page.getByText('AGENTS.md')).not.toBeVisible()
// Right-clicking the file should NOT show the context menu
await card.click({ button: 'right' })
await expect(page.getByRole('menuitem', { name: 'Delete' })).not.toBeVisible()
await expect(page.getByRole('menuitem', { name: 'Download' })).not.toBeVisible()
// The per-item overflow menu must also be unavailable on the read-only Recents page.
await expect(card.getByRole('button')).toHaveCount(0)
})
@@ -0,0 +1,69 @@
import { expect, test } from '../../fixtures'
import { openProjectMembersDialog } from '../../helpers/ui'
import { seedProjectMember } from '../../helpers/team'
test('owner deletes a member from project via the members dialog and project access is revoked', async ({
project,
browser,
prisma,
}) => {
const { page, teamId, projectId, context } = project
const member = await seedProjectMember(
prisma,
teamId,
projectId,
'editor',
'project',
browser,
context,
)
// Open project detail page and project members dialog
await page.goto(`/projects/${projectId}`)
const dialog = await openProjectMembersDialog(page)
const memberRow = dialog
.locator('div.flex.items-center.justify-between')
.filter({ hasText: member.email })
await expect(memberRow).toBeVisible()
// Click the remove (trash) button for this member
await memberRow.getByRole('button').last().click()
// Confirm removal in the alert dialog
const removeDialog = page.getByRole('alertdialog')
await expect(removeDialog).toBeVisible()
await expect(removeDialog.getByRole('heading', { name: 'Remove Member' })).toBeVisible()
await removeDialog.getByRole('button', { name: 'Remove' }).click()
// The member is removed from project members and now appears under "Add members from other projects"
await expect(memberRow.getByRole('button', { name: 'Add to Project' })).toBeVisible()
// DB verification: projectMember record is deleted
const pm = await prisma.projectMember.findUnique({
where: {
projectIdTeamMemberId: {
projectId,
teamMemberId: member.teamMemberId,
},
},
})
expect(pm).toBeNull()
// Functional verification: removed member can no longer access the project
const memberContext = member.context!
const memberPage = await memberContext.newPage()
try {
await memberPage.goto(`/teams/${teamId}`)
// API request to project returns 403 Forbidden
const res = await memberContext.request.get(`/api/projects/${projectId}`)
expect(res.status()).toBe(403)
// Direct navigation renders error boundary
await memberPage.goto(`/projects/${projectId}`)
await expect(memberPage.getByText('Something went wrong!')).toBeVisible()
} finally {
await memberContext.close()
}
})
@@ -0,0 +1,72 @@
import { expect, test } from '../../fixtures'
import { openProjectMembersDialog } from '../../helpers/ui'
import { seedTeamMember } from '../../helpers/team'
test('owner adds team member to project with role override and permissions enforce project role', async ({
project,
browser,
prisma,
}) => {
const { page, teamId, projectId, context } = project
// Seed a team-scoped member with role 'editor'
const member = await seedTeamMember(prisma, teamId, 'editor', browser, context)
// Open the project page and project members dialog
await page.goto(`/projects/${projectId}`)
const dialog = await openProjectMembersDialog(page)
// Find the team member in the "Add team member to project" section
const teamMemberSectionRow = dialog
.locator('div.flex.items-center.justify-between')
.filter({ hasText: member.email })
await expect(teamMemberSectionRow).toBeVisible()
// Select "Reviewer" role override from the dropdown in that row (initially "Editor")
await teamMemberSectionRow.getByRole('button', { name: /Editor/i }).click()
await page.getByRole('menuitemradio', { name: 'Reviewer' }).click()
// Click "Set Role" button to save the project role override
await teamMemberSectionRow.getByRole('button', { name: 'Set Role' }).click()
// DB verification: projectMember record exists with role 'reviewer'
await expect
.poll(
async () =>
prisma.projectMember.findUnique({
where: {
projectIdTeamMemberId: {
projectId,
teamMemberId: member.teamMemberId,
},
},
}),
{ timeout: 10000 },
)
.not.toBeNull()
const pm = await prisma.projectMember.findUnique({
where: {
projectIdTeamMemberId: {
projectId,
teamMemberId: member.teamMemberId,
},
},
})
expect(pm?.role).toBe('reviewer')
// Functional verification: team editor's permissions in this project are overridden to reviewer (edit returns 403)
const memberContext = member.context!
const memberPage = await memberContext.newPage()
try {
await memberPage.goto(`/teams/${teamId}`)
// Project edit action is rejected with 403 because the project role override is 'reviewer'
const res = await memberContext.request.put(`/api/projects/${projectId}`, {
data: { enableNotification: true },
})
expect(res.status()).toBe(403)
} finally {
await memberContext.close()
}
})
@@ -0,0 +1,59 @@
import { expect, test } from '../../fixtures'
import { apiUploadFile, fileCard } from '../../helpers/files'
test('owner saves a search result as a collection', async ({ project, prisma }) => {
const { page, context, teamId, rootFolderId, projectId } = project
const matching = 'quarterly-report-2024'
const other = 'annual-budget-2024'
for (const name of [matching, other]) {
await apiUploadFile(
context.request,
teamId,
rootFolderId,
name,
'application/octet-stream',
Buffer.alloc(0),
)
}
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, matching)).toBeVisible()
await expect(fileCard(page, other)).toBeVisible()
// Run a keyword search
await page.getByRole('button', { name: 'Search' }).click()
const dialog = page.getByRole('dialog')
await expect(dialog).toBeVisible()
await dialog.getByPlaceholder('Search records by name...').fill('report')
await dialog.getByRole('button', { name: 'Search' }).click()
await expect(dialog.getByText(matching, { exact: true })).toBeVisible()
await expect(dialog.getByText(other, { exact: true })).not.toBeVisible()
// Save the search result as a collection; the dialog closes and navigates
// to the new collection page
await dialog.getByRole('button', { name: 'Save as collection' }).click()
await expect(page).toHaveURL(new RegExp(`/projects/${projectId}/collections/[^/]+`))
await expect(page.getByText('Collection saved')).toBeVisible()
// The collection page is a files-only view of the saved search
await expect(fileCard(page, matching)).toBeVisible()
await expect(fileCard(page, other)).not.toBeVisible()
// DB: the collection stores the keyword search as its filter
const collection = await prisma.collection.findFirst({ where: { projectId } })
expect(collection).not.toBeNull()
expect(collection?.name).toBe('report')
expect(collection?.filter).toMatchObject({
sourceFolderId: rootFolderId,
searchFilter: {
conditions: [{ field: 'name', operator: 'contains', value: 'report' }],
recursively: true,
query: 'report',
isSemantic: false,
},
})
})
@@ -0,0 +1,162 @@
import type { APIRequestContext } from '@playwright/test'
import { expect, test } from '../../fixtures'
import { apiUploadFile, fileCard } from '../../helpers/files'
const BINARY_MIME = 'application/octet-stream'
/** Uploads an empty binary file into the project root through the API. */
async function uploadBinaryFile(
request: APIRequestContext,
teamId: string,
rootFolderId: string,
name: string,
size = 0,
): Promise<void> {
await apiUploadFile(request, teamId, rootFolderId, name, BINARY_MIME, Buffer.alloc(size))
}
test('owner searches the file list by an English name keyword', async ({ project }) => {
const { page, projectId } = project
const report2024 = 'quarterly-report-2024'
const report2025 = 'quarterly-report-2025'
const budget2024 = 'annual-budget-2024'
for (const name of [report2024, report2025, budget2024]) {
await uploadBinaryFile(project.context.request, project.teamId, project.rootFolderId, name)
}
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, report2024)).toBeVisible()
await expect(fileCard(page, report2025)).toBeVisible()
await expect(fileCard(page, budget2024)).toBeVisible()
// Open the search dialog from the toolbar
await page.getByRole('button', { name: 'Search' }).click()
const dialog = page.getByRole('dialog')
await expect(dialog).toBeVisible()
// Keyword-only search (no filter conditions)
await dialog.getByPlaceholder('Search records by name...').fill('report')
await dialog.getByRole('button', { name: 'Search' }).click()
await expect(dialog.getByText(report2024, { exact: true })).toBeVisible()
await expect(dialog.getByText(report2025, { exact: true })).toBeVisible()
await expect(dialog.getByText(budget2024, { exact: true })).not.toBeVisible()
// Apply the search to the file list
await dialog.getByRole('button', { name: 'Apply and Close' }).click()
await expect(dialog).not.toBeVisible()
await expect(fileCard(page, report2024)).toBeVisible()
await expect(fileCard(page, report2025)).toBeVisible()
await expect(fileCard(page, budget2024)).not.toBeVisible()
})
test('owner searches the file list by Chinese, Japanese, and Korean name keywords', async ({
project,
}) => {
const { page, projectId } = project
const zhDoc = '产品需求文档-2024'
const zhDesign = '产品设计文档-2025'
const jaDoc = '日本語ドキュメント'
const koDoc = '한국어문서'
for (const name of [zhDoc, zhDesign, jaDoc, koDoc]) {
await uploadBinaryFile(project.context.request, project.teamId, project.rootFolderId, name)
}
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, zhDoc)).toBeVisible()
const searchByName = async (keyword: string, visible: string[], hidden: string[]) => {
await page.getByRole('button', { name: 'Search' }).click()
const dialog = page.getByRole('dialog')
await expect(dialog).toBeVisible()
await dialog.getByPlaceholder('Search records by name...').fill(keyword)
await dialog.getByRole('button', { name: 'Search' }).click()
for (const name of visible) {
await expect(dialog.getByText(name, { exact: true })).toBeVisible()
}
for (const name of hidden) {
await expect(dialog.getByText(name, { exact: true })).not.toBeVisible()
}
await dialog.getByRole('button', { name: 'Apply and Close' }).click()
await expect(dialog).not.toBeVisible()
}
// Chinese keyword
await searchByName('需求', [zhDoc], [zhDesign, jaDoc, koDoc])
await expect(fileCard(page, zhDoc)).toBeVisible()
await expect(fileCard(page, zhDesign)).not.toBeVisible()
// Japanese keyword
await searchByName('ドキュメント', [jaDoc], [zhDoc, zhDesign, koDoc])
await expect(fileCard(page, jaDoc)).toBeVisible()
await expect(fileCard(page, zhDoc)).not.toBeVisible()
// Korean keyword
await searchByName('한국어', [koDoc], [zhDoc, zhDesign, jaDoc])
await expect(fileCard(page, koDoc)).toBeVisible()
await expect(fileCard(page, jaDoc)).not.toBeVisible()
})
test('owner searches the file list with a keyword and a filter condition', async ({ project }) => {
const { page, projectId } = project
const bigReport = 'quarterly-report-2024'
const smallReport = 'quarterly-report-2025'
await uploadBinaryFile(
project.context.request,
project.teamId,
project.rootFolderId,
bigReport,
1000,
)
await uploadBinaryFile(
project.context.request,
project.teamId,
project.rootFolderId,
smallReport,
10,
)
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, bigReport)).toBeVisible()
await expect(fileCard(page, smallReport)).toBeVisible()
// Open the search dialog from the toolbar
await page.getByRole('button', { name: 'Search' }).click()
const dialog = page.getByRole('dialog')
await expect(dialog).toBeVisible()
await dialog.getByPlaceholder('Search records by name...').fill('report')
// Add a "Size > 100" filter condition on top of the keyword
await dialog.getByRole('button', { name: '+ Add condition' }).click()
const comboboxes = dialog.getByRole('combobox')
await comboboxes.first().click()
await page.getByRole('option', { name: 'Size', exact: true }).click()
await comboboxes.nth(1).click()
await page.getByRole('option', { name: '>', exact: true }).click()
await dialog.locator('input[type="number"]').fill('100')
// The condition value is applied after the debounced input; the badge
// confirms the filter became active before we run the search.
await expect(dialog.getByText('1 active')).toBeVisible()
await dialog.getByRole('button', { name: 'Search' }).click()
await expect(dialog.getByText(bigReport, { exact: true })).toBeVisible()
await expect(dialog.getByText(smallReport, { exact: true })).not.toBeVisible()
// Apply the keyword + filter to the file list
await dialog.getByRole('button', { name: 'Apply and Close' }).click()
await expect(dialog).not.toBeVisible()
await expect(fileCard(page, bigReport)).toBeVisible()
await expect(fileCard(page, smallReport)).not.toBeVisible()
})
@@ -0,0 +1,51 @@
import { expect, test } from '../../fixtures'
import { apiAddAssetsToShare, apiCreateShare, fileCard } from '../../helpers/files'
test('owner protects a share with a password and guests must enter it to view', async ({
file,
browser,
prisma,
}) => {
const { context, page, projectId, fileName, fileId } = file
// Seed the share with the file through the API (setup, not the flow under test)
const share = await apiCreateShare(context.request, projectId, 'Shared Doc')
await apiAddAssetsToShare(context.request, share.id, [fileId])
await page.goto(`/projects/${projectId}/shares/${share.id}`)
await expect(fileCard(page, fileName)).toBeVisible()
// Open the Security accordion and enable a password
await page.getByRole('button', { name: 'Security' }).click()
const passwordRow = page.getByText('Password').locator('..')
await passwordRow.getByRole('switch').click()
const passwordInput = page.getByPlaceholder('Enter password')
await expect(passwordInput).toBeVisible()
await passwordInput.fill('secret123')
await page.getByRole('button', { name: 'Save' }).click()
await expect(page.getByText('Settings updated')).toBeVisible()
const saved = await prisma.shareLink.findUnique({ where: { id: share.id } })
expect(saved?.password).toBe('secret123')
// A guest visiting the public link hits the password gate
const guestContext = await browser.newContext()
const guestPage = await guestContext.newPage()
try {
await guestPage.goto(`/share/${share.id}`)
await expect(guestPage.getByText('Password Protected')).toBeVisible()
// Wrong password keeps the gate open with an error
await guestPage.getByPlaceholder('Enter password').fill('wrong-password')
await guestPage.getByRole('button', { name: 'Access Share' }).click()
await expect(guestPage.getByText('Incorrect password. Please try again.')).toBeVisible()
// Correct password grants access to the shared file
await guestPage.getByPlaceholder('Enter password').fill('secret123')
await guestPage.getByRole('button', { name: 'Access Share' }).click()
await expect(fileCard(guestPage, fileName)).toBeVisible()
} finally {
await guestContext.close()
}
})
@@ -0,0 +1,184 @@
import { expect, test } from '../../fixtures'
import {
apiAddAssetsToShare,
apiCreateShare,
apiEnableShareWatermark,
apiPublicDownloadUrl,
apiUpdateShare,
fileCard,
} from '../../helpers/files'
/**
* The watermarked + download-disabled combination is frequently used together:
* guests can preview the content but never download it, and any previewed proxy
* carries a watermark. This covers both watermarkable media types (image and
* video) and asserts, per type:
* - the public share list page exposes no download affordance,
* - the public share file detail page exposes no download affordance,
* - the proxy loaded in the detail page is the watermarked version, and
* - the download-url endpoint rejects guest download attempts with 403.
*/
for (const mediaType of ['image', 'video'] as const) {
// NOTE: `test.use` must be scoped per describe block — a bare `test.use`
// inside a `for` loop leaks the last iteration's value to every test.
test.describe(`${mediaType} share`, () => {
test.use({ fileOptions: { mediaType } })
// Transcoding (original + watermark) happens through the local workflow
// executor, so this test needs a much larger budget than the 60s default.
test.setTimeout(300_000)
test('with watermark + download disabled: no download buttons, watermarked proxy', async ({
file,
browser,
prisma,
}) => {
const { context, projectId, fileName, fileId } = file
// The watermark transcode re-encodes from the existing transcode slots, so
// the original transcode must finish first.
await expect
.poll(
async () => {
const asset = await prisma.asset.findUnique({ where: { id: fileId } })
return asset?.status
},
{ timeout: 60_000 },
)
.toBe('processed')
// Seed the share through the API (setup, not the flow under test)
const share = await apiCreateShare(context.request, projectId, 'Watermarked Share')
await apiAddAssetsToShare(context.request, share.id, [fileId])
await apiUpdateShare(context.request, share.id, { allowDownload: false })
await apiEnableShareWatermark(context.request, share.id)
// Wait for the watermark transcode to complete and the share to be ready
await expect
.poll(
async () => {
const link = await prisma.shareLink.findUnique({ where: { id: share.id } })
return link?.watermarkStatus
},
{ timeout: 120_000 },
)
.toBe('ready')
const saved = await prisma.shareLink.findUnique({ where: { id: share.id } })
expect(saved?.allowDownload).toBe(false)
expect(saved?.watermarkConfigId).not.toBeNull()
// The share root contains a symlink pointing at the real asset; the public
// detail page navigates with the symlink id (mirrors the real double-click flow).
const symlink = await prisma.asset.findFirst({
where: { parentId: share.rootFolderId, targetId: fileId },
})
expect(symlink).not.toBeNull()
// Original (unwatermarked) transcode keys, to prove they are never served
const asset = await prisma.asset.findUnique({ where: { id: fileId } })
const media = asset?.media as {
imageTranscodes?: Array<{ key: string }>
videoTranscodes?: Array<{ key: string }>
original?: { key: string } | null
} | null
const originalKeys =
(mediaType === 'image' ? media?.imageTranscodes : media?.videoTranscodes)?.map(
(t) => t.key,
) ?? []
const guestContext = await browser.newContext()
const guestPage = await guestContext.newPage()
try {
// ---------------- Public share list page ----------------
await guestPage.goto(`/share/${share.id}`)
await expect(fileCard(guestPage, fileName)).toBeVisible()
// Card context menu: no Download item
const card = fileCard(guestPage, fileName)
const kebab = card.locator('button', {
has: guestPage.locator('svg.lucide-ellipsis'),
})
await kebab.click()
await expect(guestPage.locator('[data-radix-menu-content]').first()).toBeAttached()
await expect(guestPage.getByRole('menuitem', { name: 'Download' })).toHaveCount(0)
await guestPage.keyboard.press('Escape')
// Wait for the Radix menu to fully unmount before interacting again:
// while its exit animation is running its dismiss layer can intercept
// the outside pointerdown and swallow the next click.
await guestPage
.waitForFunction(() => !document.querySelector('[data-radix-menu-content]'), null, {
timeout: 10_000,
})
.catch(() => {})
// Selection bar: selecting the file must not surface a Download button.
// Click the card checkbox (selects directly) instead of the card body:
// the preview area (video sprite scrubber) re-renders on hover and can
// race with a body click.
const checkbox = card.getByRole('checkbox')
await checkbox.click()
await expect(guestPage.getByText(/1 file selected/)).toBeVisible()
await expect(guestPage.getByRole('button', { name: 'Download' })).toHaveCount(0)
// ---------------- Public share file detail page ----------------
// Capture every network request so we can prove the loaded proxy is the
// watermarked one and no original transcode key is ever requested.
const watermarkedRequests: string[] = []
const allRequests: string[] = []
guestPage.on('request', (req) => {
const url = req.url()
allRequests.push(url)
if (url.includes('-watermark-')) watermarkedRequests.push(url)
})
const detailFileId = symlink!.id
await guestPage.goto(`/share/${share.id}/files/${detailFileId}`)
// The viewer (image canvas / video element) renders the proxy
if (mediaType === 'image') {
await expect(guestPage.locator('canvas').first()).toBeVisible({ timeout: 30_000 })
} else {
await expect(guestPage.locator('video').first()).toBeVisible({ timeout: 30_000 })
}
// The proxy actually loaded is the watermarked version, and no original
// (unwatermarked) transcode key was requested
await expect.poll(() => watermarkedRequests.length, { timeout: 30_000 }).toBeGreaterThan(0)
const leaked = allRequests.filter((url) => originalKeys.some((key) => url.includes(key)))
expect(leaked).toEqual([])
// No download affordance in the viewer control bar
await expect(guestPage.getByTitle('Download')).toHaveCount(0)
await expect(guestPage.getByTitle('Download original image')).toHaveCount(0)
await expect(guestPage.getByRole('button', { name: 'Download' })).toHaveCount(0)
// Top-nav file dropdown: no Download item (video: also no resolution submenu)
await guestPage.getByRole('button', { name: new RegExp(fileName) }).click()
await expect(guestPage.locator('[data-radix-menu-content]').first()).toBeAttached()
await expect(guestPage.getByRole('menuitem', { name: 'Download' })).toHaveCount(0)
if (mediaType === 'video') {
await expect(
guestPage.locator('[data-radix-menu-content]').getByText('Original', { exact: true }),
).toHaveCount(0)
await expect(
guestPage.locator('[data-radix-menu-content]').getByText('MP4', { exact: true }),
).toHaveCount(0)
}
// ---------------- Backend enforcement ----------------
// Guests cannot obtain a download URL even when they know the storage key
const originalKey = originalKeys[0] || media?.original?.key || ''
const download = await apiPublicDownloadUrl(
guestContext.request,
share.id,
detailFileId,
originalKey,
)
expect(download.status).toBe(403)
} finally {
await guestContext.close()
}
})
})
}
@@ -0,0 +1,85 @@
import { expect, test } from '../../fixtures'
import { fileCard } from '../../helpers/files'
test('owner uploads a file via the "Upload File" context menu action', async ({
project,
prisma,
}) => {
const { page, projectId } = project
const fileName = `e2e-upload-${Date.now()}.png`
// 5x5 PNG (uploading an image avoids the slow txt->pdf transcode path)
const pngBase64 =
'iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAIAAAACDbGyAAAAEElEQVR4nGNgmHkGBVHIBwBfNyLesMZaCgAAAABJRU5ErkJggg=='
await page.goto(`/projects/${projectId}`)
await expect(page.getByText('This folder is empty')).toBeVisible()
// Open the empty-area context menu and trigger the file picker
await page.getByText('This folder is empty').click({ button: 'right' })
const [fileChooser] = await Promise.all([
page.waitForEvent('filechooser'),
page.getByRole('menuitem', { name: 'Upload File' }).click(),
])
await fileChooser.setFiles({
name: fileName,
mimeType: 'image/png',
buffer: Buffer.from(pngBase64, 'base64'),
})
// The file card appears immediately (optimistic) and then gets processed
await expect(fileCard(page, fileName)).toBeVisible()
await expect
.poll(
async () => {
const asset = await prisma.asset.findFirst({ where: { name: fileName } })
return asset?.status
},
{ timeout: 30_000 },
)
.toBe('processed')
// DB: the file is stored in the project root folder
const asset = await prisma.asset.findFirst({ where: { name: fileName } })
expect(asset).not.toBeNull()
expect(asset!.type).toBe('file')
expect(asset!.projectId).toBe(projectId)
const projectRow = await prisma.project.findUnique({ where: { id: projectId } })
expect(asset!.parentId).toBe(projectRow?.rootFolderId)
})
test('owner uploads an unsupported file via context menu action', async ({ project, prisma }) => {
const { page, projectId } = project
const fileName = `e2e-upload-${Date.now()}.bin`
const binBuffer = Buffer.from([0x00, 0x01, 0x02, 0x03])
await page.goto(`/projects/${projectId}`)
await expect(page.getByText('This folder is empty')).toBeVisible()
await page.getByText('This folder is empty').click({ button: 'right' })
const [fileChooser] = await Promise.all([
page.waitForEvent('filechooser'),
page.getByRole('menuitem', { name: 'Upload File' }).click(),
])
await fileChooser.setFiles({
name: fileName,
mimeType: 'application/octet-stream',
buffer: binBuffer,
})
const card = fileCard(page, fileName)
await expect(card).toBeVisible()
await expect
.poll(
async () => {
const asset = await prisma.asset.findFirst({ where: { name: fileName } })
return asset?.status
},
{ timeout: 30_000 },
)
.toBe('processed')
// Verify that the file card shows the generic file icon and is not stuck in uploading state
await expect(card.locator('svg.lucide-file')).toBeVisible({ timeout: 10_000 })
})
@@ -0,0 +1,70 @@
import { expect, test } from '../../fixtures'
import { seedFile } from '../../helpers/assets'
import { dragCard } from '../../helpers/dnd'
import { fileCard } from '../../helpers/files'
test('owner moves a file onto a version stack to create a new version', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const fileA = await seedFile(prisma, projectId, rootFolderId, `e2e-ver-a-${Date.now()}`)
const fileB = await seedFile(prisma, projectId, rootFolderId, `e2e-ver-b-${Date.now()}`)
const fileC = await seedFile(prisma, projectId, rootFolderId, `e2e-ver-c-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, fileA.name)).toBeVisible()
await expect(fileCard(page, fileB.name)).toBeVisible()
await expect(fileCard(page, fileC.name)).toBeVisible()
// 1. First create a version stack by dropping file B onto file A
await dragCard(page, fileCard(page, fileB.name), fileCard(page, fileA.name))
await expect(page.getByText('v2', { exact: true })).toBeVisible()
await expect
.poll(async () => {
return prisma.asset.count({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
})
.toBe(1)
let stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
expect(stack.fileCount).toBe(2)
// 2. Drop file C onto the stack card -> a new version is added to the stack
// (the stack card currently displays the latest version's name, file B)
await dragCard(page, fileCard(page, fileC.name), fileCard(page, fileB.name))
// The newly added file becomes the latest version: the stack card shows file C
// with a v3 badge and file B's card disappears from the folder
await expect(page.getByText('v3', { exact: true })).toBeVisible()
await expect(fileCard(page, fileC.name)).toBeVisible()
await expect(fileCard(page, fileB.name)).not.toBeVisible()
// DB: the stack now has 3 versions, including file C
await expect
.poll(async () => {
const s = await prisma.asset.findFirst({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
return s?.fileCount
})
.toBe(3)
stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
const versions = await prisma.asset.findMany({
where: { parentId: stack.id, type: 'file' },
orderBy: { sortIndex: 'asc' },
})
expect(versions).toHaveLength(3)
expect(new Set(versions.map((v) => v.id))).toEqual(new Set([fileA.id, fileB.id, fileC.id]))
const movedFileC = await prisma.asset.findUnique({ where: { id: fileC.id } })
expect(movedFileC?.parentId).toBe(stack.id)
})
@@ -0,0 +1,48 @@
import { expect, test } from '../../fixtures'
import { seedFile } from '../../helpers/assets'
import { dragCard } from '../../helpers/dnd'
import { fileCard } from '../../helpers/files'
test('owner moves a file onto another file to create a version stack', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const fileA = await seedFile(prisma, projectId, rootFolderId, `e2e-stack-a-${Date.now()}`)
const fileB = await seedFile(prisma, projectId, rootFolderId, `e2e-stack-b-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, fileA.name)).toBeVisible()
await expect(fileCard(page, fileB.name)).toBeVisible()
// Drag file B onto file A -> a version stack is created with both files
await dragCard(page, fileCard(page, fileB.name), fileCard(page, fileA.name))
// The stack card shows the latest version's name (the dragged file) and a v2 badge;
// the original two file cards are gone from the folder
await expect(fileCard(page, fileB.name)).toBeVisible()
await expect(page.getByText('v2', { exact: true })).toBeVisible()
await expect(fileCard(page, fileA.name)).not.toBeVisible()
// DB: exactly one version stack in the project root with both files as versions
await expect
.poll(async () => {
return prisma.asset.count({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
})
.toBe(1)
const stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
expect(stack.fileCount).toBe(2)
const versions = await prisma.asset.findMany({
where: { parentId: stack.id, type: 'file' },
orderBy: { sortIndex: 'asc' },
})
expect(versions).toHaveLength(2)
expect(new Set(versions.map((v) => v.id))).toEqual(new Set([fileA.id, fileB.id]))
})
@@ -0,0 +1,233 @@
import { expect, test } from '../../fixtures'
import { seedFile } from '../../helpers/assets'
import { dragCard } from '../../helpers/dnd'
import { apiAddAssetsToShare, apiCreateShare, fileCard } from '../../helpers/files'
test('owner manages versions: removes a version from a 3-version stack', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const fileA = await seedFile(prisma, projectId, rootFolderId, `e2e-mng-a-${Date.now()}`)
const fileB = await seedFile(prisma, projectId, rootFolderId, `e2e-mng-b-${Date.now()}`)
const fileC = await seedFile(prisma, projectId, rootFolderId, `e2e-mng-c-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, fileA.name)).toBeVisible()
await expect(fileCard(page, fileB.name)).toBeVisible()
await expect(fileCard(page, fileC.name)).toBeVisible()
// 1. Create a 3-version stack: B on A -> stack (v2), then C on stack -> stack (v3)
await dragCard(page, fileCard(page, fileB.name), fileCard(page, fileA.name))
await expect(page.getByText('v2', { exact: true })).toBeVisible()
await dragCard(page, fileCard(page, fileC.name), fileCard(page, fileB.name))
await expect(page.getByText('v3', { exact: true })).toBeVisible()
const stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
expect(stack.fileCount).toBe(3)
// 2. Navigate to file viewer for the stack
await page.goto(`/projects/${projectId}/files/${stack.id}`)
// 3. Open breadcrumb menu and click "Manage versions..."
// Click on the asset name / badge dropdown trigger
await page.getByRole('button', { name: new RegExp(fileC.name) }).click()
// Hover or click on Versions sub-menu trigger
await page.getByRole('menuitem', { name: 'Versions', exact: true }).hover()
// Click "Manage versions..."
await page.getByRole('button', { name: /manage versions$/i }).click()
// 4. Verify Manage versions dialog is visible with all 3 versions
const dialog = page.getByRole('dialog')
await expect(dialog).toBeVisible()
await expect(dialog.getByText('Manage versions')).toBeVisible()
await expect(dialog.getByText(fileA.name)).toBeVisible()
await expect(dialog.getByText(fileB.name)).toBeVisible()
await expect(dialog.getByText(fileC.name)).toBeVisible()
// 5. Remove fileB from stack: hover its row and click Remove from stack
const rowB = dialog.locator('div.group', { hasText: fileB.name })
await rowB.hover()
const moreButton = rowB.getByRole('button', { name: 'More options' })
await moreButton.click()
await page.getByRole('menuitem', { name: /remove from stack/i }).click()
// 6. Verify toast and that fileB is removed from dialog
await expect(page.getByText(/version removed from stack/i)).toBeVisible()
await expect(dialog.getByText(fileB.name)).not.toBeVisible()
await expect(dialog.getByText(fileA.name)).toBeVisible()
await expect(dialog.getByText(fileC.name)).toBeVisible()
// DB: stack now has 2 versions (fileA and fileC), and fileB is standalone in root folder
await expect
.poll(async () => {
const s = await prisma.asset.findUnique({ where: { id: stack.id } })
return s?.fileCount
})
.toBe(2)
const updatedFileB = await prisma.asset.findUnique({ where: { id: fileB.id } })
expect(updatedFileB?.parentId).toBe(rootFolderId)
})
test('owner removes version from 2-version stack dissolving the stack into standalone files', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const fileA = await seedFile(prisma, projectId, rootFolderId, `e2e-dis-a-${Date.now()}`)
const fileB = await seedFile(prisma, projectId, rootFolderId, `e2e-dis-b-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
await dragCard(page, fileCard(page, fileB.name), fileCard(page, fileA.name))
await expect(page.getByText('v2', { exact: true })).toBeVisible()
const stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
// Open file view
await page.goto(`/projects/${projectId}/files/${stack.id}`)
// Open breadcrumb menu and click "Manage versions..."
await page.getByRole('button', { name: new RegExp(fileB.name) }).click()
await page.getByRole('menuitem', { name: 'Versions', exact: true }).hover()
await page.getByRole('button', { name: /manage versions$/i }).click()
const dialog = page.getByRole('dialog')
await expect(dialog).toBeVisible()
// Remove fileB -> stack dissolves -> only fileA remains
const rowB = dialog.locator('div.group', { hasText: fileB.name })
await rowB.hover()
const moreButton = rowB.getByRole('button', { name: 'More options' })
await moreButton.click()
await page.getByRole('menuitem', { name: /remove from stack/i }).click()
// Dialog should close automatically when stack dissolves
await expect(dialog).not.toBeVisible()
// DB: stack asset is deleted
await expect
.poll(async () => {
const s = await prisma.asset.findUnique({ where: { id: stack.id } })
return s === null
})
.toBe(true)
// Both fileA and fileB are now standalone files in rootFolder
const updatedFileA = await prisma.asset.findUnique({ where: { id: fileA.id } })
const updatedFileB = await prisma.asset.findUnique({ where: { id: fileB.id } })
expect(updatedFileA?.parentId).toBe(rootFolderId)
expect(updatedFileB?.parentId).toBe(rootFolderId)
})
test('dissolving a shared version stack renders the remaining standalone file name on the share page', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const fileA = await seedFile(prisma, projectId, rootFolderId, `e2e-shd-a-${Date.now()}`)
const fileB = await seedFile(prisma, projectId, rootFolderId, `e2e-shd-b-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
await dragCard(page, fileCard(page, fileB.name), fileCard(page, fileA.name))
await expect(page.getByText('v2', { exact: true })).toBeVisible()
const stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
// Share the version stack
await fileCard(page, fileB.name).click({ button: 'right' })
await page.getByRole('menuitem', { name: 'Create Share Link' }).click()
const toast = page.locator('[data-sonner-toast]').filter({ hasText: 'Share link created' })
await expect(toast).toBeVisible()
const share = await prisma.shareLink.findFirstOrThrow({
where: { projectId },
orderBy: { createdAt: 'desc' },
})
// Open file viewer for the stack
await page.goto(`/projects/${projectId}/files/${stack.id}`)
// Open breadcrumb menu and click "Manage versions"
await page.getByRole('button', { name: new RegExp(fileB.name) }).click()
await page.getByRole('menuitem', { name: 'Versions', exact: true }).hover()
await page.getByRole('button', { name: /manage versions$/i }).click()
const dialog = page.getByRole('dialog')
await expect(dialog).toBeVisible()
// Remove fileB -> stack dissolves -> only fileA remains
const rowB = dialog.locator('div.group', { hasText: fileB.name })
await rowB.hover()
const moreButton = rowB.getByRole('button', { name: 'More options' })
await moreButton.click()
await page.getByRole('menuitem', { name: /remove from stack/i }).click()
await expect(dialog).not.toBeVisible()
// Navigate to public share page: verify fileA is displayed with its non-empty name
await page.goto(`/share/${share.id}`)
await expect(fileCard(page, fileA.name)).toBeVisible()
await expect(fileCard(page, fileB.name)).not.toBeVisible()
})
test('moving a shared file into a shared version stack leaves only 1 stack card on the share page', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const fileA = await seedFile(prisma, projectId, rootFolderId, `e2e-mvsh-a-${Date.now()}`)
const fileB = await seedFile(prisma, projectId, rootFolderId, `e2e-mvsh-b-${Date.now()}`)
const fileC = await seedFile(prisma, projectId, rootFolderId, `e2e-mvsh-c-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
// 1. Create a version stack from file B and file A (stack has v2, displays file B)
await dragCard(page, fileCard(page, fileB.name), fileCard(page, fileA.name))
await expect(page.getByText('v2', { exact: true })).toBeVisible()
const stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
// 2. Create a share link and add both stack AB and file C to it
const share = await apiCreateShare(project.context.request, projectId, `Test Share ${Date.now()}`)
await apiAddAssetsToShare(project.context.request, share.id, [stack.id, fileC.id])
// Verify initial share page has both stack and file C
await page.goto(`/share/${share.id}`)
await expect(fileCard(page, fileB.name)).toBeVisible()
await expect(fileCard(page, fileC.name)).toBeVisible()
// 3. In project page, drop file C onto stack card (displays file B)
await page.goto(`/projects/${projectId}`)
await dragCard(page, fileCard(page, fileC.name), fileCard(page, fileB.name))
await expect(page.getByText('v3', { exact: true })).toBeVisible()
// 4. Return to public share page: now only 1 card exists (the stack showing file C with v3)
await page.goto(`/share/${share.id}`)
await expect(page.getByText('v3', { exact: true })).toBeVisible()
await expect(fileCard(page, fileC.name)).toBeVisible()
await expect(fileCard(page, fileB.name)).not.toBeVisible()
// DB: share rootFolder fileCount is 1 and exactly 1 symlink exists
const symlinks = await prisma.asset.findMany({
where: { parentId: share.rootFolderId, type: 'symlink' },
})
expect(symlinks).toHaveLength(1)
expect(symlinks[0].targetId).toBe(stack.id)
const updatedShareRoot = await prisma.asset.findUnique({ where: { id: share.rootFolderId } })
expect(updatedShareRoot?.fileCount).toBe(1)
})
@@ -0,0 +1,54 @@
import { expect, test } from '../../fixtures'
import { openMembersDialog } from '../../helpers/ui'
import { seedTeamMember } from '../../helpers/team'
test('owner changes a team member role in the members dialog and permissions update', async ({
owner,
browser,
prisma,
}) => {
const { page, teamId, context } = owner
const member = await seedTeamMember(prisma, teamId, 'reviewer', browser, context)
// Reload so the members dialog fetches the newly seeded member
await page.reload()
const dialog = await openMembersDialog(page)
const memberRow = dialog
.locator('div.flex.items-center.justify-between')
.filter({ hasText: member.email })
await expect(memberRow).toBeVisible()
// Click role dropdown trigger for this member (currently Reviewer)
await memberRow.getByRole('button', { name: /Reviewer/i }).click()
// Select "Editor" from the dropdown menu
await page.getByRole('menuitemradio', { name: 'Editor' }).click()
// Verify UI displays updated role "Editor"
await expect(memberRow.getByRole('button', { name: /Editor/i })).toBeVisible()
// DB verification: teamMember role is updated to editor
const tm = await prisma.teamMember.findUnique({
where: { teamIdUserId: { teamId, userId: member.userId } },
})
expect(tm?.role).toBe('editor')
// Functional verification: member can now create projects in their pre-authenticated session
const memberContext = member.context!
const memberPage = await memberContext.newPage()
try {
await memberPage.goto(`/teams/${teamId}`)
// Verify "Create Project" button is visible for editor
await expect(memberPage.getByRole('button', { name: 'Create Project' })).toBeVisible()
// Verify project creation API succeeds for editor
const res = await memberContext.request.post(`/api/teams/${teamId}/projects`, {
data: { name: `Editor Project ${Date.now()}` },
})
expect(res.status()).toBe(200)
} finally {
await memberContext.close()
}
})
@@ -0,0 +1,58 @@
import { expect, test } from '../../fixtures'
import { E2E_PASSWORD, uniqueEmail } from '../../helpers/auth'
import { generateInviteLink, openMembersDialog } from '../../helpers/ui'
test('owner invites a member via invite link and the invitee joins', async ({
owner,
browser,
prisma,
}) => {
const { page, teamId } = owner
// 1. Open the team members dialog and generate an invite link
const dialog = await openMembersDialog(page)
const inviteLink = await generateInviteLink(dialog)
const inviteCode = new URL(inviteLink).searchParams.get('inviteCode')
if (!inviteCode) {
throw new Error(`Invite link is missing the inviteCode param: ${inviteLink}`)
}
const origin = new URL(page.url()).origin
expect(inviteLink.startsWith(`${origin}/signup?inviteCode=`)).toBe(true)
// 2. The invitee opens the link in a brand-new session (no cookies)
const inviteeContext = await browser.newContext()
const inviteePage = await inviteeContext.newPage()
const inviteeEmail = uniqueEmail('invitee')
try {
await inviteePage.goto(inviteLink)
// The invite context banner is shown on the signup page
await expect(inviteePage.getByText(/invited you to join/)).toBeVisible()
// 3. The invitee signs up through the invite flow
await inviteePage.fill('#email', inviteeEmail)
await inviteePage.fill('#password', E2E_PASSWORD)
await inviteePage.click('button[type="submit"]')
// The invitee is redirected to the shared team
await expect(inviteePage).toHaveURL(/\/teams\/[^/]+/)
} finally {
await inviteeContext.close()
}
// 4. The owner reloads and sees the new member in the members dialog
await page.reload()
const membersDialog = await openMembersDialog(page)
await expect(membersDialog).toContainText(inviteeEmail)
// 5. DB: the invitee is a team member and the invite was consumed
const membershipCount = await prisma.teamMember.count({ where: { teamId } })
expect(membershipCount).toBe(2)
const invite = await prisma.invite.findUnique({ where: { code: inviteCode } })
expect(invite?.used).toBe(true)
const inviteeUser = await prisma.user.findUnique({ where: { email: inviteeEmail } })
expect(inviteeUser).not.toBeNull()
})
@@ -0,0 +1,126 @@
import { expect, test } from '../../fixtures'
test('owner manages resource quotas in settings and resets usage from the dashboard', async ({
owner,
prisma,
}) => {
const { page, teamId, email } = owner
// 1. Navigate to team settings
await page.goto(`/teams/${teamId}/settings`)
// 2. Click the Quotas sidebar tab
await page.getByRole('button', { name: /Quotas|配额/i }).click()
// Verify URL contains #quotas and remains after reload
expect(page.url()).toContain('#quotas')
await page.reload()
expect(page.url()).toContain('#quotas')
// 3. Verify Quotas settings card is displayed with empty state
await expect(page.getByRole('heading', { name: /^(Resource Quotas|资源配额)$/i })).toBeVisible()
await expect(page.getByText(/No resource quotas configured|暂未配置资源配额/i)).toBeVisible()
// 4. Open Create Quota Dialog
await page
.getByRole('button', { name: /Add Quota|添加配额/i })
.first()
.click()
await expect(page.getByText(/Create Quota Rule|创建配额规则/i)).toBeVisible()
// 5. Fill and submit new quota rule
await page.fill('#quota-limit', '50000')
await page.getByRole('button', { name: /^(Create|创建)$/i }).click()
// 6. Verify quota rule card appears in the list
await expect(page.getByText(/50,000 tokens|50000 tokens/i)).toBeVisible()
await expect(page.getByText(/Each Member|每个成员/i)).toBeVisible()
// Verify in database
const createdRule = await prisma.quotaRule.findFirst({
where: { teamId, limit: 50000 },
})
expect(createdRule).not.toBeNull()
expect(createdRule?.enabled).toBe(true)
// 7. Usage is now monitored from the dashboard, not the settings rule card.
await expect(page.getByRole('button', { name: /View Usage|查看使用量/i })).not.toBeVisible()
// 8. Toggle quota rule switch
await page.getByRole('switch').click()
await expect(page.getByRole('switch')).not.toBeChecked()
const disabledRule = await prisma.quotaRule.findFirst({
where: { id: createdRule?.id },
})
expect(disabledRule?.enabled).toBe(false)
// Toggle back on
await page.getByRole('switch').click()
await expect(page.getByRole('switch')).toBeChecked()
// 9. Open Edit Dialog by clicking the quota card
await page.getByText(/50,000 tokens|50000 tokens/i).click()
await expect(page.getByText(/Edit Quota Rule|编辑配额规则/i)).toBeVisible()
await expect(page.getByRole('button', { name: /Each Member|每个成员/i })).toBeDisabled()
await expect(page.locator('#quota-resource')).toBeDisabled()
// 10. Update limit
await page.fill('#quota-limit', '75000')
await page.getByRole('button', { name: /^(Save|保存)$/i }).click()
// Verify updated limit in UI and DB
await expect(page.getByText(/75,000 tokens|75000 tokens/i)).toBeVisible()
const updatedRule = await prisma.quotaRule.findFirst({
where: { id: createdRule?.id },
})
expect(updatedRule?.limit).toBe(75000)
// 11. Verify quota usage and reset it from the dashboard
const ownerUser = await prisma.user.findUnique({ where: { email } })
expect(ownerUser).not.toBeNull()
await prisma.quotaRecord.create({
data: {
ruleId: createdRule?.id as string,
teamId,
userId: ownerUser?.id,
periodStart: new Date(),
periodEnd: new Date(Date.now() + 24 * 60 * 60 * 1000),
consumed: 100,
},
})
await page.goto(`/teams/${teamId}/dashboard`)
await page.getByRole('button', { name: /Quotas|配额/i }).click()
await expect(
page.getByText(/Monitor live quota usage|在仪表盘中监控实时配额使用情况/),
).toBeVisible()
const quotaRuleButton = page.getByRole('button', { name: /Expand quota rule|展开配额规则/i })
await quotaRuleButton.click()
await expect(page.getByText(/^100 \/ 75,000 tokens$|^100 \/ 75000 tokens$/i)).toBeVisible()
await page.getByRole('button', { name: /Reset Usage|重置使用量/i }).click()
await expect(page.getByRole('alertdialog')).toBeVisible()
await page.getByRole('button', { name: /Confirm|确认/i }).click()
await expect(page.getByText(/^0 \/ 75,000 tokens$|^0 \/ 75000 tokens$/i)).toBeVisible()
const resetRecord = await prisma.quotaRecord.findFirst({
where: { ruleId: createdRule?.id, userId: ownerUser?.id },
})
expect(resetRecord?.consumed).toBe(0)
// 12. Delete quota rule from settings
await page.goto(`/teams/${teamId}/settings`)
await page.getByRole('button', { name: /Quotas|配额/i }).click()
await page.getByRole('button', { name: 'Quota actions' }).first().click()
await page.getByRole('menuitem', { name: /Delete|删除/i }).click()
await page
.getByRole('button', { name: /Delete|删除/i })
.last()
.click()
// Verify empty state returns
await expect(page.getByText(/No resource quotas configured|暂未配置资源配额/i)).toBeVisible()
const deletedRule = await prisma.quotaRule.findFirst({
where: { id: createdRule?.id },
})
expect(deletedRule).toBeNull()
})
@@ -0,0 +1,38 @@
import { expect, test } from '../../fixtures'
import { openMembersDialog } from '../../helpers/ui'
import { seedTeamMember } from '../../helpers/team'
test('owner removes a member via the members dialog and the membership is deleted', async ({
owner,
prisma,
}) => {
const { page, teamId } = owner
const member = await seedTeamMember(prisma, teamId, 'reviewer')
// Reload so the members dialog fetches the newly seeded member
await page.reload()
const dialog = await openMembersDialog(page)
const memberRow = dialog
.locator('div.flex.items-center.justify-between')
.filter({ hasText: member.email })
await expect(memberRow).toBeVisible()
// The last button in the row is the remove (trash) button
await memberRow.getByRole('button').last().click()
const removeDialog = page.getByRole('alertdialog')
await expect(removeDialog).toBeVisible()
await expect(removeDialog.getByRole('heading', { name: 'Remove Member' })).toBeVisible()
await removeDialog.getByRole('button', { name: 'Remove' }).click()
// The member row disappears from the dialog
await expect(memberRow).not.toBeVisible()
// DB: the owner is the only remaining team member, but the user still exists
const remainingMembers = await prisma.teamMember.count({ where: { teamId } })
expect(remainingMembers).toBe(1)
const user = await prisma.user.findUnique({ where: { id: member.userId } })
expect(user).not.toBeNull()
})
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/web-app",
"version": "0.3.2",
"version": "0.4.2",
"private": true,
"type": "module",
"dependencies": {
+25
View File
@@ -13,12 +13,37 @@ import { initTranscodeWorkflows } from '@shumai/transcode'
import { workflowService } from '@shumai/workflow-core'
import { handleDaemonCommands } from '@shumai/core/src/utils/daemon'
import { authService } from '@shumai/core/src/auth/auth'
if (process.argv.includes('--check')) {
console.log('✅ Web app evaluated successfully!')
process.exit(0)
}
const cliArgs = process.argv.slice(2)
const resetCmdIndex = cliArgs.findIndex(
(arg) => arg === 'reset-password' || arg === 'reset-passowrd',
)
if (resetCmdIndex !== -1) {
const email = cliArgs[resetCmdIndex + 1]
if (!email || email.startsWith('-')) {
console.error('Error: Please provide a valid email address.')
console.error('Usage: shumai reset-password <email>')
process.exit(1)
}
try {
const link = await authService.generatePasswordResetLink(email)
console.log(link)
process.exit(0)
} catch (err: unknown) {
const message = err instanceof Error ? err.message : 'Unknown error'
console.error(`Error: ${message}`)
process.exit(1)
}
}
async function run() {
// Initialize workflows and activities for local executor mode
initAgentWorkflows()
+42 -27
View File
@@ -33,7 +33,7 @@
},
"apps/agent": {
"name": "@shumai/agent-app",
"version": "0.2.3",
"version": "0.3.7",
"bin": {
"shumai-agent": "./src/index.ts",
},
@@ -45,7 +45,7 @@
},
"apps/cli": {
"name": "@shumai/cli",
"version": "0.2.3",
"version": "0.3.7",
"bin": {
"shumai-cli": "./src/index.ts",
},
@@ -60,7 +60,7 @@
},
"apps/transcode": {
"name": "@shumai/transcode-app",
"version": "0.2.3",
"version": "0.3.7",
"bin": {
"shumai-transcode": "./src/index.ts",
},
@@ -72,7 +72,7 @@
},
"apps/web": {
"name": "@shumai/web-app",
"version": "0.2.3",
"version": "0.3.7",
"dependencies": {
"@shumai/agent": "workspace:*",
"@shumai/api": "workspace:*",
@@ -92,22 +92,22 @@
},
"packages/agent": {
"name": "@shumai/agent",
"version": "0.2.3",
"version": "0.3.7",
"dependencies": {
"@anthropic-ai/sandbox-runtime": "0.0.66",
"@earendil-works/pi-agent-core": "0.78.0",
"@earendil-works/pi-ai": "0.80.7",
"@earendil-works/pi-agent-core": "0.82.1",
"@earendil-works/pi-ai": "0.82.1",
"@google/genai": "1.52.0",
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@shumai/workflow-core": "workspace:*",
"@sinclair/typebox": "0.34.49",
"@temporalio/activity": "1.17.2",
"@temporalio/worker": "1.17.2",
"@temporalio/workflow": "1.17.2",
"adm-zip": "0.5.17",
"jittered-fractional-indexing": "1.0.1",
"typebox": "1.1.38",
"ulid": "3.0.2",
"zod": "4.4.3",
},
@@ -117,7 +117,7 @@
},
"packages/api": {
"name": "@shumai/api",
"version": "0.2.3",
"version": "0.3.7",
"dependencies": {
"@hono/zod-validator": "0.7.6",
"@shumai/core": "workspace:*",
@@ -131,9 +131,11 @@
},
"packages/core": {
"name": "@shumai/core",
"version": "0.2.3",
"version": "0.3.7",
"dependencies": {
"@earendil-works/pi-agent-core": "0.78.0",
"@earendil-works/pi-agent-core": "0.82.1",
"@earendil-works/pi-ai": "0.82.1",
"@modelcontextprotocol/client": "2.0.0",
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@sinclair/typebox": "0.34.49",
@@ -150,11 +152,13 @@
"pino": "10.3.1",
"sharp": "0.34.5",
"tweetnacl": "1.0.3",
"typebox": "1.1.38",
"ulid": "3.0.2",
"yaml": "2.9.0",
"zod": "4.4.3",
},
"devDependencies": {
"@modelcontextprotocol/server": "2.0.0",
"@types/adm-zip": "0.5.8",
"@types/extract-zip": "2.0.3",
"@types/pdfkit": "0.17.6",
@@ -162,9 +166,9 @@
},
"packages/db": {
"name": "@shumai/db",
"version": "0.2.3",
"version": "0.3.7",
"dependencies": {
"@earendil-works/pi-agent-core": "0.78.0",
"@earendil-works/pi-agent-core": "0.82.1",
"@prisma/adapter-pg": "7.8.0",
"@prisma/client": "7.8.0",
"pg": "8.21.0",
@@ -178,17 +182,17 @@
},
"packages/dtos": {
"name": "@shumai/dtos",
"version": "0.2.3",
"version": "0.3.7",
"dependencies": {
"@earendil-works/pi-agent-core": "0.78.0",
"@earendil-works/pi-agent-core": "0.82.1",
"zod": "4.4.3",
},
},
"packages/e2e": {
"name": "@shumai/e2e",
"version": "0.2.3",
"version": "0.3.7",
"dependencies": {
"@earendil-works/pi-agent-core": "0.78.0",
"@earendil-works/pi-agent-core": "0.82.1",
"@google/genai": "1.52.0",
"@shumai/agent": "workspace:*",
"@shumai/core": "workspace:*",
@@ -201,11 +205,11 @@
},
"packages/tableprinter": {
"name": "@shumai/tableprinter",
"version": "0.2.3",
"version": "0.3.7",
},
"packages/transcode": {
"name": "@shumai/transcode",
"version": "0.2.3",
"version": "0.3.7",
"dependencies": {
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
@@ -219,7 +223,7 @@
},
"packages/webui": {
"name": "@shumai/webui",
"version": "0.2.3",
"version": "0.3.7",
"dependencies": {
"@base-ui/react": "1.5.0",
"@dnd-kit/abstract": "0.4.0",
@@ -317,7 +321,7 @@
},
"packages/workflow-core": {
"name": "@shumai/workflow-core",
"version": "0.2.3",
"version": "0.3.7",
"dependencies": {
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
@@ -329,6 +333,9 @@
},
},
},
"patchedDependencies": {
"@earendil-works/pi-ai@0.82.1": "patches/@earendil-works%2Fpi-ai@0.82.1.patch",
},
"packages": {
"@alloc/quick-lru": ["@alloc/quick-lru@5.2.0", "", {}, "sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw=="],
@@ -464,9 +471,9 @@
"@dnd-kit/state": ["@dnd-kit/state@0.4.0", "", { "dependencies": { "@preact/signals-core": "^1.10.0", "tslib": "^2.6.2" } }, "sha512-vVdwOY9VsYdMNa7Z0xQhTXlzHqCcCugGuoM1kzvZhnZ0tYVPRdmIhWfeO6Y2ZoN92JwYAyJRRNl4ICkEe2mneg=="],
"@earendil-works/pi-agent-core": ["@earendil-works/pi-agent-core@0.78.0", "", { "dependencies": { "@earendil-works/pi-ai": "^0.78.0", "ignore": "7.0.5", "typebox": "1.1.38", "yaml": "2.9.0" } }, "sha512-xhWd59Qzd8yO88gYQw2S4dEQstJJEiUtxRP01//YzVJ61jCtUASMfcyAmYhgGYR4Onp7GmwEAbBBGOiV6Iwk9g=="],
"@earendil-works/pi-agent-core": ["@earendil-works/pi-agent-core@0.82.1", "", { "dependencies": { "@earendil-works/pi-ai": "^0.82.1", "diff": "8.0.4", "ignore": "7.0.5", "typebox": "1.1.38", "yaml": "2.9.0" } }, "sha512-Z3kloziJIE2dmrisRckZX8zDca/gIv9/YdFAzeoqpHiLV2wsni6bL4hInNSjVKLbqT+4kqLIkph2JQLKvSepjg=="],
"@earendil-works/pi-ai": ["@earendil-works/pi-ai@0.80.7", "", { "dependencies": { "@anthropic-ai/sdk": "0.91.1", "@aws-sdk/client-bedrock-runtime": "3.1048.0", "@google/genai": "1.52.0", "@mistralai/mistralai": "2.2.6", "@opentelemetry/api": "1.9.0", "@smithy/node-http-handler": "4.7.3", "http-proxy-agent": "7.0.2", "https-proxy-agent": "7.0.6", "openai": "6.26.0", "partial-json": "0.1.7", "typebox": "1.1.38" }, "bin": { "pi-ai": "dist/cli.js" } }, "sha512-8RLKLwe5TFM9kKFMNu/lTzveduq4GxZbnlG6ba8FAhLeb5wJP4zbj1eBumKBRvggpFQnW5R/Vo2a8zTlHsV9SQ=="],
"@earendil-works/pi-ai": ["@earendil-works/pi-ai@0.82.1", "", { "dependencies": { "@anthropic-ai/sdk": "0.91.1", "@aws-sdk/client-bedrock-runtime": "3.1048.0", "@google/genai": "1.52.0", "@mistralai/mistralai": "2.2.6", "@opentelemetry/api": "1.9.0", "@smithy/node-http-handler": "4.7.3", "http-proxy-agent": "7.0.2", "https-proxy-agent": "7.0.6", "openai": "6.26.0", "partial-json": "0.1.7", "typebox": "1.1.38" }, "bin": { "pi-ai": "dist/cli.js" } }, "sha512-3WFYRhEp3lQB3444EhPMBcM7zSaEUE3eJgHOR7s4081NLqbw/FsWilIKWXSua0Gv3sRr7m9xMidR3pPDE7jI/A=="],
"@electric-sql/pglite": ["@electric-sql/pglite@0.4.1", "", {}, "sha512-mZ9NzzUSYPOCnxHH1oAHPRzoMFJHY472raDKwXl/+6oPbpdJ7g8LsCN4FSaIIfkiCKHhb3iF/Zqo3NYxaIhU7Q=="],
@@ -734,6 +741,12 @@
"@mistralai/mistralai": ["@mistralai/mistralai@2.2.6", "", { "dependencies": { "@opentelemetry/semantic-conventions": "^1.40.0", "ws": "^8.18.0", "zod": "^3.25.0 || ^4.0.0", "zod-to-json-schema": "^3.25.0" }, "peerDependencies": { "@opentelemetry/api": "^1.9.0" }, "optionalPeers": ["@opentelemetry/api"] }, "sha512-W8pX7zHxjJvMIpw8JMxeJEleapXX0Q9NPszdNzqkM3MIEoIGPObdodujj+WHteXEvGfaP/AMwlNyRfEzSY6dQQ=="],
"@modelcontextprotocol/client": ["@modelcontextprotocol/client@2.0.0", "", { "dependencies": { "@modelcontextprotocol/core": "2.0.0", "cross-spawn": "^7.0.5", "eventsource": "^3.0.2", "eventsource-parser": "^3.0.0", "jose": "^6.1.3", "pkce-challenge": "^5.0.0", "zod": "^4.2.0" } }, "sha512-8f1OghQ2rjzIOfqgUCP+8GiUWqRs89njoWLNqAe8kWmDePv3s1fZXseej+QXemssEuuOvLLmLO/kqM3IQHtISw=="],
"@modelcontextprotocol/core": ["@modelcontextprotocol/core@2.0.0", "", { "dependencies": { "zod": "^4.2.0" } }, "sha512-pJCEwGG7Lfr/+PQp9ZTwKXNeO5wzbfKL7H3MYpCorM4oFBoQrdjnBgEoqG+RjhsvS1FKrDbKux+M1HhlnGWqcA=="],
"@modelcontextprotocol/server": ["@modelcontextprotocol/server@2.0.0", "", { "dependencies": { "@modelcontextprotocol/core": "2.0.0", "zod": "^4.2.0" } }, "sha512-YhHWdHfpFMQfd0prsEnxKeS3Qz3ytIGmsS0sth4KDjnacIT7hxk6hXHkJ9KysxlkvTM+WZAtQbbcUhdoP4Hvtw=="],
"@napi-rs/canvas": ["@napi-rs/canvas@1.0.2", "", { "optionalDependencies": { "@napi-rs/canvas-android-arm64": "1.0.2", "@napi-rs/canvas-darwin-arm64": "1.0.2", "@napi-rs/canvas-darwin-x64": "1.0.2", "@napi-rs/canvas-linux-arm-gnueabihf": "1.0.2", "@napi-rs/canvas-linux-arm64-gnu": "1.0.2", "@napi-rs/canvas-linux-arm64-musl": "1.0.2", "@napi-rs/canvas-linux-riscv64-gnu": "1.0.2", "@napi-rs/canvas-linux-x64-gnu": "1.0.2", "@napi-rs/canvas-linux-x64-musl": "1.0.2", "@napi-rs/canvas-win32-arm64-msvc": "1.0.2", "@napi-rs/canvas-win32-x64-msvc": "1.0.2" } }, "sha512-EYEqlMYaCbpZDz+IgDH5xp9MTd3ui4dmGqbQYryhMLnSRxrhHKq5KQWHHKxFUcEP4Hp8/BWgvqXocX4j7iSbOQ=="],
"@napi-rs/canvas-android-arm64": ["@napi-rs/canvas-android-arm64@1.0.2", "", { "os": "android", "cpu": "arm64" }, "sha512-IMXKVQod0ol4vt3gmClUfXz4JAgHYESGPCUqmH3lQxBoL0K/2greJaQE1HVBVxWWFKfLc4OLZVdxg7kXVyXv+g=="],
@@ -1760,6 +1773,10 @@
"events-universal": ["events-universal@1.0.1", "", { "dependencies": { "bare-events": "^2.7.0" } }, "sha512-LUd5euvbMLpwOF8m6ivPCbhQeSiYVNb8Vs0fQ8QjXo0JTkEHpz8pxdQf0gStltaPpw0Cca8b39KxvK9cfKRiAw=="],
"eventsource": ["eventsource@3.0.7", "", { "dependencies": { "eventsource-parser": "^3.0.1" } }, "sha512-CRT1WTyuQoD771GW56XEZFQ/ZoSfWid1alKGDYMmkt2yl8UXrVR4pspqWNEcqKvVIzg6PAltWjxcSSPrboA4iA=="],
"eventsource-parser": ["eventsource-parser@3.1.0", "", {}, "sha512-kJezFj9YFAMLeORyi7aCLxLbD5/qWMQnoMVlVPyHIll7lgRJCc3JVln9Vgl9nwQi0YkMnhdGTMNn7CkRRAptMg=="],
"expect-type": ["expect-type@1.3.0", "", {}, "sha512-knvyeauYhqjOYvQ66MznSMs83wmHrCycNEN6Ao+2AeYEfxUIkuiVxdEa1qlGEPK+We3n0THiDciYSsCcgW/DoA=="],
"exsolve": ["exsolve@1.0.8", "", {}, "sha512-LmDxfWXwcTArk8fUEnOfSZpHOJ6zOMUJKOtFLFqJLoKJetuQG874Uc7/Kki7zFLzYybmZhp1M7+98pfMqeX8yA=="],
@@ -2328,6 +2345,8 @@
"pino-std-serializers": ["pino-std-serializers@7.1.0", "", {}, "sha512-BndPH67/JxGExRgiX1dX0w1FvZck5Wa4aal9198SrRhZjH3GxKQUKIBnYJTdj2HDN3UQAS06HlfcSbQj2OHmaw=="],
"pkce-challenge": ["pkce-challenge@5.0.1", "", {}, "sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ=="],
"pkcs7": ["pkcs7@1.0.4", "", { "dependencies": { "@babel/runtime": "^7.5.5" }, "bin": { "pkcs7": "bin/cli.js" } }, "sha512-afRERtHn54AlwaF2/+LFszyAANTCggGilmcmILUzEjvs3XgFZT+xE6+QWQcAGmu4xajy+Xtj7acLOPdx5/eXWQ=="],
"pkg-types": ["pkg-types@2.3.1", "", { "dependencies": { "confbox": "^0.2.4", "exsolve": "^1.0.8", "pathe": "^2.0.3" } }, "sha512-y+ichcgc2LrADuhLNAx8DFjVfgz91pRxfZdI3UDhxHvcVEZsenLO+7XaU5vOp0u/7V/wZ+plyuQxtrDlZJ+yeg=="],
@@ -2820,8 +2839,6 @@
"@aws-sdk/nested-clients/@smithy/node-http-handler": ["@smithy/node-http-handler@4.7.6", "", { "dependencies": { "@smithy/core": "^3.24.6", "@smithy/types": "^4.14.3", "tslib": "^2.6.2" } }, "sha512-3fya8i7GrJilQouk4cZJKdy5k8MWQBpjfXrRNaXDedH8r779tr0jcxyH3+yoTmsluc2+vF4S343yFbnvu8ExDQ=="],
"@earendil-works/pi-agent-core/@earendil-works/pi-ai": ["@earendil-works/pi-ai@0.78.0", "", { "dependencies": { "@anthropic-ai/sdk": "0.91.1", "@aws-sdk/client-bedrock-runtime": "3.1048.0", "@google/genai": "1.52.0", "@mistralai/mistralai": "2.2.1", "@smithy/node-http-handler": "4.7.3", "http-proxy-agent": "7.0.2", "https-proxy-agent": "7.0.6", "openai": "6.26.0", "partial-json": "0.1.7", "typebox": "1.1.38" }, "bin": { "pi-ai": "dist/cli.js" } }, "sha512-q0hUrvT6ngT6cgBX0oIbzfQfmzztgdkZobP8OTL+sCOOBlnG6+1YRt8g7zO9CC/4NdeYEqa7uGqWdQhH0fjCLA=="],
"@earendil-works/pi-agent-core/ignore": ["ignore@7.0.5", "", {}, "sha512-Hs59xBNfUIunMFgWAbGX5cq6893IbWg4KnrjbYwX3tx0ztorVgTDA6B2sxf8ejHJ4wz8BqGUMYlnzNBer5NvGg=="],
"@eslint-community/eslint-utils/eslint-visitor-keys": ["eslint-visitor-keys@3.4.3", "", {}, "sha512-wpc+LXeiyiisxPlEkUzU6svyS1frIO3Mgxj1fdy7Pm8Ygzguax2N3Fa/D/ag1WqbOprdI+uY6wMUl8/a2G+iag=="],
@@ -3246,8 +3263,6 @@
"yauzl/buffer-crc32": ["buffer-crc32@0.2.13", "", {}, "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ=="],
"@earendil-works/pi-agent-core/@earendil-works/pi-ai/@mistralai/mistralai": ["@mistralai/mistralai@2.2.1", "", { "dependencies": { "ws": "^8.18.0", "zod": "^3.25.0 || ^4.0.0", "zod-to-json-schema": "^3.25.0" } }, "sha512-uKU8CZmL2RzYKmplsU01hii4p3pe4HqJefpWNRWXm1Tcm0Sm4xXfwSLIy4k7ZCPlbETCGcp69E7hZs+WOJ5itQ=="],
"@isaacs/cliui/string-width/emoji-regex": ["emoji-regex@9.2.2", "", {}, "sha512-L18DaJsXSUk2+42pv8mLs5jJT2hqFkFE4j21wOmgbUqsZ2hL72NsUU785g9RXgo3s0ZNgVl42TiHp3ZtOv/Vyg=="],
"@isaacs/cliui/strip-ansi/ansi-regex": ["ansi-regex@6.2.2", "", {}, "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg=="],
+10 -3
View File
@@ -34,7 +34,8 @@
"using-shumai/notifications",
"using-shumai/metadata",
"using-shumai/collections",
"using-shumai/share-links"
"using-shumai/share-links",
"using-shumai/gotenberg"
]
},
{
@@ -42,14 +43,20 @@
"pages": [
"using-shumai/agent-providers",
"using-shumai/agents",
"using-shumai/agents-md",
"using-shumai/agent-skills",
"using-shumai/agent-tools",
"using-shumai/mcp-servers",
"using-shumai/agent-sandbox"
]
},
{
"group": "Media Transcoding Settings",
"pages": ["using-shumai/transcode-settings"]
"group": "Team Settings",
"pages": [
"using-shumai/transcode-settings",
"using-shumai/quotas-settings",
"using-shumai/password-reset"
]
},
{
"group": "Configuration",
Binary file not shown.

Before

Width:  |  Height:  |  Size: 309 KiB

After

Width:  |  Height:  |  Size: 131 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 45 KiB

After

Width:  |  Height:  |  Size: 82 KiB

+1 -1
View File
@@ -25,7 +25,7 @@ To protect your workspace environments and sensitive data, the sandbox enforces
## Domain Whitelisting
By default, sandboxed scripts cannot make network requests to the internet. Team Owners must explicitly authorize external APIs by managing domain access:
By default, Network Sandbox is **disabled** (allowing all outbound network requests). Team Owners can toggle Network Sandbox **ON** in Sandbox Settings to enforce strict domain whitelisting:
### Allowed Domains
+12 -12
View File
@@ -30,36 +30,36 @@ The following tools can be individually toggled in the agent configuration dialo
- **Description:** Allows the agent to load and read skill definitions to perform specialized actions.
- **Why it matters:** Enables the agent to inspect the instructions/configurations of custom skills registered in your workspace before executing them.
### 3. Analyze Image (`analyze_image`)
### 3. Read Asset (`read_asset`)
- **Description:** Allows the agent to view and analyze the visual contents of an image file.
- **Why it matters:** Enables multimodal agents to "see" image assets uploaded to Shumai, allowing them to answer comments or write metadata about images.
- **Description:** Allows the agent to inspect images, extract still frames from video files, render PDF document pages, and read text/markdown documents.
- **Why it matters:** Enables multimodal agents to view visual media assets uploaded to Shumai, extract frame-by-frame video screenshots, inspect document layouts, and answer questions about uploaded creative assets.
### 4. Video Frame Extraction (`screenshot`)
- **Description:** Allows the agent to extract still image frames from video files for analysis.
- **Why it matters:** Enables the agent to perform frame-by-frame analysis on videos (e.g. for comments, annotation, or search).
### 5. Browse Shumai (`list_assets`)
### 4. Browse Shumai (`list_assets`)
- **Description:** Allows the agent to list folders and browse files in Shumai.
- **Why it matters:** Gives the agent visual mapping of the project structure so it can search for other files or locate related assets.
### 6. Create Folder (`create_folder`)
### 5. Create Folder (`create_folder`)
- **Description:** Allows the agent to create new folders in Shumai.
- **Why it matters:** Needed if the agent organizes outputs or places files in custom subdirectory folders.
### 7. Create File (`create_file`)
### 6. Create File (`create_file`)
- **Description:** Allows the agent to create and upload new files to Shumai.
- **Why it matters:** Allows the agent to generate new assets (e.g. text logs, transcoded clips, reports) and save them directly back into your project.
### 8. Create Version (`create_version`)
### 7. Create Version (`create_version`)
- **Description:** Allows the agent to upload and stack a new version of an existing file in Shumai.
- **Why it matters:** Allows the agent to update files in a clean, non-destructive way by stacking revisions rather than creating duplicate files.
### 8. Download Asset (`download_asset`)
- **Description:** Allows the agent to download a workspace asset or specific storage S3 key to the local `.pi` directory for processing with local scripts or bash commands.
- **Why it matters:** Enables agents executing custom skill scripts to fetch files or intermediate transcode artifacts to local disk.
---
## Restricting Tools
+53
View File
@@ -0,0 +1,53 @@
---
title: AGENTS.md Context & Instructions
description: Provide hierarchical, folder-specific instructions and context to AI agents.
---
In addition to global agent system prompts and skills, Shumai allows you to define project-level and folder-level instructions using **`AGENTS.md`**.
When an AI agent (such as a Chat Agent or Autofill Agent) operates within a project or folder, Shumai automatically loads the corresponding `AGENTS.md` files to provide context, operational rules, and project-specific knowledge.
---
## Hierarchical Traversal & Inheritance
Shumai uses a hierarchical context model for `AGENTS.md`:
- **Upward Traversal**: When an agent works on a file or folder, Shumai traverses the folder tree upwards from the current folder all the way to the **project root level**.
- **Combined Context**: All `AGENTS.md` files found along this path are loaded into the agent's context in order (from the project root down to the current folder).
- **Layered Rules**: This lets you define broad, project-wide rules and guidelines at the project root `AGENTS.md`, while adding specific rules, vocabulary, or autofill hints in nested subfolder `AGENTS.md` files.
---
## What to Include in AGENTS.md
You can use `AGENTS.md` to define any instructions you want the agent to follow when working in that folder or project:
- **Project & Folder Overview**: Describe what assets are stored in the folder and how they relate to the broader project.
- **Agent Behavioral Rules**: Specify guardrails, tone, required formats, or things the agent should and shouldn't do when answering queries or leaving comments.
- **Terminology & Domain Knowledge**: Define specialized vocabulary, acronyms, brand naming conventions, or project-specific jargon.
- **Metadata & Autofill Instructions**: Guide the **Autofill Agent** on how to summarize assets, generate tags, or populate specific custom metadata fields for files in this folder.
---
## Accessing and Editing AGENTS.md
### Opening the Editor
To view or edit `AGENTS.md`:
1. Open any project or folder in the Shumai web application.
2. Click the **`AGENTS.md`** button in the top navigation toolbar, or right-click any folder in the file browser and choose **AGENTS.md**.
3. A dedicated WYSIWYG Markdown editor opens with full formatting support (headings, lists, code blocks, bold/italics, and links).
### Auto-Save
As you make changes in the editor, Shumai automatically saves your edits in the background with visual save status indicators (`Saving...`, `Saved`).
---
## Permissions & Read-Only Mode
- **Viewing**: All team members who have read access to the project can view `AGENTS.md`.
- **Editing**: **Only Team Owners** are permitted to create, edit, or delete `AGENTS.md` files.
- **Read-Only State**: Non-owner members will see a clear read-only indicator in the toolbar explaining that only team owners can edit instructions for the file.
+82
View File
@@ -0,0 +1,82 @@
---
title: Document PDF Conversion with Gotenberg
description: Convert Office documents, HTML, Markdown, CSV, and TXT files into high-fidelity PDF proxies for in-browser previewing.
---
Office documents, HTML pages, Markdown files, spreadsheets, and plain-text files cannot be rendered natively in web browsers. Shumai uses [Gotenberg](https://gotenberg.dev), an open-source, Docker powered conversion API — to turn these files into PDF proxy assets for seamless in-browser previewing.
Gotenberg performs the conversion using:
- **LibreOffice** for Office documents, spreadsheets, and plain text.
- **Headless Chromium** for HTML and Markdown files.
---
## Supported File Types
When Gotenberg is configured, Shumai automatically generates a PDF proxy for the following uploads:
| Category | Extensions | Converter |
| ------------------ | ---------------------- | ----------- |
| Word | `.doc`, `.docx` | LibreOffice |
| Excel | `.xls`, `.xlsx` | LibreOffice |
| PowerPoint | `.ppt`, `.pptx` | LibreOffice |
| OpenDocument | `.odt`, `.ods`, `.odp` | LibreOffice |
| Rich Text | `.rtf` | LibreOffice |
| HTML | `.html`, `.htm` | Chromium |
| Markdown | `.md`, `.markdown` | Chromium |
| Spreadsheets / CSV | `.csv` | LibreOffice |
| Plain text | `.txt` | LibreOffice |
---
## Configuration
Gotenberg support is opt-in and controlled by the following environment variables on the machine running the transcode workers:
| Variable | Required | Description |
| ------------------------------- | --------------- | ---------------------------------------------------------------- |
| `GOTENBERG_URL` | Yes (to enable) | Base URL of the Gotenberg service, e.g. `http://gotenberg:3000`. |
| `GOTENBERG_BASIC_AUTH_USERNAME` | No | Username when Basic Authentication is enabled on Gotenberg. |
| `GOTENBERG_BASIC_AUTH_PASSWORD` | No | Password when Basic Authentication is enabled on Gotenberg. |
### Docker Quickstart
You can start a Gotenberg container with a single command:
```bash
docker run -d --name gotenberg -p 3000:3000 gotenberg/gotenberg:8
```
If you wish to enable Basic Authentication:
```bash
docker run -d --name gotenberg -p 3000:3000 gotenberg/gotenberg:8 gotenberg --gotenberg-basic-auth-username=admin --gotenberg-basic-auth-password=secret
```
### Docker Compose Setup
Add a Gotenberg service to your `docker-compose.yaml`:
```yaml
gotenberg:
image: gotenberg/gotenberg:8
restart: unless-stopped
ports:
- '3000:3000'
```
Then point Shumai at it:
```bash
GOTENBERG_URL=http://localhost:3000
```
---
## Behavior Without Gotenberg
If Gotenberg is not configured or unreachable, Shumai degrades gracefully:
- **Office documents and HTML**: the upload completes as `processed` without a PDF proxy.
- **CSV, Markdown, and TXT**: a lightweight built-in PDF generator produces the proxy instead, with reduced fidelity.
+115
View File
@@ -0,0 +1,115 @@
---
title: MCP Servers
description: Connect your agents to external tools and APIs using Model Context Protocol.
---
Model Context Protocol (MCP) is an open standard that enables AI models to safely interface with external tools, microservices, and APIs. In Shumai, you can register remote MCP servers using standard HTTP transports (`Streamable HTTP` or `SSE`) and assign them to your AI agents.
---
## How MCP Works in Shumai
Rather than overloading model context windows with dozens of complex parameter schemas, Shumai connects to MCP servers efficiently:
- **Automatic Discovery & Metadata Caching**: When you add an MCP server, Shumai records it in your team's settings and connects to the endpoint to discover available tools, server-reported usage instructions, and description metadata. Discovery runs when you refresh the server, complete an OAuth flow, or the first time an agent calls one of its tools. These schemas are cached in your database for fast retrieval.
- **Context-Efficient Proxy Tool (`mcp`)**: Assigned MCP servers are multiplexed through a single unified `mcp` proxy tool injected into your Chat Agents. The agent can dynamically inspect server status, search tools, view parameter schemas, and invoke functions as needed without wasting LLM tokens.
- **Direct Tools Option**: For high-frequency actions, you can optionally configure specific tools as **Direct Tools** to expose them directly to the agent as top-level LLM functions alongside the `mcp` proxy.
---
## Adding an MCP Server
To register a new MCP server in Shumai:
1. Navigate to **Team Settings** and click the **MCP** tab.
2. Click **Add MCP Server**.
3. In the dialog box, enter your **Server Endpoint URL** (e.g. `https://mcp.example.com/sse` or `http://localhost:8080/mcp`).
4. Click **Create**.
> **Note:** Only the **Server Endpoint URL** is required. Shumai derives a placeholder server name from the URL immediately; the server's actual name, status, instructions, and tool inventory are discovered the first time the server is connected (via the Refresh button in the edit dialog or the first tool call by an agent).
### Optional Server Configuration
After creating a server (the create dialog only requires a URL), open its edit dialog to configure advanced settings:
- **Transport**: Choose between `Streamable HTTP` (default) or `Server-Sent Events (SSE)`.
- **Auth Strategy**: Select `Auto-detect` (default), `None`, `Bearer Token`, or `OAuth 2.0`.
- **Minimum Permission Role**: Set the minimum team role required to invoke tools on this server (`Reviewer`, `Editor`, or `Owner`).
- **Direct Tools**: Specify a list of tool names that should be exposed directly to agents as native LLM tools.
---
## Understanding Tool Configuration
In the server's edit dialog, the **Tools** section lists every discovered tool with a three-state toggle (plus **All Off** / **All Proxy** / **All Direct** bulk actions):
- **Off**: The tool is excluded and completely invisible to agents — it cannot be listed, searched, or called through the proxy, nor registered as a direct tool. Use this to hide noisy or dangerous tools.
- **Proxy** (default): The tool is available through the single `mcp` proxy tool.
- **Direct**: The tool is exposed to agents as a native top-level function with a `{server}_{tool}` prefixed name.
### The `mcp` Proxy Tool
The `mcp` proxy tool is a single function injected into an agent's context that multiplexes every MCP server assigned to that agent. Instead of loading each tool's full parameter schema up front, the agent interacts with servers on demand:
- `mcp({})` — connection and auth status of all assigned servers
- `mcp({ server: "name" })` — list a server's tools
- `mcp({ search: "query" })` — search for tools across servers (schemas included by default)
- `mcp({ describe: "tool" })` — view one tool's full parameter schema
- `mcp({ instructions: "name" })` — view a server's usage instructions
- `mcp({ server: "name", tool: "tool_name", args: {...} })` — call a tool (pass either the `{server}_{tool}` prefixed name or the original name)
- `mcp({ action: "auth-start", server: "name" })` — start OAuth for a server
Because only one compact tool (plus a short server/tool listing) occupies the context window, the proxy keeps token usage low even when a server exposes dozens of tools. The trade-off is an extra discovery round trip: the agent typically lists or searches tools first, then calls them.
### Direct Tools
Direct tools flip the trade-off in the other direction. A tool configured as **Direct** is registered as a native top-level function (e.g. `tavily_mcp_tavily_search`) that the agent can invoke immediately, exactly like any built-in tool.
Why use direct tools:
- **Faster, more reliable calls** for high-frequency actions: the full parameter schema is always in the context window, so the agent can call the tool in a single step without first discovering it through the proxy.
- **Better tool discovery**: the model sees the tool and its description up front and is more likely to reach for it at the right moment.
The cost: every direct tool permanently consumes context tokens for its schema, so reserve **Direct** for frequently used or critical tools and keep large MCP servers on **Proxy** to avoid unnecessary context overhead. Direct tools are still subject to the same role-based permission checks as proxy calls, and the `mcp` proxy tool remains available for discovery, auth, and status.
---
## Authentication Modes
Shumai supports several authentication strategies for securing access to MCP endpoints:
### 1. No Auth (`none`)
Used for public or local MCP servers that do not require credentials.
### 2. Bearer Token (`bearer`)
Provide a static API token or key. Shumai attaches this token in the `Authorization: Bearer <token>` header for all transport requests.
### 3. OAuth 2.0 (`oauth`)
For MCP endpoints protected by OAuth identity providers:
- **Authorization Code + PKCE**: Click **Connect** on the server card in Team Settings. Shumai opens a secure OAuth popup window directing you to the identity provider. After approving permissions, the browser automatically redirects back to Shumai, completes token exchange, and refreshes server tools.
- **Client Credentials**: Provide a `Client ID` and `Client Secret` for automated server-to-server token acquisition.
---
## Assigning MCP Servers to Agents
By default, newly added MCP servers are **not assigned to any agent**. To give an agent access to an MCP server:
1. Navigate to **Team Settings** > **Agents**.
2. Click on the target agent to open its configuration dialog.
3. Scroll down to the **MCP Servers** section.
4. Toggle the switch next to the MCP server(s) you want to authorize for this agent.
5. Click **Save Changes**.
---
## Role-Based Access Control
Each MCP server has a **Minimum Required Role** setting (`Reviewer`, `Editor`, or `Owner`):
- When an agent attempts to call a tool on an MCP server, Shumai verifies the role of the user interacting with the agent.
- If the user's role does not meet the minimum requirement, tool execution is blocked with a permission error message, preventing unauthorized actions.
+60
View File
@@ -0,0 +1,60 @@
---
title: Password Reset
description: How administrators generate secure password reset links for team members.
---
Because Shumai does not require an external email delivery service (SMTP) to be configured for standard self-hosted deployments, password resets are initiated securely by an administrator using the Shumai CLI.
---
## Overview
When a user forgets their password, an administrator can generate a one-time password reset link directly from the command line.
- **Link Expiration**: The generated password reset link is valid for **1 hour** (`3600 seconds`).
- **One-Time Use**: Once used, the token is permanently invalidated.
- **Session Revocation**: When a user resets their password, all existing active sessions for that account are automatically revoked across all devices.
- **Latest Link Validity**: Generating a new reset link for a user immediately supersedes any previously generated pending links.
---
## Generating a Password Reset Link
### Running on Local Host / Server
Run the `reset-password` subcommand from your Shumai installation directory:
```bash
shumai reset-password <user-email>
```
_(Alternatively, `bun apps/web/src/index.ts reset-password <user-email>`)_
**Example Output:**
```text
http://localhost:3000/reset-password?token=bb3xg4uO3cwN0FQv4cgUK6Hh
```
### Running inside Docker
If Shumai is running inside a Docker container, execute the command using `docker exec`:
```bash
docker exec -it <container-name-or-id> shumai reset-password <user-email>
```
> [!TIP]
> Both `shumai` and the common alias `shuami` (e.g. `shuami reset-password <user-email>`) are supported.
> [!NOTE]
> For remote servers or production deployments, ensure `BETTER_AUTH_URL` is set in your environment (e.g., `BETTER_AUTH_URL=https://shumai.example.com`) so the CLI generates the reset link with your public domain instead of `localhost`.
---
## Member Reset Experience
1. The administrator sends the generated URL to the team member via their preferred secure communication channel.
2. The user opens the link in their browser.
3. The user inputs their **New Password** and **Confirm Password** (minimum 3 characters).
4. Upon clicking **Reset Password**, the password is securely updated, all previous active sessions are invalidated, and the user is redirected to `/login` to sign in.
+121
View File
@@ -0,0 +1,121 @@
---
title: Resource Quotas
description: Configure team resource quotas, monitor live usage, and reset quota windows from the dashboard.
---
AI workflows and media processing tools can consume significant compute and API credits. Shumai provides a **Resource Quotas** system that allows Team Owners to set granular rate limits, govern budgets, and monitor real-time usage across team members and roles.
---
## Overview
Resource Quotas allow you to define maximum usage limits across specific time windows (e.g., hourly, daily, or weekly). When a member or team reaches their assigned quota, further actions for that resource are safely blocked until the quota period resets.
Quota configuration and usage monitoring are separate parts of the team experience:
- To create and manage quota rules, open **Team Settings** and navigate to the **Quotas** tab.
- To monitor live usage, open the **Team Dashboard** and select **Quotas** under the **TEAM** section.
The quota dashboard is available to Team Owners.
---
## Scope Modes
When creating or editing a quota rule, you can define how the limit is applied:
### 1. Each Member
- **Behavior**: Every member in the scope receives their own independent limit. Usage by one member does not affect the remaining quota of others.
- **Optional Role Filter**: You can restrict the quota to members with a specific role (e.g. only **Editors** or **Reviewers**), or apply it to the entire team.
- **Use Case**: Give each editor a daily budget of 100,000 tokens.
### 2. All Members (Shared Pool)
- **Behavior**: The quota limit is shared as a collective pool across all matching members. Every member's consumption draws from the same total.
- **Optional Role Filter**: Apply the shared pool to all team members or restrict it to a specific role.
- **Use Case**: Cap the total team spending on AI models to $50 per day.
### 3. Selected Members
- **Behavior**: Assign individual limits to specific chosen team members.
- **Use Case**: Grant higher limits to senior artists or technical directors while keeping standard limits for others.
---
## Supported Resource Types
Shumai supports quota enforcement across multiple layers of the AI and agent architecture:
| Resource Type | Description | Target Specification |
| :-------------------------- | :-------------------------------------------------------------------------------------- | :----------------------------------- |
| **Total Tokens** | Total LLM input and output tokens consumed in chat, comments, and autofill. | Team, Role, or Members |
| **AI Cost ($)** | Dollar expenditure accrued from AI model providers. | Team, Role, or Members |
| **MCP Server Calls** | Number of tool invocations directed to a connected Model Context Protocol (MCP) server. | Target MCP Server ID |
| **Sandboxed Bash Commands** | Shell commands executed by the agent inside the isolated sandbox. | Wildcard pattern (e.g. `npm *`, `*`) |
| **Agent Tool Calls** | Calls to specific native tools (e.g. `read_asset`, `download_asset`, `create_file`). | Tool name |
### Bash Command Wildcard Matching
For sandboxed bash execution, you can restrict specific command patterns using wildcards (`*` and `?`):
- `*`: Matches any bash command execution.
- `npm *`: Matches any npm command (e.g., `npm install`, `npm test`).
- `git commit*`: Matches git commit invocations.
---
## Time Periods & Rollover
Quotas can be configured with the following reset periods:
- **1 Hour** (`1hour`)
- **5 Hours** (`5hour`)
- **1 Day** (`1day`)
- **7 Days** (`7day`)
### Lazy Reset Mechanism
Shumai tracks usage windows automatically. When a period window expires, the system resets the quota upon the member's next request, seamlessly starting a fresh window without requiring background maintenance tasks.
---
## Monitoring Live Usage
Team Owners can inspect live consumption from the team dashboard:
1. Open the **Team Dashboard**.
2. Select **Quotas** under the **TEAM** section.
3. Quota rules are shown as first-level collapsible entries.
4. Expand a rule to view its usage records.
Each usage record displays:
- **Consumption Status**: Progress bar showing consumed units versus the total limit.
- **Remaining Allowance**: Percentage and numerical amount remaining.
- **Refresh Countdown**: Remaining time until the active window resets (e.g., _"Refreshes in 45m"_).
- **Target**: The team shared pool, role pool, or individual member associated with the record.
- **Member Search**: For member-scoped rules, search the usage records by member name or email.
Usage records are loaded when a rule is expanded and refresh automatically while the rule is open.
### Manually Resetting a Usage Record
Each usage record includes a **Reset Usage** action. After confirming the action:
- Consumption is set to zero.
- The quota period starts immediately.
- The period ends after the configured quota duration.
- For example, resetting a one-day quota at 10:00 starts a new window from 10:00 until 10:00 the following day.
A reset can also initialize a zero-consumption record for a member or shared pool that has not received usage yet. Resets are recorded in the team audit log.
---
## Managing Quota Rules
- **Enable / Disable**: Toggle the switch on any quota card in Team Settings to temporarily pause or resume enforcement without deleting the rule.
- **Edit Rules**: Click any quota card in Team Settings to update limits, change reset periods, or adjust member selections.
- **Delete Rules**: Open the card action menu (`⋮`) in Team Settings and select **Delete** to remove the quota rule.
Quota usage is monitored and reset from the Team Dashboard rather than from the settings rule cards.
+26
View File
@@ -14,6 +14,8 @@ Shumai allows you to collaborate with external stakeholders who do not have acco
3. Configure the share link in the right sidebar. You can:
- Add password protection
- Set an expiration date
- Allow or disallow downloads
- Add a watermark to previews
- Configure which metadata fields are visible to recipients
4. Click **Copy Link** at the bottom of the sidebar, then send the link to your client or stakeholder.
@@ -41,6 +43,30 @@ Set an optional date and time after which the link becomes inactive. Once the da
Secure sensitive work by adding a password. External users visiting the link will be prompted to enter the correct password before they can view the assets.
### Allow Download
Control whether visitors can download the shared files. When **Allow download** is turned off:
- All download buttons are hidden on the public share page (file cards, selection bar, viewer control bars, and the top navigation).
Guests can still preview the files; only downloading is blocked. This is often combined with a watermark so previews remain viewable but protected.
### Watermark
Overlay a watermark (e.g. a "CONFIDENTIAL" text label or a logo image) on every preview shown through the share link. Open the watermark editor, arrange the blocks, and save.
Watermarking is **lazy and per-share-link** to keep storage costs low:
- The watermark transcode is only triggered when you **enable the watermark on a share link** — Shumai does **not** pre-generate watermark copies for every file in your workspace.
- All video and image assets already in the share are transcoded at that moment (the sidebar shows a _processing_ status until it finishes).
- Files and folders **added to the share link later are automatically transcoded too**, so the watermark always covers the full share without any manual steps.
- Since watermark copies are only produced for assets that are actually shared through a watermarked link, files that are never shared (or shared through non-watermarked links) never consume extra storage for watermark proxies. This can save a very large amount of storage on big libraries.
Notes:
- Only **video and image** proxies are watermarked; other media types (PDF, audio, ...) are not affected.
- While the watermark is still processing, previews are hidden rather than exposing the original, unwatermarked proxy.
### Default Sort Order
Set a default sorting rule for the shared page (e.g. sorting by name or version) to ensure the client views the files in the intended order.
+1
View File
@@ -74,6 +74,7 @@ export default tseslint.config(
'**/packages/db/src/generated/**',
'**/packages/core/src/generated/**',
'**/data/**',
'**/packages/core/src/provider/generator/**',
'**/packages/webui/components/ui/**',
'**/packages/webui/paraglide/**',
'**/packages/webui/public/**',
+7 -5
View File
@@ -1,6 +1,6 @@
{
"name": "shumai",
"version": "0.3.2",
"version": "0.4.2",
"private": true,
"type": "module",
"repository": {
@@ -18,17 +18,16 @@
"typecheck": "tsc --noEmit",
"lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
"test": "bun --bun vitest run",
"test:e2e": "playwright test",
"test:e2e:app": "playwright test --project=app-chromium",
"test:e2e:webui": "E2E_SKIP_APP_SERVER=true playwright test --project=harness-chromium",
"test:e2e:all": "playwright test",
"test:e2e:workflow": "bun --bun vitest run -c packages/e2e/vitest.config.ts",
"test:e2e:ui": "playwright test --ui",
"test:e2e:report": "playwright show-report",
"test:e2e:install": "playwright install --with-deps",
"harness:dev": "bun run packages/webui/e2e/harness/serve.ts",
"format": "prettier --write .",
"db:migrate": "bun run prisma migrate dev",
"db:deploy": "bun run prisma migrate deploy",
"db:generate": "cd packages/db && bunx prisma generate",
"generate-providers": "bun run scripts/generate-builtin-providers.ts",
"worker:agent": "bun apps/agent/src/index.ts",
"worker:transcode": "bun apps/transcode/src/index.ts",
"build": "bun run build.ts",
@@ -66,5 +65,8 @@
"typescript-eslint": "8.60.1",
"vite-tsconfig-paths": "6.1.1",
"vitest": "4.1.8"
},
"patchedDependencies": {
"@earendil-works/pi-ai@0.82.1": "patches/@earendil-works%2Fpi-ai@0.82.1.patch"
}
}
+4 -4
View File
@@ -1,25 +1,25 @@
{
"name": "@shumai/agent",
"version": "0.3.2",
"version": "0.4.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
"types": "./src/index.ts",
"dependencies": {
"@anthropic-ai/sandbox-runtime": "0.0.66",
"@earendil-works/pi-agent-core": "0.78.0",
"@earendil-works/pi-ai": "0.80.7",
"@earendil-works/pi-agent-core": "0.82.1",
"@earendil-works/pi-ai": "0.82.1",
"@google/genai": "1.52.0",
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@shumai/workflow-core": "workspace:*",
"@sinclair/typebox": "0.34.49",
"@temporalio/activity": "1.17.2",
"@temporalio/worker": "1.17.2",
"@temporalio/workflow": "1.17.2",
"adm-zip": "0.5.17",
"jittered-fractional-indexing": "1.0.1",
"typebox": "1.1.38",
"ulid": "3.0.2",
"zod": "4.4.3"
},
+465 -248
View File
@@ -13,11 +13,12 @@ import {
getCommentActivity,
getProjectAutofillFieldsActivity,
updateAssetMetadataActivity,
getAssetPathHierarchyActivity,
getAssetPathContextActivity,
executeAgentToolActivity,
generateSessionNameActivity,
type GenerateSessionNameParams,
getUserTeamInfoActivity,
formatProjectContextPrompt,
} from './agent'
import * as piAgent from '../index'
import { type AgentHarness, type Session } from '@earendil-works/pi-agent-core'
@@ -35,8 +36,6 @@ import {
type Asset,
} from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import { authzService } from '@shumai/core/src/authz/authz'
import { uploadService } from '@shumai/core/src/upload/upload'
vi.mock('../index', async () => {
const actual = await vi.importActual('../index')
@@ -70,7 +69,8 @@ describe('Agent Activities', () => {
vi.mocked(piAgent.createAgentSession).mockResolvedValue({
session: mockSession as unknown as Session<DatabaseSessionMetadata>,
harness: mockHarness as unknown as AgentHarness,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
})
const context = {
@@ -137,7 +137,8 @@ describe('Agent Activities', () => {
vi.mocked(piAgent.createAgentSession).mockResolvedValue({
session: mockSession as unknown as Session<DatabaseSessionMetadata>,
harness: mockHarness as unknown as AgentHarness,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
})
const context = {
@@ -198,7 +199,8 @@ describe('Agent Activities', () => {
vi.mocked(piAgent.createAgentSession).mockResolvedValue({
session: mockSession as unknown as Session<DatabaseSessionMetadata>,
harness: mockHarness as unknown as AgentHarness,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
})
const context = {
@@ -266,7 +268,8 @@ describe('Agent Activities', () => {
vi.mocked(piAgent.createAgentSession).mockResolvedValue({
session: mockSession as unknown as Session<DatabaseSessionMetadata>,
harness: mockHarness as unknown as AgentHarness,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
})
const context = {
@@ -331,7 +334,8 @@ describe('Agent Activities', () => {
vi.mocked(piAgent.createAgentSession).mockResolvedValue({
session: mockSession as unknown as Session<DatabaseSessionMetadata>,
harness: mockHarness as unknown as AgentHarness,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
})
const context = {
@@ -371,7 +375,8 @@ describe('Agent Activities', () => {
vi.mocked(piAgent.createAgentSession).mockResolvedValue({
session: mockSession as unknown as Session<DatabaseSessionMetadata>,
harness: mockHarness as unknown as AgentHarness,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
})
const context = {
@@ -430,7 +435,8 @@ describe('Agent Activities', () => {
}
return {
session: mockSession as unknown as Session<DatabaseSessionMetadata>,
harness: mockHarness as unknown as AgentHarness,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
}
})
@@ -451,11 +457,66 @@ describe('Agent Activities', () => {
expect(res.text).toBe('{"f1":"val"}')
expect(res.usage.inputTokens).toBe(5)
})
it('should use the base autofill prompt without any creation-context injection', async () => {
const mockHarness = {
subscribe: vi.fn(),
prompt: vi.fn().mockResolvedValue({
content: [{ type: 'text', text: 'Captured' }],
usage: { input: 5, output: 5 },
}),
}
const mockSession = {
getEntries: vi.fn().mockResolvedValue([]),
getStorage: vi.fn().mockReturnValue({ sessionId: 'mock-session-id' }),
}
vi.mocked(piAgent.createAgentSession).mockImplementation(async (config: unknown) => {
const params = config as {
customTools: Array<{
name: string
execute: (id: string, args: Record<string, unknown>) => Promise<unknown>
}>
}
const tool = params.customTools.find((t) => t.name === 'autofill_metadata')
if (tool) {
await tool.execute('1', { f1: 'val' })
}
return {
session: mockSession as unknown as Session<DatabaseSessionMetadata>,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
}
})
const context = {
agent: { id: 'b1', provider: { name: 'google' }, modelRef: { modelId: 'gemini' } },
dbProviders: [],
teamSkills: [],
allowedDomains: [],
} as unknown as AgentExecutionContext
await autofillAiActivity({
teamId: 't1',
images: [],
fields: [{ id: 'f1', config: { name: 'F1', type: 'text' } }],
context,
})
const capturedPrompt = mockHarness.prompt.mock.calls[0]?.[0] ?? ''
expect(capturedPrompt).toContain('Analyze the provided images and extract metadata.')
expect(capturedPrompt).not.toContain('<context>')
})
})
describe('Agent Database Activities Integration', () => {
setupTestDbHooks()
// Space out comment creations: ULID ids have 1ms timestamp granularity, so
// back-to-back creates landing in the same millisecond sort by random suffix
// and can break the id-ordered DAG spine in initializeAgentSessionActivity.
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms))
let team: Team
let user: User
let project: Project
@@ -504,8 +565,18 @@ describe('Agent Database Activities Integration', () => {
describe('initializeAgentSessionActivity', () => {
it('should initialize a session, create agent/user records if missing, and store existing comments context', async () => {
const attAsset = await prisma.asset.create({
data: {
name: 'attached-doc.pdf',
type: AssetType.file,
mediaType: 'application/pdf',
status: AssetStatus.uploaded,
projectId: project.id,
},
})
// Create first comment so it acts as context
await prisma.assetComment.create({
const firstComment = await prisma.assetComment.create({
data: {
assetId: asset.id,
message: 'Hello <@' + user.id + '> check this',
@@ -513,6 +584,14 @@ describe('Agent Database Activities Integration', () => {
},
})
await prisma.assetCommentAttachment.create({
data: {
commentId: firstComment.id,
assetId: attAsset.id,
},
})
await sleep(2)
// Create second comment that user responds with, triggering session initialization
const userComment = await prisma.assetComment.create({
data: {
@@ -550,9 +629,20 @@ describe('Agent Database Activities Integration', () => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- entry data is stored as Json in DB and needs casting to check properties
const parsedEntry = pathEntries[0] as any
expect(parsedEntry.message.content[0].text).toContain(
`[${user.name} (owner)]: Hello <@${user.name}> check this`,
)
expect(parsedEntry.type).toBe('custom_message')
expect(parsedEntry.customType).toBe('shumai_message')
expect(parsedEntry.content).toContain(`Hello <@${user.name}> check this`)
expect(parsedEntry.details.user.name).toBe(user.name)
expect(parsedEntry.details.user.role).toBe('owner')
expect(parsedEntry.details.attachedFiles).toEqual([
{
id: attAsset.id,
name: 'attached-doc.pdf',
type: 'file',
mediaType: undefined,
mimeType: 'application/pdf',
},
])
})
it('should distinguish Main Session (userCommentId: null) and Thread Session (userCommentId: rootId) during lazy sync', async () => {
@@ -560,11 +650,13 @@ describe('Agent Database Activities Integration', () => {
const mainComment1 = await prisma.assetComment.create({
data: { assetId: asset.id, message: 'Main Comment 1', creatorId: user.id },
})
await sleep(2)
const mainComment2 = await prisma.assetComment.create({
data: { assetId: asset.id, message: 'Main Comment 2', creatorId: user.id },
})
// 2. Thread reply under mainComment1
await sleep(2)
const threadReply1 = await prisma.assetComment.create({
data: {
assetId: asset.id,
@@ -619,7 +711,7 @@ describe('Agent Database Activities Integration', () => {
expect(threadSession1?.userCommentId).toBe(mainComment1.id) // Thread Session has userCommentId = rootComment.id
})
it('should dynamically tag top-level comments with [Thread ID: id] even if thread was created after initial sync', async () => {
it('should dynamically tag top-level comments with thread metadata in context even if thread was created after initial sync', async () => {
const msg3 = await prisma.assetComment.create({
data: { assetId: asset.id, message: 'Question in msg3', creatorId: user.id },
})
@@ -664,8 +756,13 @@ describe('Agent Database Activities Integration', () => {
const msg3Entry = pathEntries.find((e) => e.id.includes(msg3.id))
expect(msg3Entry).toBeDefined()
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const text = (msg3Entry as any).message.content[0].text
expect(text).toContain(`[Thread ID: ${msg3.id}] [Replies: 1]`)
expect((msg3Entry as any).details?.thread).toEqual({
id: msg3.id,
replyCount: 1,
})
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const text = (msg3Entry as any).content ?? (msg3Entry as any).message?.content[0]?.text
expect(text).not.toContain(`[Thread ID: ${msg3.id}]`)
})
it('should create separate thread sessions when agent is mentioned in top-level comments (msg3 & msg5) sharing DAG entries without duplication', async () => {
@@ -673,10 +770,12 @@ describe('Agent Database Activities Integration', () => {
const msg1 = await prisma.assetComment.create({
data: { assetId: asset.id, message: 'msg1', creatorId: user.id },
})
await sleep(2)
// msg2 (top level)
const msg2 = await prisma.assetComment.create({
data: { assetId: asset.id, message: 'msg2', creatorId: user.id },
})
await sleep(2)
// msg3 (top level asking agent)
const msg3 = await prisma.assetComment.create({
data: { assetId: asset.id, message: '[ask agent] msg3', creatorId: user.id },
@@ -691,6 +790,7 @@ describe('Agent Database Activities Integration', () => {
})
// agent response msg4
await sleep(2)
const msg4 = await prisma.assetComment.create({
data: {
assetId: asset.id,
@@ -702,6 +802,7 @@ describe('Agent Database Activities Integration', () => {
})
// msg5 (top level asking agent)
await sleep(2)
const msg5 = await prisma.assetComment.create({
data: { assetId: asset.id, message: '[ask agent] msg5', creatorId: user.id },
})
@@ -725,6 +826,7 @@ describe('Agent Database Activities Integration', () => {
expect(session3Id).not.toBe(session5Id)
// msg7 (reply to msg3 asking agent again)
await sleep(2)
const msg7 = await prisma.assetComment.create({
data: {
assetId: asset.id,
@@ -773,10 +875,12 @@ describe('Agent Database Activities Integration', () => {
const msg1 = await prisma.assetComment.create({
data: { assetId: asset.id, message: 'msg1', creatorId: user.id },
})
await sleep(2)
// msg2 (top level)
const msg2 = await prisma.assetComment.create({
data: { assetId: asset.id, message: 'msg2', creatorId: user.id },
})
await sleep(2)
// msg3 (top level asking agent)
const msg3 = await prisma.assetComment.create({
data: { assetId: asset.id, message: '<@shumai-x1> what did i say', creatorId: user.id },
@@ -811,10 +915,12 @@ describe('Agent Database Activities Integration', () => {
const msg1 = await prisma.assetComment.create({
data: { assetId: asset.id, message: 'msg1', creatorId: user.id },
})
await sleep(2)
// msg2 (top level)
const msg2 = await prisma.assetComment.create({
data: { assetId: asset.id, message: 'msg2', creatorId: user.id },
})
await sleep(2)
// msg3 (top level asking agent)
const msg3 = await prisma.assetComment.create({
data: { assetId: asset.id, message: '<@shumai-x1> what did i say', creatorId: user.id },
@@ -878,7 +984,7 @@ describe('Agent Database Activities Integration', () => {
).resolves.not.toThrow()
})
it('should not tag Thread ID when only __CHAT__ placeholder comment exists under root comment', async () => {
it('should not tag thread when only __CHAT__ placeholder comment exists under root comment', async () => {
const rootComment = await prisma.assetComment.create({
data: { assetId: asset.id, message: 'Root comment asking agent', creatorId: user.id },
})
@@ -905,8 +1011,7 @@ describe('Agent Database Activities Integration', () => {
expect(rootEntry).toBeDefined()
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- entry data structure parsing
const content = (rootEntry as any)?.message?.content?.[0]?.text || ''
expect(content).not.toContain(`[Thread ID: ${rootComment.id}]`)
expect((rootEntry as any)?.details?.thread).toBeUndefined()
})
it('should return user name and role for team member', async () => {
@@ -981,6 +1086,24 @@ describe('Agent Database Activities Integration', () => {
},
})
// Create two team skills, only one of which is enabled for the agent
const enabledSkill = await prisma.skill.create({
data: {
name: 'Enabled Chat Skill',
assetId: 'asset-enabled',
hash: 'enabled-hash',
teamId: team.id,
},
})
const disabledSkill = await prisma.skill.create({
data: {
name: 'Disabled Chat Skill',
assetId: 'asset-disabled',
hash: 'disabled-hash',
teamId: team.id,
},
})
const agent = await prisma.agent.create({
data: {
id: agentUser.id,
@@ -992,6 +1115,9 @@ describe('Agent Database Activities Integration', () => {
provider: 'openai',
model: 'gpt-4',
},
skills: {
create: [{ skillId: enabledSkill.id }],
},
},
})
@@ -1002,6 +1128,59 @@ describe('Agent Database Activities Integration', () => {
expect(chatCtx.agent.id).toBe(agent.id)
expect(chatCtx.dbProviders[0].name).toBe('openai')
// Only the enabled skill must be advertised, and disabled team skills excluded
expect(chatCtx.teamSkills.map((s) => s.id)).toEqual([enabledSkill.id])
expect(chatCtx.enabledSkillIds).toEqual([enabledSkill.id])
expect(chatCtx.teamSkills.map((s) => s.id)).not.toContain(disabledSkill.id)
// Test permission enforcement in getAgentChatContextActivity
// Create owner-only agent
const ownerAgentUser = await prisma.user.create({
data: { name: 'Owner Bot', email: `ownerbot-${Date.now()}@shumai.ai`, type: 'agent' },
})
const ownerAgent = await prisma.agent.create({
data: {
id: ownerAgentUser.id,
teamId: team.id,
type: 'chat',
permission: 'owner',
providerId: provider.id,
modelId: model.id,
config: { provider: 'openai', model: 'gpt-4' },
},
})
// Create owner and reviewer members
const ownerUser = await prisma.user.create({
data: { name: 'Owner User', email: `owner-${Date.now()}@shumai.ai` },
})
await prisma.teamMember.create({
data: { teamId: team.id, userId: ownerUser.id, role: 'owner' },
})
const reviewerUser = await prisma.user.create({
data: { name: 'Reviewer User', email: `reviewer-${Date.now()}@shumai.ai` },
})
await prisma.teamMember.create({
data: { teamId: team.id, userId: reviewerUser.id, role: 'reviewer' },
})
// Owner should succeed
const ownerCtx = await getAgentChatContextActivity({
teamId: team.id,
agentId: ownerAgent.id,
userId: ownerUser.id,
})
expect(ownerCtx.agent.id).toBe(ownerAgent.id)
// Reviewer should fail with non-retryable ApplicationFailure
await expect(
getAgentChatContextActivity({
teamId: team.id,
agentId: ownerAgent.id,
userId: reviewerUser.id,
}),
).rejects.toThrow('Permission denied')
// Create User for autofill agent
const autofillUser = await prisma.user.create({
@@ -1024,6 +1203,9 @@ describe('Agent Database Activities Integration', () => {
provider: 'openai',
model: 'gpt-4',
},
skills: {
create: [{ skillId: enabledSkill.id }],
},
},
})
@@ -1032,6 +1214,77 @@ describe('Agent Database Activities Integration', () => {
})
expect(autofillCtx.agent.id).toBe(autofillAgent.id)
// Only the enabled skill must be advertised for the autofill agent as well
expect(autofillCtx.teamSkills.map((s) => s.id)).toEqual([enabledSkill.id])
expect(autofillCtx.enabledSkillIds).toEqual([enabledSkill.id])
expect(autofillCtx.teamSkills.map((s) => s.id)).not.toContain(disabledSkill.id)
})
it('should return an empty enabled skill list when the agent has no skills enabled', async () => {
const provider = await prisma.provider.create({
data: {
teamId: team.id,
name: 'openai-noskill',
config: {
api: 'openai-completions',
apiKey: 'mock-key',
},
},
})
const model = await prisma.model.create({
data: {
providerId: provider.id,
modelId: 'gpt-4',
name: 'GPT 4',
config: {
reasoning: false,
input: ['text'],
contextWindow: 8192,
maxTokens: 2048,
cost: { input: 0.01, output: 0.03, cacheRead: 0, cacheWrite: 0 },
},
},
})
await prisma.skill.create({
data: {
name: 'Team Wide Skill',
assetId: 'asset-team',
hash: 'team-hash',
teamId: team.id,
},
})
const agentUser = await prisma.user.create({
data: {
name: 'No Skill Agent User',
email: 'noskillagent@shumai.ai',
type: 'agent',
},
})
const agent = await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'chat',
providerId: provider.id,
modelId: model.id,
config: {
provider: 'openai-noskill',
model: 'gpt-4',
},
},
})
const chatCtx = await getAgentChatContextActivity({
teamId: team.id,
agentId: agent.id,
})
// Team-wide skill exists but is not enabled for this agent
expect(chatCtx.teamSkills).toEqual([])
expect(chatCtx.enabledSkillIds).toEqual([])
})
})
@@ -1120,10 +1373,14 @@ describe('Agent Database Activities Integration', () => {
})
const pathCtx = await getAssetPathContextActivity(targetAsset.id)
expect(pathCtx).toContain('Path: RootFolder/SubFolder/TargetImage.webp')
expect(pathCtx).toContain(`name: RootFolder, id: ${rootFolder.id}`)
expect(pathCtx).toContain(`name: SubFolder, id: ${subfolder.id}`)
expect(pathCtx).toContain(`name: TargetImage.webp, id: ${targetAsset.id}`)
expect(pathCtx).toBe('RootFolder/SubFolder/TargetImage.webp')
const hierarchy = await getAssetPathHierarchyActivity(targetAsset.id)
expect(hierarchy.path).toBe('RootFolder/SubFolder/TargetImage.webp')
expect(hierarchy.ancestors).toEqual([
{ id: rootFolder.id, name: 'RootFolder' },
{ id: subfolder.id, name: 'SubFolder' },
])
})
})
@@ -1135,8 +1392,7 @@ describe('Agent Database Activities Integration', () => {
key: 'field_key_1',
scope: 'TEAM',
teamId: team.id,
aiAutofill: true,
config: { name: 'Title Extractor', type: 'text' },
config: { name: 'Title Extractor', type: 'text', autofillSource: 'CONTENT' },
},
})
@@ -1156,227 +1412,6 @@ describe('Agent Database Activities Integration', () => {
})
})
describe('executeAgentToolActivity', () => {
beforeEach(() => {
vi.spyOn(authzService, 'hasPermission').mockResolvedValue()
vi.spyOn(uploadService, 'triggerPostUploadWorkflows').mockResolvedValue()
})
describe('list_assets', () => {
it('should list assets inside a folder with cursor pagination', async () => {
const folder = await prisma.asset.create({
data: {
name: 'WorkspaceFolder',
type: AssetType.folder,
status: AssetStatus.uploaded,
projectId: project.id,
},
})
await prisma.asset.create({
data: {
name: 'file1.txt',
type: AssetType.file,
status: AssetStatus.uploaded,
projectId: project.id,
parentId: folder.id,
},
})
await prisma.asset.create({
data: {
name: 'subfolder1',
type: AssetType.folder,
status: AssetStatus.uploaded,
projectId: project.id,
parentId: folder.id,
},
})
const res = await executeAgentToolActivity({
taskId: 'task-1',
toolName: 'list_assets',
args: { parent: folder.id, page: 1, pageSize: 1 },
userId: user.id,
})
expect(res.assets.length).toBe(1)
expect(res.pageInfo.cursor).toBeDefined()
})
})
describe('create_folder', () => {
it('should create folder and increment parent fileCount', async () => {
const folder = await prisma.asset.create({
data: {
name: 'WorkspaceFolder',
type: AssetType.folder,
status: AssetStatus.uploaded,
projectId: project.id,
fileCount: 0,
},
})
const res = await executeAgentToolActivity({
taskId: 'task-1',
toolName: 'create_folder',
args: { parent: folder.id, name: 'subfolder' },
userId: user.id,
})
expect(res.name).toBe('subfolder')
expect(res.type).toBe('folder')
const parentFolder = await prisma.asset.findUnique({ where: { id: folder.id } })
expect(parentFolder?.fileCount).toBe(1)
})
})
describe('create_file', () => {
it('should create file, increment parent fileCount, and update parent sizes', async () => {
const folder = await prisma.asset.create({
data: {
name: 'WorkspaceFolder',
type: AssetType.folder,
status: AssetStatus.uploaded,
projectId: project.id,
fileCount: 0,
sizeByte: 0,
},
})
const res = await executeAgentToolActivity({
taskId: 'task-1',
toolName: 'create_file',
args: {
parent: folder.id,
s3Key: 'uploads/testfile.txt',
name: 'testfile.txt',
size: 500,
contentType: 'text/plain',
},
userId: user.id,
})
expect(res.name).toBe('testfile.txt')
expect(res.size).toBe(500)
const parentFolder = await prisma.asset.findUnique({ where: { id: folder.id } })
expect(parentFolder?.fileCount).toBe(1)
expect(Number(parentFolder?.sizeByte)).toBe(500)
expect(uploadService.triggerPostUploadWorkflows).toHaveBeenCalled()
})
})
describe('create_version', () => {
it('should create a version stack when parent is a regular file', async () => {
const folder = await prisma.asset.create({
data: {
name: 'WorkspaceFolder',
type: AssetType.folder,
status: AssetStatus.uploaded,
projectId: project.id,
fileCount: 1,
sizeByte: 200,
},
})
const fileAsset = await prisma.asset.create({
data: {
name: 'v1.txt',
type: AssetType.file,
status: AssetStatus.uploaded,
projectId: project.id,
parentId: folder.id,
sizeByte: 200,
},
})
const res = await executeAgentToolActivity({
taskId: 'task-1',
toolName: 'create_version',
args: {
parent: fileAsset.id,
s3Key: 'uploads/v2.txt',
name: 'v2.txt',
size: 300,
contentType: 'text/plain',
},
userId: user.id,
})
expect(res.name).toBe('v2.txt')
expect(res.size).toBe(300)
// Check version stack was created
const stack = await prisma.asset.findFirst({
where: { type: AssetType.version_stack, projectId: project.id },
include: { children: true },
})
expect(stack).not.toBeNull()
expect(stack?.children.length).toBe(2) // v1.txt and v2.txt should both be inside the stack
})
it('should add to version stack when parent is already inside a version stack', async () => {
const folder = await prisma.asset.create({
data: {
name: 'WorkspaceFolder',
type: AssetType.folder,
status: AssetStatus.uploaded,
projectId: project.id,
},
})
const stack = await prisma.asset.create({
data: {
name: 'VersionStack',
type: AssetType.version_stack,
status: AssetStatus.uploaded,
projectId: project.id,
parentId: folder.id,
fileCount: 1,
sizeByte: 100,
},
})
const existingFileVersion = await prisma.asset.create({
data: {
name: 'v1.txt',
type: AssetType.file,
status: AssetStatus.uploaded,
projectId: project.id,
parentId: stack.id,
sizeByte: 100,
},
})
const res = await executeAgentToolActivity({
taskId: 'task-1',
toolName: 'create_version',
args: {
parent: existingFileVersion.id,
s3Key: 'uploads/v2.txt',
name: 'v2.txt',
size: 250,
contentType: 'text/plain',
},
userId: user.id,
})
expect(res.name).toBe('v2.txt')
const updatedStack = await prisma.asset.findUnique({
where: { id: stack.id },
include: { children: true },
})
expect(updatedStack?.fileCount).toBe(2)
expect(Number(updatedStack?.sizeByte)).toBe(350)
})
})
})
describe('generateSessionNameActivity', () => {
it('should generate and save session name for chat sessions', async () => {
await prisma.team.create({
@@ -1420,7 +1455,8 @@ describe('Agent Database Activities Integration', () => {
}
vi.mocked(piAgent.createAgentSession).mockResolvedValue({
session: {} as unknown as Session<DatabaseSessionMetadata>,
harness: mockHarness as unknown as AgentHarness,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
})
await generateSessionNameActivity({
@@ -1556,4 +1592,185 @@ describe('Agent Database Activities Integration', () => {
expect(piAgent.createAgentSession).not.toHaveBeenCalled()
})
})
describe('Nested AGENTS.md prompt injection', () => {
it('formatProjectContextPrompt formats context files into XML project_context block', () => {
expect(formatProjectContextPrompt([])).toBe('')
const formatted = formatProjectContextPrompt([
{ path: '/AGENTS.md', content: 'Root rules' },
{ path: '/subfolder/AGENTS.md', content: 'Subfolder rules' },
])
expect(formatted).toBe(
'\n\n<project_context>\n\nProject-specific instructions and guidelines:\n\n<project_instructions path="/AGENTS.md">\nRoot rules\n</project_instructions>\n\n<project_instructions path="/subfolder/AGENTS.md">\nSubfolder rules\n</project_instructions>\n\n</project_context>',
)
})
it('agentChatActivity injects nested AGENTS.md instructions into systemPrompt', async () => {
const user = await prisma.user.create({
data: { name: 'User_Chat_Md', email: `chat-md-${Date.now()}@example.com` },
})
const team = await prisma.team.create({
data: { name: 'Team_Chat_Md_' + Date.now() },
})
const project = await prisma.project.create({
data: { name: 'Project_Chat_Md', teamId: team.id },
})
const rootFolder = await prisma.asset.create({
data: {
name: 'Root',
type: AssetType.root,
projectId: project.id,
status: AssetStatus.uploaded,
},
})
await prisma.assetAgentMd.create({
data: {
assetId: rootFolder.id,
content: '# Root Policy',
},
})
const subfolder = await prisma.asset.create({
data: {
name: 'subfolder',
type: AssetType.folder,
parentId: rootFolder.id,
projectId: project.id,
status: AssetStatus.uploaded,
},
})
await prisma.assetAgentMd.create({
data: {
assetId: subfolder.id,
content: '# Subfolder Policy',
},
})
const mockHarness = {
subscribe: vi.fn(),
prompt: vi.fn().mockResolvedValue({
content: [{ type: 'text', text: 'Chat reply' }],
usage: { input: 10, output: 20 },
}),
}
const mockSession = {
getEntries: vi.fn().mockResolvedValue([]),
getStorage: vi.fn().mockReturnValue({ sessionId: 'mock-session-id' }),
}
vi.mocked(piAgent.createAgentSession).mockResolvedValue({
session: mockSession as unknown as Session<DatabaseSessionMetadata>,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
})
const context = {
agent: { id: 'agent-1', provider: { name: 'google' }, modelRef: { modelId: 'gemini' } },
dbProviders: [],
teamSkills: [],
allowedDomains: [],
} as unknown as AgentExecutionContext
await agentChatActivity({
teamId: team.id,
agentId: 'agent-1',
message: 'Hello',
imageUrls: [],
projectId: project.id,
folderId: subfolder.id,
assetId: subfolder.id,
sessionId: 'mock-session-id',
userId: user.id,
context,
})
expect(piAgent.createAgentSession).toHaveBeenCalledWith(
expect.objectContaining({
systemPrompt: expect.stringContaining('<project_context>'),
}),
)
const lastCall = vi.mocked(piAgent.createAgentSession).mock.calls.at(-1)
const systemPromptArg = lastCall?.[0].systemPrompt as string
expect(systemPromptArg).toContain(
'<project_instructions path="/AGENTS.md">\n# Root Policy\n</project_instructions>',
)
expect(systemPromptArg).toContain(
'<project_instructions path="/subfolder/AGENTS.md">\n# Subfolder Policy\n</project_instructions>',
)
})
it('agentChatActivity injects # Comment Threads into systemPrompt when userCommentId is present', async () => {
const user = await prisma.user.create({
data: { name: 'User_Comment_Prompt', email: `comment-prompt-${Date.now()}@example.com` },
})
const team = await prisma.team.create({
data: { name: 'Team_Comment_Prompt_' + Date.now() },
})
const project = await prisma.project.create({
data: { name: 'Project_Comment_Prompt', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'Asset_1',
type: AssetType.file,
projectId: project.id,
status: AssetStatus.uploaded,
},
})
const comment = await prisma.assetComment.create({
data: {
assetId: asset.id,
message: 'Question',
creatorId: user.id,
},
})
const mockHarness = {
subscribe: vi.fn(),
prompt: vi.fn().mockResolvedValue({
content: [{ type: 'text', text: 'Chat reply' }],
usage: { input: 10, output: 20 },
}),
}
const mockSession = {
getEntries: vi.fn().mockResolvedValue([]),
getStorage: vi.fn().mockReturnValue({ sessionId: 'mock-session-id' }),
}
vi.mocked(piAgent.createAgentSession).mockResolvedValue({
session: mockSession as unknown as Session<DatabaseSessionMetadata>,
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Mock AgentHarness instance for activity test
harness: mockHarness as unknown as AgentHarness<any, any, any, any>,
})
const context = {
agent: { id: 'agent-1', provider: { name: 'google' }, modelRef: { modelId: 'gemini' } },
dbProviders: [],
teamSkills: [],
allowedDomains: [],
} as unknown as AgentExecutionContext
await agentChatActivity({
teamId: team.id,
agentId: 'agent-1',
message: 'Hello',
imageUrls: [],
projectId: project.id,
folderId: asset.id,
assetId: asset.id,
sessionId: 'mock-session-id',
userCommentId: comment.id,
userId: user.id,
context,
})
const lastCall = vi.mocked(piAgent.createAgentSession).mock.calls.at(-1)
const systemPromptArg = lastCall?.[0].systemPrompt as string
expect(systemPromptArg).toContain('# Comment Threads')
expect(systemPromptArg).toContain('<thread id="..." reply_count="..." />')
expect(systemPromptArg).toContain('read_thread')
})
})
})
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,354 @@
import { describe, it, expect } from 'vitest'
import { escapeXmlAttr, serializeContextToXml } from './serialize-context'
import type { ShumaiMessageContext } from '@shumai/dtos'
describe('serializeContextToXml', () => {
describe('escapeXmlAttr', () => {
it('escapes &, <, >, ", and \' characters correctly', () => {
expect(escapeXmlAttr('Alice & Bob <Admin> "v1" \'test\'')).toBe(
'Alice &amp; Bob &lt;Admin&gt; &quot;v1&quot; &apos;test&apos;',
)
})
it('handles clean strings without modification', () => {
expect(escapeXmlAttr('normal-string_123.mp4')).toBe('normal-string_123.mp4')
})
})
describe('Tag Serialization', () => {
it('returns empty string for null, undefined, or empty context', () => {
expect(serializeContextToXml(null)).toBe('')
expect(serializeContextToXml(undefined)).toBe('')
expect(serializeContextToXml({})).toBe('')
})
it('serializes minimal context with only user', () => {
const context: ShumaiMessageContext = {
user: { id: 'usr_1', name: 'Alice', role: 'editor' },
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <user id="usr_1" name="Alice" role="editor" />\n</context>',
)
})
it('serializes context with user and thread tag with reply_count', () => {
const context: ShumaiMessageContext = {
user: { id: 'usr_1', name: 'Alice', role: 'editor' },
thread: { id: 'th_01JXYZ', replyCount: 3 },
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <user id="usr_1" name="Alice" role="editor" />\n <thread id="th_01JXYZ" reply_count="3" />\n</context>',
)
})
it('serializes context with thread tag without reply_count', () => {
const context: ShumaiMessageContext = {
thread: { id: 'th_01JXYZ' },
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <thread id="th_01JXYZ" />\n</context>',
)
})
it('serializes minimal context with only currentAsset (without ancestors)', () => {
const context: ShumaiMessageContext = {
currentAsset: {
id: 'ast_1',
name: 'clip.mp4',
type: 'file',
mediaType: 'video',
path: 'clip.mp4',
},
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <current_asset id="ast_1" name="clip.mp4" type="file" media_type="video" path="clip.mp4" />\n</context>',
)
})
it('serializes full video context with duration, timestamp, annotation, attached files, and referenced assets', () => {
const context: ShumaiMessageContext = {
user: { id: 'usr_01', name: 'Alice Chen', role: 'editor' },
currentAsset: {
id: 'ast_01JC04_CURRENT',
name: 'product_launch_v2.mp4',
type: 'file',
mediaType: 'video',
parentId: 'ast_01JC03_PARENT',
path: 'Marketing/2026_Launch/Footage/product_launch_v2.mp4',
durationSeconds: 85.4,
ancestors: [
{ id: 'ast_01JC01_ROOT', name: 'Marketing' },
{ id: 'ast_01JC02_SUB', name: '2026_Launch' },
{ id: 'ast_01JC03_PARENT', name: 'Footage' },
],
},
position: { type: 'time', seconds: 14.25 },
annotation: true,
attachedFiles: [
{
id: 'ast_01JC05_ATT1',
name: 'brand_guidelines_2026.pdf',
type: 'file',
mediaType: 'pdf',
path: 'Brand/brand_guidelines_2026.pdf',
},
],
referencedAssets: [
{
id: 'ast_01JC07_REF2',
name: 'Color_LUTs',
type: 'folder',
path: 'Assets/Color_LUTs',
},
{
id: 'ast_01JC06_REF1',
name: 'logo_white.png',
type: 'file',
mediaType: 'image',
path: 'Brand/Logos/logo_white.png',
},
],
}
const expected = `<context>
<user id="usr_01" name="Alice Chen" role="editor" />
<current_asset id="ast_01JC04_CURRENT" name="product_launch_v2.mp4" type="file" media_type="video" parent_id="ast_01JC03_PARENT" path="Marketing/2026_Launch/Footage/product_launch_v2.mp4" duration_seconds="85.40">
<ancestors>
<folder id="ast_01JC01_ROOT" name="Marketing" />
<folder id="ast_01JC02_SUB" name="2026_Launch" />
<folder id="ast_01JC03_PARENT" name="Footage" />
</ancestors>
</current_asset>
<position type="time" seconds="14.2500" />
<annotation />
<attached_files>
<file id="ast_01JC05_ATT1" name="brand_guidelines_2026.pdf" type="file" media_type="pdf" path="Brand/brand_guidelines_2026.pdf" />
</attached_files>
<referenced_assets>
<asset id="ast_01JC06_REF1" name="logo_white.png" type="file" media_type="image" path="Brand/Logos/logo_white.png" />
<folder id="ast_01JC07_REF2" name="Color_LUTs" path="Assets/Color_LUTs" />
</referenced_assets>
</context>`
expect(serializeContextToXml(context)).toBe(expected)
})
it('serializes PDF document context with total_pages and position.type === "page"', () => {
const context: ShumaiMessageContext = {
currentAsset: {
id: 'pdf_1',
name: 'report.pdf',
type: 'file',
mediaType: 'pdf',
totalPages: 12,
},
position: { type: 'page', page: 3 },
}
const expected = `<context>
<current_asset id="pdf_1" name="report.pdf" type="file" media_type="pdf" total_pages="12" />
<position type="page" page="3" />
</context>`
expect(serializeContextToXml(context)).toBe(expected)
})
it('serializes Video context with total_frames and duration_seconds', () => {
const context: ShumaiMessageContext = {
currentAsset: {
id: 'vid_1',
name: 'clip.mp4',
type: 'file',
mediaType: 'video',
mimeType: 'video/mp4',
durationSeconds: 9,
totalFrames: 268,
},
}
const expected = `<context>
<current_asset id="vid_1" name="clip.mp4" type="file" media_type="video" mime_type="video/mp4" duration_seconds="9.00" total_frames="268" />
</context>`
expect(serializeContextToXml(context)).toBe(expected)
})
it('serializes navigation context with navigated="true" and ancestors hierarchy', () => {
const context: ShumaiMessageContext = {
currentAsset: {
id: 'f_sub',
name: 'Subfolder',
type: 'folder',
navigated: true,
ancestors: [{ id: 'f_root', name: 'Root' }],
},
}
const expected = `<context>
<current_asset id="f_sub" name="Subfolder" type="folder" navigated="true">
<ancestors>
<folder id="f_root" name="Root" />
</ancestors>
</current_asset>
</context>`
expect(serializeContextToXml(context)).toBe(expected)
})
it('serializes mime_type alongside media_type for current asset and attached files', () => {
const context: ShumaiMessageContext = {
currentAsset: {
id: 'img_1',
name: 'photo.png',
type: 'file',
mediaType: 'image',
mimeType: 'image/png',
},
attachedFiles: [
{
id: 'doc_1',
name: 'spec.pdf',
type: 'file',
mediaType: 'pdf',
mimeType: 'application/pdf',
},
],
}
const expected = `<context>
<current_asset id="img_1" name="photo.png" type="file" media_type="image" mime_type="image/png" />
<attached_files>
<file id="doc_1" name="spec.pdf" type="file" media_type="pdf" mime_type="application/pdf" />
</attached_files>
</context>`
expect(serializeContextToXml(context)).toBe(expected)
})
})
describe('XML Escaping & Security', () => {
it('escapes user names, asset names, and paths with special characters', () => {
const context: ShumaiMessageContext = {
user: {
id: 'u_1',
name: 'Alice & Bob <Admin>',
role: 'reviewer "lead"',
},
currentAsset: {
id: 'ast_1',
name: 'Final <Cut> "v2" & Test.mp4',
type: 'file',
path: 'Folder A > Folder B & C/clip.mp4',
ancestors: [{ id: 'anc_1', name: 'Folder B & C' }],
},
attachedFiles: [
{
id: 'att_1',
name: "Don't <Touch> & Go.pdf",
type: 'file',
path: "Path 'A' & 'B'/doc.pdf",
},
],
}
const xml = serializeContextToXml(context)
expect(xml).toContain('name="Alice &amp; Bob &lt;Admin&gt;"')
expect(xml).toContain('role="reviewer &quot;lead&quot;"')
expect(xml).toContain('name="Final &lt;Cut&gt; &quot;v2&quot; &amp; Test.mp4"')
expect(xml).toContain('path="Folder A &gt; Folder B &amp; C/clip.mp4"')
expect(xml).toContain('name="Folder B &amp; C"')
expect(xml).toContain('name="Don&apos;t &lt;Touch&gt; &amp; Go.pdf"')
expect(xml).toContain('path="Path &apos;A&apos; &amp; &apos;B&apos;/doc.pdf"')
// Verify no unescaped dangerous characters exist inside attribute quotes
const attrMatch = xml.match(/\w+="([^"]*)"/g)
expect(attrMatch).not.toBeNull()
})
})
describe('Order Stability & Prompt Caching Invariants', () => {
it('produces identical XML regardless of object key order', () => {
const contextA: ShumaiMessageContext = {
user: { role: 'editor', name: 'Alice', id: 'u_1' },
currentAsset: {
name: 'video.mp4',
type: 'file',
id: 'ast_1',
mediaType: 'video',
},
position: { seconds: 12.345, type: 'time' },
}
const contextB: ShumaiMessageContext = {
position: { type: 'time', seconds: 12.345 },
currentAsset: {
id: 'ast_1',
mediaType: 'video',
type: 'file',
name: 'video.mp4',
},
user: { id: 'u_1', name: 'Alice', role: 'editor' },
}
expect(serializeContextToXml(contextA)).toBe(serializeContextToXml(contextB))
})
it('sorts attachedFiles and referencedAssets by id ascending', () => {
const context: ShumaiMessageContext = {
attachedFiles: [
{ id: 'z_file', name: 'Z', type: 'file' },
{ id: 'a_file', name: 'A', type: 'file' },
{ id: 'm_file', name: 'M', type: 'file' },
],
referencedAssets: [
{ id: 'z_asset', name: 'Z', type: 'file' },
{ id: 'a_asset', name: 'A', type: 'folder' },
{ id: 'm_asset', name: 'M', type: 'file' },
],
}
const xml = serializeContextToXml(context)
const fileOrder = [...xml.matchAll(/<file id="([^"]+)"/g)].map((m) => m[1])
const refOrder = [...xml.matchAll(/<(?:asset|folder) id="([^"]+)"/g)].map((m) => m[1])
expect(fileOrder).toEqual(['a_file', 'm_file', 'z_file'])
expect(refOrder).toEqual(['a_asset', 'm_asset', 'z_asset'])
})
it('normalizes float formatting with fixed precision (toFixed(4) for position, toFixed(2) for duration)', () => {
const context1: ShumaiMessageContext = {
currentAsset: { id: '1', name: 'a', type: 'file', durationSeconds: 14 },
position: { type: 'time', seconds: 5.5 },
}
const context2: ShumaiMessageContext = {
currentAsset: { id: '1', name: 'a', type: 'file', durationSeconds: 14.0001 },
position: { type: 'time', seconds: 5.50002 },
}
expect(serializeContextToXml(context1)).toContain('duration_seconds="14.00"')
expect(serializeContextToXml(context1)).toContain('seconds="5.5000"')
expect(serializeContextToXml(context2)).toContain('duration_seconds="14.00"')
expect(serializeContextToXml(context2)).toContain('seconds="5.5000"')
})
it('serializes annotation tag with id when context.id is provided', () => {
const context: ShumaiMessageContext = {
id: '01JABCDEF1234567890',
annotation: true,
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <annotation id="01JABCDEF1234567890" />\n</context>',
)
})
it('escapes special XML characters in annotation id', () => {
const context: ShumaiMessageContext = {
id: 'test<id>&"123"',
annotation: true,
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <annotation id="test&lt;id&gt;&amp;&quot;123&quot;" />\n</context>',
)
})
})
})
@@ -0,0 +1 @@
export { escapeXmlAttr, serializeContextToXml } from '@shumai/dtos'
@@ -5,10 +5,9 @@ import { type SessionTreeEntry } from '@earendil-works/pi-agent-core'
import { describe, expect, it, vi, afterEach } from 'vitest'
import { DatabaseSessionStorage } from './database-session-storage'
import { agentService } from '@shumai/core/src/agent/agent'
import { createAgentSession } from './index'
import { createAgentSession, type DbProviderInfo } from './index'
import * as sandboxedBashModule from './tools/sandboxed-bash'
import * as analyzeImageModule from './tools/analyze-image'
import * as screenshotModule from './tools/screenshot'
import * as readAssetModule from './tools/read-asset'
describe('DatabaseSessionStorage', () => {
setupTestDbHooks()
@@ -25,6 +24,20 @@ describe('DatabaseSessionStorage', () => {
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
}
const mockProviders: DbProviderInfo[] = [
{
name: 'test-provider',
config: { api: 'openai-responses', apiKey: 'TEST_KEY' },
models: [
{
modelId: 'test-model',
name: 'Test Model',
config: mockModelConfig,
},
],
},
]
const setupTestData = async () => {
const team = await prisma.team.create({ data: { name: 'Test Team' } })
const provider = await prisma.provider.create({
@@ -340,7 +353,7 @@ describe('DatabaseSessionStorage', () => {
allowedDomains: [],
sessionId: sessionId,
userId: user.id,
providers: [],
providers: mockProviders,
})
// 7. Verify skill environment variables are restored.
@@ -437,9 +450,8 @@ describe('DatabaseSessionStorage', () => {
})
const sessionId = initialStorage.sessionId
// 2. Spy on media tools constructors
const createAnalyzeImageToolSpy = vi.spyOn(analyzeImageModule, 'createAnalyzeImageTool')
const createScreenshotToolSpy = vi.spyOn(screenshotModule, 'createScreenshotTool')
// 2. Spy on media tool constructor
const createReadAssetToolSpy = vi.spyOn(readAssetModule, 'createReadAssetTool')
// 3. Resume the session via createAgentSession with a NEW comment ID
await createAgentSession({
@@ -453,15 +465,14 @@ describe('DatabaseSessionStorage', () => {
sessionId: sessionId,
userId: user.id,
userCommentId: 'new-comment-456',
providers: [],
providers: mockProviders,
})
// 4. Assert that the tools were instantiated using the new comment ID and user ID
expect(createAnalyzeImageToolSpy).toHaveBeenCalledWith(user.id, 'new-comment-456')
expect(createScreenshotToolSpy).toHaveBeenCalledWith(user.id, 'new-comment-456')
// 4. Assert that the tool was instantiated using user ID
expect(createReadAssetToolSpy).toHaveBeenCalledWith(user.id)
})
it('should use the current comment ID for media tools in subsequent turns (video)', async () => {
it('should instantiate media tools with user ID in subsequent turns (video)', async () => {
const { agent, user, team } = await setupTestData()
// Create a project and asset to satisfy FK constraints
@@ -491,9 +502,8 @@ describe('DatabaseSessionStorage', () => {
})
const sessionId = initialStorage.sessionId
// 2. Spy on media tools constructors
const createAnalyzeImageToolSpy = vi.spyOn(analyzeImageModule, 'createAnalyzeImageTool')
const createScreenshotToolSpy = vi.spyOn(screenshotModule, 'createScreenshotTool')
// 2. Spy on media tool constructor
const createReadAssetToolSpy = vi.spyOn(readAssetModule, 'createReadAssetTool')
// 3. Resume the session via createAgentSession with a NEW comment ID
await createAgentSession({
@@ -507,12 +517,11 @@ describe('DatabaseSessionStorage', () => {
sessionId: sessionId,
userId: user.id,
userCommentId: 'new-comment-456',
providers: [],
providers: mockProviders,
})
// 4. Assert that the tools were instantiated using the new comment ID and user ID
expect(createScreenshotToolSpy).toHaveBeenCalledWith(user.id, 'new-comment-456')
expect(createAnalyzeImageToolSpy).toHaveBeenCalledWith(user.id, 'new-comment-456')
// 4. Assert that the tool was instantiated using user ID
expect(createReadAssetToolSpy).toHaveBeenCalledWith(user.id)
})
describe('P1 Bug Reproductions', () => {
@@ -583,5 +592,223 @@ describe('DatabaseSessionStorage', () => {
updateSpy.mockRestore()
})
it('should return session name from agentSession record', async () => {
const { agent } = await setupTestData()
const storage = await DatabaseSessionStorage.create({ agentId: agent.id })
expect(await storage.getSessionName()).toBeUndefined()
await prisma.agentSession.update({
where: { id: storage.sessionId },
data: { name: 'My Chat Session' },
})
expect(await storage.getSessionName()).toBe('My Chat Session')
})
it('should aggregate session stats correctly', async () => {
const { agent } = await setupTestData()
const storage = await DatabaseSessionStorage.create({ agentId: agent.id })
const msg1: SessionTreeEntry = {
type: 'message',
id: 'stats-msg-1',
parentId: null,
timestamp: new Date().toISOString(),
message: {
role: 'user',
content: [{ type: 'text', text: 'Hello' }],
timestamp: Date.now(),
},
}
const msg2: SessionTreeEntry = {
type: 'message',
id: 'stats-msg-2',
parentId: 'stats-msg-1',
timestamp: new Date().toISOString(),
message: {
role: 'assistant',
content: [{ type: 'text', text: 'Hi there!' }],
api: 'openai-responses',
provider: 'test-provider',
model: 'test-model',
stopReason: 'stop',
timestamp: Date.now(),
usage: {
input: 100,
output: 50,
cacheRead: 20,
cacheWrite: 0,
totalTokens: 170,
cost: { input: 0.01, output: 0.02, cacheRead: 0.002, cacheWrite: 0, total: 0.032 },
},
},
}
await storage.appendEntry(msg1)
await storage.appendEntry(msg2)
const stats = await storage.getSessionStats()
expect(stats.messageCount).toBe(2)
expect(stats.cachedTokens).toBe(20)
expect(stats.uncachedTokens).toBe(150)
expect(stats.totalTokens).toBe(170)
expect(stats.costTotal).toBeCloseTo(0.032)
})
it('should save user message turn as custom_message (shumai_message) in DB when currentMessageContext is set', async () => {
const { agent, user } = await setupTestData()
const storage = await DatabaseSessionStorage.create({ agentId: agent.id, userId: user.id })
storage.currentMessageContext = {
user: { id: user.id, name: user.name, role: 'owner' },
currentAsset: { id: 'asset-123', name: 'video.mp4', type: 'file' },
}
const userEntry: SessionTreeEntry = {
type: 'message',
id: 'msg-user-1',
parentId: null,
timestamp: new Date().toISOString(),
message: {
role: 'user',
content: 'Hello, help me analyze this video',
timestamp: Date.now(),
},
}
await storage.appendEntry(userEntry)
// 1. Verify DB record type is 'custom_message', not 'message'
const record = await prisma.agentSessionEntry.findUnique({
where: { id: 'msg-user-1' },
})
expect(record).toBeDefined()
expect(record?.type).toBe('custom_message')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const data = record?.data as any
expect(data.customType).toBe('shumai_message')
expect(data.content).toBe('Hello, help me analyze this video')
expect(data.details?.user?.id).toBe(user.id)
expect(data.details?.currentAsset?.id).toBe('asset-123')
// 2. Verify getEntries() and getPathToRoot() do not throw and return proper custom_message
const entries = await storage.getEntries()
expect(entries).toHaveLength(1)
expect(entries[0].type).toBe('custom_message')
const pathEntries = await storage.getPathToRoot('msg-user-1')
expect(pathEntries).toHaveLength(1)
expect(pathEntries[0].type).toBe('custom_message')
const stats = await storage.getSessionStats()
expect(stats.messageCount).toBe(1)
})
it('should safely handle legacy records where type was message but data was shumai_message', async () => {
const { agent } = await setupTestData()
const storage = await DatabaseSessionStorage.create({ agentId: agent.id })
await prisma.agentSessionEntry.create({
data: {
id: 'legacy-corrupted-entry',
sessionId: storage.sessionId,
type: 'message',
data: {
customType: 'shumai_message',
content: 'Legacy prompt',
display: true,
details: { user: { id: 'u1', name: 'User 1' } },
},
},
})
// Must not throw in getEntries(), getPathToRoot(), or getSessionStats()
const entries = await storage.getEntries()
expect(entries).toHaveLength(1)
expect(entries[0].type).toBe('custom_message')
const path = await storage.getPathToRoot('legacy-corrupted-entry')
expect(path).toHaveLength(1)
expect(path[0].type).toBe('custom_message')
const stats = await storage.getSessionStats()
expect(stats.messageCount).toBe(1)
})
it('should dynamically inject thread metadata into details.thread in getPathToRoot when replies exist', async () => {
const { agent, user } = await setupTestData()
const project = await prisma.project.create({
data: { name: 'Test Project', teamId: agent.teamId },
})
const asset = await prisma.asset.create({
data: {
name: 'Test Asset',
type: 'file',
status: 'uploaded',
projectId: project.id,
},
})
const storage = await DatabaseSessionStorage.create({ agentId: agent.id, assetId: asset.id })
const rootComment = await prisma.assetComment.create({
data: {
id: 'comment-root-1',
assetId: asset.id,
message: 'Top-level question',
creatorId: user.id,
},
})
// Add a reply
await prisma.assetComment.create({
data: {
id: 'comment-reply-1',
assetId: asset.id,
message: 'Reply 1',
creatorId: user.id,
replyToId: rootComment.id,
},
})
// Create session entry for root comment
await prisma.agentSessionEntry.create({
data: {
id: rootComment.id,
sessionId: storage.sessionId,
assetId: asset.id,
type: 'custom_message',
data: {
customType: 'shumai_message',
content: 'Top-level question',
display: true,
details: { user: { id: user.id, name: user.name } },
},
},
})
const path = await storage.getPathToRoot(rootComment.id)
expect(path).toHaveLength(1)
expect(path[0].type).toBe('custom_message')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const details = (path[0] as any).details
expect(details.thread).toEqual({
id: rootComment.id,
replyCount: 1,
})
// eslint-disable-next-line @typescript-eslint/no-explicit-any
expect((path[0] as any).content).toBe('Top-level question')
const entries = await storage.getEntries()
expect(entries).toHaveLength(1)
expect(entries[0].type).toBe('custom_message')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
expect((entries[0] as any).details?.thread).toEqual({
id: rootComment.id,
replyCount: 1,
})
})
})
})
+174 -53
View File
@@ -8,6 +8,7 @@ import {
type SessionTreeEntry,
} from '@earendil-works/pi-agent-core'
import { agentService } from '@shumai/core/src/agent/agent'
import type { ShumaiMessageContext } from '@shumai/dtos'
export interface DatabaseSessionMetadata extends SessionMetadata {
agentId: string
@@ -72,6 +73,7 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
}
nextEntryId?: string | null
currentMessageContext?: PrismaJson.ShumaiMessageContext
async createEntryId(): Promise<string> {
if (this.nextEntryId) {
@@ -90,9 +92,13 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
data: unknown
}): SessionTreeEntry {
const payload = (record.data as Record<string, unknown>) || {}
let type = (record.type || 'message') as SessionTreeEntry['type']
if (type === 'message' && payload.customType === 'shumai_message') {
type = 'custom_message'
}
return {
id: record.id,
type: (record.type || 'message') as SessionTreeEntry['type'],
type,
parentId: record.parentId,
timestamp: record.createdAt.toISOString(),
...payload,
@@ -100,10 +106,44 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
}
async appendEntry(entry: SessionTreeEntry): Promise<void> {
const strippedEntry = structuredClone(entry)
let entryToProcess = entry
if (
entryToProcess.type === 'message' &&
entryToProcess.message &&
entryToProcess.message.role === 'user' &&
this.currentMessageContext
) {
let contentText = ''
if (typeof entryToProcess.message.content === 'string') {
contentText = entryToProcess.message.content
} else if (Array.isArray(entryToProcess.message.content)) {
contentText = entryToProcess.message.content
.map((c) =>
typeof c === 'object' && c && 'text' in c && typeof c.text === 'string' ? c.text : '',
)
.join('')
}
entryToProcess = {
type: 'custom_message',
id: entryToProcess.id,
parentId: entryToProcess.parentId,
timestamp: entryToProcess.timestamp,
customType: 'shumai_message',
content: contentText,
display: true,
details: {
...this.currentMessageContext,
id: entryToProcess.id,
},
} as unknown as SessionTreeEntry
this.currentMessageContext = undefined
}
const strippedEntry = structuredClone(entryToProcess)
// Strip image data and skill content before saving to DB
if (strippedEntry.type === 'message') {
if (strippedEntry.type === 'message' && strippedEntry.message) {
const msg = strippedEntry.message
if (msg.role === 'toolResult') {
const details = msg.details as { sourceKeys?: string[]; skillId?: string } | undefined
@@ -139,7 +179,10 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
}
}
const targetLeafId = entry.type === 'leaf' ? entry.targetId : entry.id
const targetLeafId =
strippedEntry.type === 'leaf'
? (strippedEntry as unknown as { targetId?: string }).targetId || strippedEntry.id
: strippedEntry.id
// Extract base fields and store type-specific properties in data payload
const payload = { ...strippedEntry } as Record<string, unknown>
@@ -155,17 +198,18 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
await prisma.$transaction([
prisma.agentSessionEntry.upsert({
where: { id: entry.id },
where: { id: strippedEntry.id },
create: {
id: entry.id,
id: strippedEntry.id,
sessionId: this.sessionId,
assetId: session?.assetId || null,
type: entry.type,
parentId: entry.parentId || null,
createdAt: entry.timestamp ? new Date(entry.timestamp) : new Date(),
type: strippedEntry.type,
parentId: strippedEntry.parentId || null,
createdAt: strippedEntry.timestamp ? new Date(strippedEntry.timestamp) : new Date(),
data: payload as PrismaJson.PiSessionEntryData,
},
update: {
type: strippedEntry.type,
data: payload as PrismaJson.PiSessionEntryData,
},
}),
@@ -212,6 +256,74 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
return (entry as { label?: string } | undefined)?.label
}
async getSessionName(): Promise<string | undefined> {
const session = await prisma.agentSession.findUnique({
where: { id: this.sessionId },
select: { name: true },
})
return session?.name || undefined
}
async getSessionStats(): Promise<{
messageCount: number
cachedTokens: number
uncachedTokens: number
totalTokens: number
costTotal: number
}> {
const entries = await this.getEntries()
let messageCount = 0
let cachedTokens = 0
let uncachedTokens = 0
let totalTokens = 0
let costTotal = 0
for (const entry of entries) {
if (entry.type === 'custom_message' && entry.customType === 'shumai_message') {
messageCount++
} else if (entry.type === 'message' && entry.message) {
const msg = entry.message
if (msg.role === 'user' || msg.role === 'assistant') {
messageCount++
}
if (msg.role === 'assistant' && msg.usage) {
cachedTokens += msg.usage.cacheRead || 0
uncachedTokens += (msg.usage.input || 0) + (msg.usage.output || 0)
totalTokens +=
msg.usage.totalTokens ||
(msg.usage.input || 0) + (msg.usage.output || 0) + (msg.usage.cacheRead || 0)
if (msg.usage.cost) {
costTotal += msg.usage.cost.total || 0
}
}
} else if (entry.type === 'compaction' || entry.type === 'branch_summary') {
const usage = entry.usage
if (usage) {
cachedTokens += usage.cacheRead || 0
uncachedTokens += (usage.input || 0) + (usage.output || 0)
totalTokens +=
usage.totalTokens || (usage.input || 0) + (usage.output || 0) + (usage.cacheRead || 0)
if (usage.cost) {
costTotal += usage.cost.total || 0
}
}
}
}
return {
messageCount,
cachedTokens,
uncachedTokens,
totalTokens,
costTotal,
}
}
async getPathToRootOrCompaction(leafId: string | null): Promise<SessionTreeEntry[]> {
return this.getPathToRoot(leafId)
}
async getPathToRoot(leafId: string | null): Promise<SessionTreeEntry[]> {
if (!leafId) return []
@@ -258,48 +370,7 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
await this.reinjectSkillContentAsync(entry)
}
// Dynamically tag comments that currently have reply threads
const entryIds = pathEntries.map((e) => e.id)
if (entryIds.length > 0) {
const threadCounts = await prisma.assetComment.groupBy({
by: ['replyToId'],
where: {
replyToId: { in: entryIds },
message: { not: '__CHAT__' },
},
_count: {
id: true,
},
})
const threadReplyCountMap = new Map<string, number>()
for (const tc of threadCounts) {
if (tc.replyToId) {
threadReplyCountMap.set(tc.replyToId, tc._count.id)
}
}
if (threadReplyCountMap.size > 0) {
for (let i = 0; i < pathEntries.length; i++) {
const entry = pathEntries[i]
const count = threadReplyCountMap.get(entry.id)
if (count !== undefined && count > 0 && entry.type === 'message') {
const cloned = structuredClone(entry)
const msg = cloned.message as unknown as {
content?: Array<{ type: string; text: string }>
}
if (Array.isArray(msg.content) && msg.content[0] && msg.content[0].type === 'text') {
const threadTag = `[Thread ID: ${entry.id}] [Replies: ${count}]`
if (!msg.content[0].text.includes(`[Thread ID: ${entry.id}]`)) {
msg.content[0].text = `${threadTag} ${msg.content[0].text}`
}
}
pathEntries[i] = cloned
}
}
}
}
return pathEntries
return this.injectThreadMetadataAsync(pathEntries)
}
async getEntries(): Promise<SessionTreeEntry[]> {
@@ -312,11 +383,61 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
await this.reinjectImageDataAsync(entry)
await this.reinjectSkillContentAsync(entry)
}
return this.injectThreadMetadataAsync(entries)
}
private async injectThreadMetadataAsync(
entries: SessionTreeEntry[],
): Promise<SessionTreeEntry[]> {
const entryIds = entries.map((e) => e.id)
if (entryIds.length === 0) {
return entries
}
const threadCounts = await prisma.assetComment.groupBy({
by: ['replyToId'],
where: {
replyToId: { in: entryIds },
message: { not: '__CHAT__' },
},
_count: {
id: true,
},
})
const threadReplyCountMap = new Map<string, number>()
for (const tc of threadCounts) {
if (tc.replyToId) {
threadReplyCountMap.set(tc.replyToId, tc._count.id)
}
}
if (threadReplyCountMap.size > 0) {
for (let i = 0; i < entries.length; i++) {
const entry = entries[i]
const count = threadReplyCountMap.get(entry.id)
if (count !== undefined && count > 0) {
if (entry.type === 'custom_message') {
const cloned = structuredClone(entry)
const existingDetails = ((cloned as unknown as { details?: ShumaiMessageContext })
.details || {}) as ShumaiMessageContext
;(cloned as unknown as { details: ShumaiMessageContext }).details = {
...existingDetails,
thread: {
id: entry.id,
replyCount: count,
},
}
entries[i] = cloned
}
}
}
}
return entries
}
private async reinjectImageDataAsync(entry: SessionTreeEntry) {
if (entry.type === 'message') {
if (entry.type === 'message' && entry.message) {
const msg = entry.message as AgentMessage
if (msg.role === 'toolResult') {
const details = msg.details as { sourceKeys?: string[] } | undefined
@@ -341,7 +462,7 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
}
private async reinjectSkillContentAsync(entry: SessionTreeEntry) {
if (entry.type === 'message') {
if (entry.type === 'message' && entry.message) {
const msg = entry.message as AgentMessage
if (msg.role === 'toolResult' && msg.toolName === 'read_skill') {
const details = msg.details as { skillId?: string } | undefined
+810 -11
View File
@@ -1,16 +1,21 @@
import { describe, it, expect, vi } from 'vitest'
import { describe, it, expect, vi, afterEach } from 'vitest'
import {
formatSkillsForPrompt,
createAgentSession,
getModelFromDb,
buildRestrictedUserInstructions,
type DbProviderInfo,
} from './index'
import { createSandboxedBashTool } from './tools/sandboxed-bash'
import { SandboxManager, type SandboxAskCallback } from '@anthropic-ai/sandbox-runtime'
import { quotaService } from '@shumai/core/src/quota/quota-service'
import { prisma } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import { type TSchema } from 'typebox'
import { Value } from 'typebox/value'
import * as path from 'path'
import * as childProcess from 'node:child_process'
import * as fs from 'fs'
// eslint-disable-next-line @typescript-eslint/no-explicit-any
let mockSpawn: any = null
@@ -28,6 +33,30 @@ vi.mock('node:child_process', async () => {
}
})
// Mock fs so tests can stub skill content loading (matching read-skill.test.ts)
vi.mock('fs')
const mockProviders: DbProviderInfo[] = [
{
name: 'google',
config: { api: 'google-generative-ai', apiKey: 'GOOGLE_API_KEY' },
models: [
{
modelId: 'gemini',
name: 'Gemini',
config: {
api: 'google-generative-ai',
reasoning: false,
input: ['text'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 8192,
maxTokens: 4096,
},
},
],
},
]
describe('formatSkillsForPrompt', () => {
it('should return empty string if skills array is empty', () => {
expect(formatSkillsForPrompt([])).toBe('')
@@ -63,10 +92,16 @@ describe('Sandbox Network isolation integration', () => {
setupTestDbHooks()
it('should register callback that pushes blocked hosts to database and propagates to bash tool', async () => {
// 1. Create a team
// 1. Create a team and enabled sandbox
const team = await prisma.team.create({
data: { name: 'Test Sandbox Team' },
})
await prisma.sandbox.create({
data: {
teamId: team.id,
networkSandboxEnabled: true,
},
})
// Ensure agent user and agent exist in database to avoid foreign key violation
await prisma.user.create({
@@ -104,7 +139,7 @@ describe('Sandbox Network isolation integration', () => {
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: [],
providers: mockProviders,
})
// 4. Verify SandboxManager.initialize was called and extract the callback
@@ -151,7 +186,7 @@ describe('Sandbox Network isolation integration', () => {
return mockChild
}
const bashPromise = bashTool.execute('1', { command: 'curl blocked-api.com' })
const bashPromise = bashTool.execute('1', { command: 'curl blocked-api.com', source: 'user' })
try {
await expect(bashPromise).rejects.toThrow(
@@ -172,6 +207,12 @@ describe('Sandbox Network isolation integration', () => {
const team2 = await prisma.team.create({
data: { name: 'Team 2' },
})
await prisma.sandbox.create({
data: { teamId: team1.id, networkSandboxEnabled: true },
})
await prisma.sandbox.create({
data: { teamId: team2.id, networkSandboxEnabled: true },
})
// Ensure agent user and agents exist in database
await prisma.user.create({
@@ -232,7 +273,7 @@ describe('Sandbox Network isolation integration', () => {
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: [],
providers: mockProviders,
})
// 3. Call createAgentSession for Session 2
@@ -244,7 +285,7 @@ describe('Sandbox Network isolation integration', () => {
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: [],
providers: mockProviders,
})
expect(registeredCallback).toBeDefined()
@@ -264,6 +305,64 @@ describe('Sandbox Network isolation integration', () => {
}
})
it('should bypass domain blocking and pending domain recording when networkSandboxEnabled is false', async () => {
const team = await prisma.team.create({
data: { name: 'Disabled Sandbox Team' },
})
await prisma.sandbox.create({
data: { teamId: team.id, networkSandboxEnabled: false },
})
await prisma.user.create({
data: {
id: 'agent-disabled',
name: 'Ai Agent Disabled',
email: 'disabled@shumai.ai',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'agent-disabled',
teamId: team.id,
type: 'chat',
config: { provider: 'openai', model: 'gpt-4' },
},
})
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
vi.spyOn(SandboxManager, 'wrapWithSandbox').mockImplementation(async (cmd) => cmd)
await createAgentSession({
teamId: team.id,
agentId: 'agent-disabled',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: ['*'],
providers: mockProviders,
})
expect(initializeSpy).toHaveBeenCalledWith(
expect.objectContaining({
network: expect.objectContaining({
allowedDomains: ['*'],
}),
}),
expect.any(Function),
)
const callback = initializeSpy.mock.calls[0][1]
const res = await callback!({ host: 'any-external-domain.com', port: 443 })
expect(res).toBe(true)
const sandbox = await prisma.sandbox.findUnique({
where: { teamId: team.id },
})
expect(sandbox?.pendingDomains).toEqual([])
})
it('should pass thinkingLevel to AgentHarness and default to off', async () => {
const team = await prisma.team.create({
data: { name: 'Test Team' },
@@ -314,7 +413,7 @@ describe('Sandbox Network isolation integration', () => {
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: [],
providers: mockProviders,
thinkingLevel: 'high',
})
expect(harnessHigh.getThinkingLevel()).toBe('high')
@@ -328,7 +427,7 @@ describe('Sandbox Network isolation integration', () => {
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: [],
providers: mockProviders,
})
expect(harnessDefault.getThinkingLevel()).toBe('off')
} finally {
@@ -336,6 +435,49 @@ describe('Sandbox Network isolation integration', () => {
}
})
it('should pass retry configuration to AgentHarness', async () => {
const team = await prisma.team.create({
data: { name: 'Test Team Retry' },
})
await prisma.user.create({
data: {
id: 'test-agent-retry',
name: 'Agent Retry',
email: 'agent-retry@test.com',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'test-agent-retry',
teamId: team.id,
type: 'chat',
config: { provider: 'google', model: 'gemini' },
},
})
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
try {
const { harness } = await createAgentSession({
teamId: team.id,
agentId: 'test-agent-retry',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: mockProviders,
maxRetries: 5,
baseDelayMs: 1500,
})
// @ts-expect-error accessing private property for verification
expect(harness.streamOptions?.maxRetries).toBe(5)
} finally {
initializeSpy.mockRestore()
}
})
it('should filter out denied tools from AgentHarness', async () => {
const team = await prisma.team.create({
data: { name: 'Test Team 3' },
@@ -367,7 +509,7 @@ describe('Sandbox Network isolation integration', () => {
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: [],
providers: mockProviders,
})
const tools = harness.getTools()
const bashTool = tools.find((t) => t.name === 'bash')
@@ -410,6 +552,12 @@ describe('getModelFromDb', () => {
expect(model?.provider).toBe('google')
expect(model?.api).toBe('google-generative-ai')
})
it('should throw an error when provider or model is not found in database configuration', () => {
expect(() => getModelFromDb([], 'google', 'non-existent')).toThrow(
'Provider "google" or model "non-existent" not found in database configuration',
)
})
})
describe('createAgentSession sandbox options', () => {
@@ -454,7 +602,7 @@ describe('createAgentSession sandbox options', () => {
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: [],
providers: mockProviders,
})
expect(initializeSpy).toHaveBeenCalledWith(
expect.objectContaining({ enableWeakerNestedSandbox: false }),
@@ -471,7 +619,7 @@ describe('createAgentSession sandbox options', () => {
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: [],
providers: mockProviders,
})
expect(initializeSpy).toHaveBeenLastCalledWith(
expect.objectContaining({ enableWeakerNestedSandbox: true }),
@@ -486,3 +634,654 @@ describe('createAgentSession sandbox options', () => {
initializeSpy.mockRestore()
})
})
describe('buildRestrictedUserInstructions', () => {
it('should describe the restricted user bash rules', () => {
const instructions = buildRestrictedUserInstructions()
expect(instructions).toContain('# Restricted User Context')
expect(instructions).toContain(
'NEVER execute a bash command that the user asks you to run directly',
)
expect(instructions).toContain('loaded via the `read_skill` tool')
expect(instructions).toContain('parameter to "skill"')
expect(instructions).toContain('source="user" will be rejected')
})
})
describe('createAgentSession bash restriction for non-owner users', () => {
setupTestDbHooks()
afterEach(() => {
vi.restoreAllMocks()
})
type TestHarness = Awaited<ReturnType<typeof createAgentSession>>['harness']
async function setupSession(opts: {
role: 'owner' | 'editor' | 'reviewer' | 'none' | 'non-member'
deniedTools?: string[]
sessionId?: string
}): Promise<{ team: { id: string }; userId?: string; harness: TestHarness }> {
const team = await prisma.team.create({ data: { name: 'Bash Restriction Team' } })
await prisma.user.create({
data: {
id: 'bash-agent',
name: 'Ai Agent Bash',
email: 'bash-agent@shumai.ai',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'bash-agent',
teamId: team.id,
type: 'chat',
config: {
provider: 'google',
model: 'gemini',
...(opts.deniedTools ? { deniedTools: opts.deniedTools } : {}),
},
},
})
let userId: string | undefined
if (opts.role === 'non-member') {
const user = await prisma.user.create({
data: { name: 'Non Member', email: 'non-member@shumai.ai' },
})
userId = user.id // no teamMember record created
} else if (opts.role !== 'none') {
const user = await prisma.user.create({
data: { name: 'Bash User', email: 'bash-user@shumai.ai' },
})
await prisma.teamMember.create({
data: { teamId: team.id, userId: user.id, role: opts.role },
})
userId = user.id
}
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
try {
const { harness } = await createAgentSession({
teamId: team.id,
agentId: 'bash-agent',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: mockProviders,
userId,
sessionId: opts.sessionId,
})
return { team, userId, harness }
} finally {
initializeSpy.mockRestore()
}
}
it('should include the bash tool from the start for team owners', async () => {
const { harness } = await setupSession({ role: 'owner' })
expect(harness.getTools().some((t) => t.name === 'bash')).toBe(true)
})
it('should include the bash tool from the start when there is no user context', async () => {
const { harness } = await setupSession({ role: 'none' })
expect(harness.getTools().some((t) => t.name === 'bash')).toBe(true)
})
it('should treat a user who is not a team member as restricted (fail-closed)', async () => {
const { harness } = await setupSession({ role: 'non-member' })
const tools = harness.getTools()
expect(tools.some((t) => t.name === 'bash')).toBe(false)
expect(tools.some((t) => t.name === 'read_skill')).toBe(true)
// @ts-expect-error accessing private property for verification
const prompt = await harness.systemPrompt()
expect(prompt).toContain('# Restricted User Context')
})
it.each(['editor', 'reviewer'] as const)(
'should exclude the bash tool initially for %s users',
async (role) => {
const { harness } = await setupSession({ role })
const tools = harness.getTools()
expect(tools.some((t) => t.name === 'bash')).toBe(false)
expect(tools.some((t) => t.name === 'read_skill')).toBe(true)
},
)
it('should include the bash tool from the start when a skill was already loaded earlier in the session', async () => {
const team = await prisma.team.create({ data: { name: 'Restore Team' } })
const user = await prisma.user.create({
data: { name: 'Restore User', email: 'restore@shumai.ai' },
})
await prisma.teamMember.create({
data: { teamId: team.id, userId: user.id, role: 'reviewer' },
})
await prisma.user.create({
data: {
id: 'bash-agent-restore',
name: 'Ai Agent Restore',
email: 'bash-agent-restore@shumai.ai',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'bash-agent-restore',
teamId: team.id,
type: 'chat',
config: { provider: 'google', model: 'gemini' },
},
})
const session = await prisma.agentSession.create({
data: {
agentId: 'bash-agent-restore',
userId: user.id,
cwd: process.cwd(),
type: 'chat',
},
})
await prisma.agentSessionEntry.create({
data: {
id: 'restore-entry-1',
sessionId: session.id,
type: 'message',
data: {
message: {
role: 'toolResult',
content: [{ type: 'text', text: 'skill content restored' }],
toolName: 'read_skill',
isError: false,
details: { skillId: 'previously-loaded-skill' },
timestamp: Date.now(),
},
},
},
})
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
try {
const { harness } = await createAgentSession({
teamId: team.id,
agentId: 'bash-agent-restore',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: mockProviders,
userId: user.id,
sessionId: session.id,
})
expect(harness.getTools().some((t) => t.name === 'bash')).toBe(true)
} finally {
initializeSpy.mockRestore()
}
})
it('should inject the bash tool after a skill is loaded via read_skill for restricted users', async () => {
const { team, harness } = await setupSession({ role: 'reviewer' })
const skill = await prisma.skill.create({
data: { name: 'Inject Skill', assetId: 'asset1', hash: 'inject-hash', teamId: team.id },
})
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs readFileSync which has complex overloaded signatures
vi.spyOn(fs, 'readFileSync').mockImplementation((p: any) => {
if (p.toString().endsWith('.hash')) return 'inject-hash'
if (p.toString().endsWith('SKILL.md')) return '# Inject Skill'
return ''
})
const readSkillTool = harness.getTools().find((t) => t.name === 'read_skill')
expect(readSkillTool).toBeDefined()
expect(harness.getTools().some((t) => t.name === 'bash')).toBe(false)
expect(harness.getActiveTools().some((t) => t.name === 'bash')).toBe(false)
await readSkillTool!.execute('1', { skillId: skill.id }, undefined, undefined, undefined)
expect(harness.getTools().some((t) => t.name === 'bash')).toBe(true)
// bash must be part of the ACTIVE tool set, not just the registry, or the model
// would never receive it on the next turn
expect(harness.getActiveTools().some((t) => t.name === 'bash')).toBe(true)
})
it('should not inject the bash tool for restricted users when bash is in deniedTools', async () => {
const { team, harness } = await setupSession({ role: 'editor', deniedTools: ['bash'] })
const skill = await prisma.skill.create({
data: { name: 'Denied Skill', assetId: 'asset1', hash: 'denied-hash', teamId: team.id },
})
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs readFileSync which has complex overloaded signatures
vi.spyOn(fs, 'readFileSync').mockImplementation((p: any) => {
if (p.toString().endsWith('.hash')) return 'denied-hash'
if (p.toString().endsWith('SKILL.md')) return '# Denied Skill'
return ''
})
const readSkillTool = harness.getTools().find((t) => t.name === 'read_skill')
expect(readSkillTool).toBeDefined()
await readSkillTool!.execute('1', { skillId: skill.id }, undefined, undefined, undefined)
expect(harness.getTools().some((t) => t.name === 'bash')).toBe(false)
})
it('should include restricted user instructions in the system prompt for non-owners', async () => {
const { harness } = await setupSession({ role: 'editor' })
// @ts-expect-error accessing private property for verification
const prompt = await harness.systemPrompt()
expect(prompt).toContain('# Restricted User Context')
expect(prompt).toContain('source="user" will be rejected')
})
it('should not include restricted user instructions in the system prompt for owners', async () => {
const { harness } = await setupSession({ role: 'owner' })
// @ts-expect-error accessing private property for verification
const prompt = await harness.systemPrompt()
expect(prompt).not.toContain('# Restricted User Context')
})
})
describe('createAgentSession typed metadata schema', () => {
setupTestDbHooks()
afterEach(() => {
vi.restoreAllMocks()
})
async function setupSessionWithProject(opts: { withCreationFields: boolean }) {
const team = await prisma.team.create({ data: { name: 'Typed Metadata Team' } })
const project = await prisma.project.create({
data: { name: 'Typed Metadata Project', teamId: team.id },
})
const user = await prisma.user.create({
data: { name: 'Typed Metadata User', email: 'typed-metadata@shumai.ai' },
})
await prisma.teamMember.create({
data: { teamId: team.id, userId: user.id, role: 'owner' },
})
await prisma.user.create({
data: {
id: 'typed-metadata-agent',
name: 'Ai Agent Typed Metadata',
email: 'typed-metadata-agent@shumai.ai',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'typed-metadata-agent',
teamId: team.id,
type: 'chat',
config: { provider: 'google', model: 'gemini' },
},
})
if (opts.withCreationFields) {
await prisma.metadataField.create({
data: {
key: 'prompt',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Prompt', type: 'text', autofillSource: 'CREATION_CONTEXT' },
},
})
await prisma.metadataField.create({
data: {
key: 'source',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: {
name: 'Source',
type: 'select',
autofillSource: 'CREATION_CONTEXT',
select: {
options: [{ id: 'gemini', displayName: 'Gemini', color: '#ffffff' }],
},
},
},
})
await prisma.metadataField.create({
data: {
key: 'manualNotes',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Manual Notes', type: 'text', autofillSource: 'NONE' },
},
})
}
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
try {
const { harness } = await createAgentSession({
teamId: team.id,
agentId: 'typed-metadata-agent',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: mockProviders,
userId: user.id,
projectId: project.id,
})
return { harness }
} finally {
initializeSpy.mockRestore()
}
}
it('should build create_file/create_version with the project CREATION_CONTEXT fields', async () => {
const { harness } = await setupSessionWithProject({ withCreationFields: true })
const tools = harness.getTools()
expect(tools.some((t) => t.name === 'list_autofill_fields')).toBe(false)
const createFile = tools.find((t) => t.name === 'create_file')
const createVersion = tools.find((t) => t.name === 'create_version')
expect(createFile).toBeDefined()
expect(createVersion).toBeDefined()
const checkFile = (args: Record<string, unknown>) =>
Value.Check(createFile!.parameters as unknown as TSchema, {
parent: 'f1',
path: '/tmp/x.md',
data: null,
...args,
})
const checkVersion = (args: Record<string, unknown>) =>
Value.Check(createVersion!.parameters as unknown as TSchema, {
parent: 'f1',
path: '/tmp/x.md',
...args,
})
for (const check of [checkFile, checkVersion]) {
// Declared CREATION_CONTEXT fields (all-required, nullable) validate; other keys do not
expect(check({ metadata: { prompt: 'hello', source: null } })).toBe(true)
expect(check({ metadata: { prompt: null, source: 'gemini' } })).toBe(true)
expect(check({ metadata: null })).toBe(true)
expect(check({ metadata: { prompt: null, source: null, manualNotes: 'no' } })).toBe(false)
expect(check({ metadata: { prompt: null, source: null, bogus: 1 } })).toBe(false)
expect(check({ metadata: { prompt: 'hello' } })).toBe(false)
}
})
it('should omit the metadata parameter when the project has no CREATION_CONTEXT fields', async () => {
const { harness } = await setupSessionWithProject({ withCreationFields: false })
const createFile = harness.getTools().find((t) => t.name === 'create_file')
const createVersion = harness.getTools().find((t) => t.name === 'create_version')
expect(createFile).toBeDefined()
expect(createVersion).toBeDefined()
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
const fileProps = (createFile!.parameters as any).properties
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
const versionProps = (createVersion!.parameters as any).properties
expect('metadata' in fileProps).toBe(false)
expect('metadata' in versionProps).toBe(false)
})
})
describe('createAgentSession enabled skill filtering', () => {
setupTestDbHooks()
afterEach(() => {
vi.restoreAllMocks()
})
it('should only advertise and allow reading skills enabled for the agent', async () => {
const team = await prisma.team.create({ data: { name: 'Skill Filter Team' } })
const enabledSkill = await prisma.skill.create({
data: {
name: 'Enabled Skill',
assetId: 'asset-enabled',
hash: 'enabled-hash',
teamId: team.id,
},
})
const disabledSkill = await prisma.skill.create({
data: {
name: 'Disabled Skill',
assetId: 'asset-disabled',
hash: 'disabled-hash',
teamId: team.id,
},
})
await prisma.user.create({
data: {
id: 'skill-filter-agent',
name: 'Ai Agent Skill Filter',
email: 'skill-filter-agent@shumai.ai',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'skill-filter-agent',
teamId: team.id,
type: 'chat',
config: { provider: 'google', model: 'gemini' },
skills: {
create: [{ skillId: enabledSkill.id }],
},
},
})
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
try {
const { harness } = await createAgentSession({
teamId: team.id,
agentId: 'skill-filter-agent',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [{ id: enabledSkill.id, name: enabledSkill.name }],
enabledSkillIds: [enabledSkill.id],
allowedDomains: [],
providers: mockProviders,
})
// The disabled skill must not be advertised in the available skills list
// @ts-expect-error accessing private property for verification
const prompt = await harness.systemPrompt()
expect(prompt).toContain(enabledSkill.id)
expect(prompt).not.toContain(disabledSkill.id)
// read_skill must reject the disabled skill id
const readSkillTool = harness.getTools().find((t) => t.name === 'read_skill')
expect(readSkillTool).toBeDefined()
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs readFileSync which has complex overloaded signatures
vi.spyOn(fs, 'readFileSync').mockImplementation((p: any) => {
if (p.toString().endsWith('.hash')) return 'enabled-hash'
if (p.toString().endsWith('SKILL.md')) return '# Enabled Skill'
return ''
})
await expect(
readSkillTool!.execute('1', { skillId: disabledSkill.id }, undefined, undefined, undefined),
).rejects.toThrow('not enabled for this agent')
const result = await readSkillTool!.execute(
'1',
{ skillId: enabledSkill.id },
undefined,
undefined,
undefined,
)
// eslint-disable-next-line @typescript-eslint/no-explicit-any
expect((result.content[0] as any).text).toBe('# Enabled Skill')
} finally {
initializeSpy.mockRestore()
}
})
it('should reject all read_skill calls when no skills are enabled for the agent', async () => {
const team = await prisma.team.create({ data: { name: 'Skill None Team' } })
const skill = await prisma.skill.create({
data: { name: 'Unassigned Skill', assetId: 'asset-none', hash: 'none-hash', teamId: team.id },
})
await prisma.user.create({
data: {
id: 'skill-none-agent',
name: 'Ai Agent Skill None',
email: 'skill-none-agent@shumai.ai',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'skill-none-agent',
teamId: team.id,
type: 'chat',
config: { provider: 'google', model: 'gemini' },
},
})
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
try {
const { harness } = await createAgentSession({
teamId: team.id,
agentId: 'skill-none-agent',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [],
enabledSkillIds: [],
allowedDomains: [],
providers: mockProviders,
})
const readSkillTool = harness.getTools().find((t) => t.name === 'read_skill')
expect(readSkillTool).toBeDefined()
await expect(
readSkillTool!.execute('1', { skillId: skill.id }, undefined, undefined, undefined),
).rejects.toThrow('not enabled for this agent')
} finally {
initializeSpy.mockRestore()
}
})
it('enforces agent tool call count quota on tool execution', async () => {
const team = await prisma.team.create({ data: { name: 'Tool Quota Enforcement Team' } })
const user = await prisma.user.create({
data: { name: 'Member User', email: 'member-tool-quota@shumai.ai', password: 'pw' },
})
await prisma.teamMember.create({
data: { teamId: team.id, userId: user.id, role: 'editor' },
})
await prisma.user.create({
data: {
id: 'tool-quota-agent',
name: 'Tool Quota Agent',
email: 'tool-quota-agent@shumai.ai',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'tool-quota-agent',
teamId: team.id,
type: 'chat',
config: { provider: 'google', model: 'gemini' },
},
})
await quotaService.createRule(team.id, {
scopeMode: 'all_members',
resource: 'agent_tool_call_count',
resourceData: { name: 'read_thread' },
limit: 1,
period: '1hour',
enabled: true,
})
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
try {
const { harness } = await createAgentSession({
teamId: team.id,
userId: user.id,
agentId: 'tool-quota-agent',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [],
enabledSkillIds: [],
allowedDomains: [],
providers: mockProviders,
})
const readThreadTool = harness.getTools().find((t) => t.name === 'read_thread')
expect(readThreadTool).toBeDefined()
// First call executes and consumes quota
const res1 = await readThreadTool!.execute(
'call-1',
{ threadId: 'non-existent' },
undefined,
undefined,
undefined,
)
expect(res1).toBeDefined()
// Second call exceeds quota and returns error
const res2 = await readThreadTool!.execute(
'call-2',
{ threadId: 'non-existent' },
undefined,
undefined,
undefined,
)
expect(res2).toBeDefined()
const textContent = res2.content[0]
if (textContent && textContent.type === 'text') {
expect(textContent.text).toContain('Quota exceeded')
} else {
expect.unreachable('Expected text content')
}
} finally {
initializeSpy.mockRestore()
}
})
})
describe('getModelFromDb', () => {
it('should prioritize provider active api protocol over model config api', () => {
const providers: DbProviderInfo[] = [
{
name: 'custom-provider',
config: { api: 'anthropic-messages', apiKey: 'KEY' },
models: [
{
modelId: 'my-model',
name: 'My Model',
config: {
api: 'openai-responses', // stale model config
reasoning: false,
input: ['text'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 128000,
maxTokens: 4096,
},
},
],
},
]
const model = getModelFromDb(providers, 'custom-provider', 'my-model')
expect(model.api).toBe('anthropic-messages')
})
})
+294 -99
View File
@@ -5,26 +5,31 @@ import {
type AgentTool,
type ThinkingLevel,
} from '@earendil-works/pi-agent-core'
import { NodeExecutionEnv } from '@earendil-works/pi-agent-core/node'
import { getModel } from '@earendil-works/pi-ai/compat'
import type { Api, Model } from '@earendil-works/pi-ai'
import { createModels, InMemoryCredentialStore } from '@earendil-works/pi-ai'
import { builtinProviders } from '@earendil-works/pi-ai/providers/all'
import { agentService } from '@shumai/core/src/agent/agent'
import { resolveEffectiveRole } from '@shumai/core/src/authz/authz'
import { quotaService, QuotaExceededError } from '@shumai/core/src/quota/quota-service'
import { logger } from '@shumai/core/src/logger'
import { prisma } from '@shumai/db'
import { Type, type TSchema } from '@sinclair/typebox'
import { Type, type TSchema } from 'typebox'
import * as fs from 'fs'
import * as os from 'os'
import * as path from 'path'
import { DatabaseSessionStorage } from './database-session-storage'
import { createAnalyzeImageTool } from './tools/analyze-image'
import { metadataService } from '@shumai/core/src/metadata/metadata'
import { createCreateFileTool } from './tools/create-file'
import { createCreateFolderTool } from './tools/create-folder'
import { createCreateVersionTool } from './tools/create-version'
import { createDownloadAssetTool } from './tools/download-asset'
import { createListAssetsTool } from './tools/list-assets'
import { createReadPdfPagesTool } from './tools/read-pdf-pages'
import { createReadAssetTool } from './tools/read-asset'
import { createReadSkillTool } from './tools/read-skill'
import { createReadThreadTool } from './tools/read-thread'
import { createSandboxedBashTool } from './tools/sandboxed-bash'
import { createScreenshotTool } from './tools/screenshot'
import { serializeContextToXml } from './context/serialize-context'
import type { ShumaiMessageContext } from '@shumai/dtos'
export interface DbModelInfo {
modelId: string
@@ -42,47 +47,37 @@ export function getModelFromDb(providers: DbProviderInfo[], providerName: string
const dbProvider = providers.find((p) => p.name === providerName)
const dbModel = dbProvider?.models.find((m) => m.modelId === modelId)
if (!dbProvider || !dbModel) return undefined
if (!dbProvider || !dbModel) {
throw new Error(
`Provider "${providerName}" or model "${modelId}" not found in database configuration`,
)
}
// Model contains complex internal types from pi-ai
// eslint-disable-next-line @typescript-eslint/no-explicit-any
let m: any
try {
// Try to get built-in model as template
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const builtIn = (getModel as any)(providerName, modelId)
if (!builtIn) {
throw new Error(`Model ${modelId} is not a built-in model`)
}
m = { ...builtIn }
} catch {
// Not a built-in model
m = {
id: modelId,
name: dbModel.name || modelId,
provider: providerName,
api: 'openai-responses',
baseUrl: '',
reasoning: false,
input: ['text'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 4096,
maxTokens: 4096,
}
const m = {
id: modelId,
name: dbModel.name || modelId,
provider: providerName,
api: 'openai-responses',
baseUrl: '',
reasoning: false,
input: ['text'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 4096,
maxTokens: 4096,
}
// Override with database config
if (dbProvider.config.baseUrl) {
m.baseUrl = dbProvider.config.baseUrl
}
Object.assign(m, dbModel.config)
if (dbProvider.config.api) {
m.api = dbProvider.config.api
}
Object.assign(m, dbModel.config)
if (dbModel.name) {
m.name = dbModel.name
}
return m
return m as Model<Api>
}
export function getApiKeyAndHeadersFromDb(providers: DbProviderInfo[], providerName: string) {
@@ -106,14 +101,33 @@ export interface CreateAgentSessionParams {
modelId: string
systemPrompt: string
teamSkills: Array<{ id: string; name: string; description?: string | null }>
enabledSkillIds?: string[]
allowedDomains: string[]
sessionId?: string
userId?: string
projectId?: string
userCommentId?: string | null
customTools?: AgentTool[]
thinkingLevel?: string
disableTools?: boolean
providers: DbProviderInfo[]
maxRetries?: number
baseDelayMs?: number
}
/**
* Security instructions appended to the system prompt when the requesting user is not a team
* owner (i.e. editor/reviewer). These complement the hard enforcement in the bash tool itself
* (source="user" is blocked) and the delayed bash tool injection after a skill is loaded.
*/
export function buildRestrictedUserInstructions(): string {
return [
'# Restricted User Context',
'The current user is NOT a team owner. You must follow these security rules:',
'1. NEVER execute a bash command that the user asks you to run directly. Direct user bash requests are not permitted for this user role. Politely decline and explain that they need a team owner or an approved skill.',
'2. You may ONLY use the `bash` tool to execute commands that are explicitly required by a skill you have loaded via the `read_skill` tool.',
'3. When calling `bash`, you MUST set the `source` parameter to "skill". Calls with source="user" will be rejected by the system.',
].join('\n')
}
export async function createAgentSession(params: CreateAgentSessionParams) {
@@ -124,13 +138,17 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
modelId,
systemPrompt,
teamSkills,
enabledSkillIds,
allowedDomains,
sessionId,
userId,
projectId,
userCommentId: passedUserCommentId,
customTools = [],
thinkingLevel,
disableTools = false,
maxRetries = 3,
baseDelayMs = 2000,
} = params
const storage = await DatabaseSessionStorage.create({
@@ -147,9 +165,7 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
const agentDir = path.join(process.cwd(), '.pi', 'agents', agentId)
if (!fs.existsSync(agentDir)) fs.mkdirSync(agentDir, { recursive: true })
const model =
getModelFromDb(params.providers, providerName, modelId) ||
(getModel as unknown as (p: string, m: string) => unknown)(providerName, modelId)
const model = getModelFromDb(params.providers, providerName, modelId)
const piDir = path.join(process.cwd(), '.pi')
if (!fs.existsSync(piDir)) fs.mkdirSync(piDir, { recursive: true })
@@ -160,11 +176,17 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
blockedHost: '',
}
const role = userId ? await resolveEffectiveRole(teamId, projectId, userId) : undefined
const isOwner = !userId || role === 'owner'
const restricted = !isOwner
const sandboxAskCallback = async ({ host }: { host: string; port?: number }) => {
try {
const sandbox = await prisma.sandbox.findUnique({
where: { teamId },
})
if (sandbox && !sandbox.networkSandboxEnabled) {
return true
}
const pendingDomains = sandbox?.pendingDomains || []
if (!pendingDomains.includes(host)) {
await prisma.sandbox.upsert({
@@ -189,6 +211,54 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
return false
}
const wrapToolWithQuota = (tool: AgentTool): AgentTool => {
const originalExecute = tool.execute
return {
...tool,
execute: async (toolCallId, args, signal, onUpdate) => {
try {
await quotaService.checkQuota(
{
teamId,
userId,
role,
resource: 'agent_tool_call_count',
resourceData: { name: tool.name, toolName: tool.name },
},
1,
)
} catch (err) {
if (err instanceof QuotaExceededError) {
return {
content: [{ type: 'text', text: `Quota exceeded: ${err.message}` }],
details: { error: err.message },
}
}
throw err
}
const result = await originalExecute(toolCallId, args, signal, onUpdate)
try {
await quotaService.consumeQuota(
{
teamId,
userId,
role,
resource: 'agent_tool_call_count',
resourceData: { name: tool.name, toolName: tool.name },
},
1,
)
} catch (err) {
logger.error({ err }, 'Failed to record quota usage for agent tool call')
}
return result
},
}
}
// SandboxManager.initialize is a global operation that applies to the entire process.
// We reset it first to ensure any previous process-global SOCKS/HTTP proxy servers
// and callbacks are cleaned up, allowing the new callback and allowedDomains to take effect.
@@ -215,18 +285,21 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
}
// Restore env variables from previously loaded skills in this session
let skillLoadedInSession = false
if (sessionId) {
try {
const entries = await storage.getEntries()
for (const entry of entries) {
if (
entry.type === 'message' &&
entry.message &&
entry.message.role === 'toolResult' &&
entry.message.toolName === 'read_skill' &&
!entry.message.isError
) {
const details = entry.message.details as { skillId?: string } | undefined
if (details?.skillId) {
skillLoadedInSession = true
const envs = await agentService.getSkillEnvs(details.skillId)
onEnvsAdded(envs)
}
@@ -237,36 +310,9 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
}
}
const metadata = await storage.getMetadata()
const userCommentId =
passedUserCommentId !== undefined ? passedUserCommentId : metadata.userCommentId
const mediaTools: AgentTool[] = []
if (userId) {
mediaTools.push(
createAnalyzeImageTool(userId, userCommentId),
createScreenshotTool(userId, userCommentId),
createReadPdfPagesTool(userId, userCommentId),
)
}
const sandboxedBash = createSandboxedBashTool(process.cwd(), skillEnvs, {
getBlockedHost: () => sandboxState.blockedHost,
clearBlockedHost: () => {
sandboxState.blockedHost = ''
},
})
const readSkill = createReadSkillTool(userId, onEnvsAdded)
const systemTools: AgentTool[] = []
if (userId) {
systemTools.push(
createListAssetsTool(userId),
createCreateFolderTool(userId),
createCreateFileTool(userId),
createCreateVersionTool(userId),
createDownloadAssetTool(userId),
)
mediaTools.push(createReadAssetTool(userId))
}
const agent = await prisma.agent.findUnique({
@@ -274,24 +320,93 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
})
const agentConfig = agent?.config as PrismaJson.AgentConfig | null | undefined
const deniedTools = agentConfig?.deniedTools || []
const sandboxedBash = createSandboxedBashTool(process.cwd(), skillEnvs, {
teamId,
userId,
role,
getBlockedHost: () => sandboxState.blockedHost,
clearBlockedHost: () => {
sandboxState.blockedHost = ''
},
// Non-owner users can only run bash commands required by a loaded skill
restrictedUser: restricted,
})
// For non-owner users, the bash tool is not injected initially. It becomes available
// only after a skill is successfully loaded via the read_skill tool (or immediately when
// a skill was already loaded earlier in this session).
let harness: AgentHarness | undefined = undefined
const bashIncludedFromStart = isOwner || skillLoadedInSession
let bashInjected = bashIncludedFromStart
const onSkillLoaded = async () => {
if (!harness || bashInjected || deniedTools.includes('bash')) return
if (harness.getTools().some((t) => t.name === 'bash')) return
bashInjected = true
// Pass the full post-update tool list as activeToolNames: setTools only updates the
// registry otherwise, and the model would never see the bash tool on the next turn.
const next = [...harness.getTools(), wrapToolWithQuota(sandboxedBash)]
await harness.setTools(
next,
next.map((t) => t.name),
)
}
const readSkill = createReadSkillTool(
userId,
onEnvsAdded,
restricted ? onSkillLoaded : undefined,
enabledSkillIds,
projectId,
)
const systemTools: AgentTool[] = []
if (userId) {
const metadataSchema = await buildProjectMetadataSchema(projectId)
systemTools.push(
createListAssetsTool(userId),
createCreateFolderTool(userId),
createCreateFileTool(userId, metadataSchema),
createCreateVersionTool(userId, metadataSchema),
createDownloadAssetTool(userId),
)
}
const readThread = createReadThreadTool()
const allTools = [
const rawTools = [
...mediaTools,
readSkill,
readThread,
sandboxedBash,
...(bashIncludedFromStart ? [sandboxedBash] : []),
...systemTools,
...customTools,
]
const allTools = rawTools.map((tool) => wrapToolWithQuota(tool))
const enabledTools = disableTools
? []
: allTools.filter((tool) => !deniedTools.includes(tool.name))
const harness = new AgentHarness({
env: new NodeExecutionEnv({ cwd: process.cwd() }),
const credentials = new InMemoryCredentialStore()
const resolveKey = (key: string | undefined) => (key ? process.env[key] || key : undefined)
for (const p of params.providers) {
const apiKey = resolveKey(p.config?.apiKey)
if (apiKey) {
await credentials.modify(p.name, async () => ({ type: 'api_key' as const, key: apiKey }))
}
}
const modelsStore = createModels({ credentials })
for (const provider of builtinProviders()) {
modelsStore.setProvider(provider)
}
harness = new AgentHarness({
session,
models: modelsStore,
model,
thinkingLevel: (thinkingLevel || 'off') as ThinkingLevel,
streamOptions: {
maxRetries,
maxRetryDelayMs: baseDelayMs,
},
systemPrompt: async () => {
let prompt = systemPrompt
@@ -299,6 +414,10 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
return prompt
}
if (restricted) {
prompt += `\n\n${buildRestrictedUserInstructions()}`
}
// Sandbox environment restrictions
prompt +=
'\n\n' +
@@ -316,12 +435,32 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
}
return prompt
},
getApiKeyAndHeaders: async (targetModel) => {
return getApiKeyAndHeadersFromDb(params.providers, targetModel.provider)
},
tools: enabledTools,
})
harness.on('context', async ({ messages }) => {
return {
messages: messages.map((msg) => {
if (
msg.role === 'custom' &&
msg.customType === 'shumai_message' &&
msg.details &&
typeof msg.details === 'object'
) {
const context = msg.details as ShumaiMessageContext
const xml = serializeContextToXml(context)
const text = typeof msg.content === 'string' ? msg.content : ''
const fullContent = xml ? `${text}\n\n${xml}`.trim() : text
return {
...msg,
content: fullContent,
}
}
return msg
}),
}
})
return { session, harness }
}
@@ -332,58 +471,116 @@ export interface AutofillField {
}
export function fieldsToTypeBoxSchema(fields: AutofillField[]) {
const newOptionSchema = Type.Object(
{
newOption: Type.Object(
{
value: Type.String({
minLength: 1,
description:
'The new option display name/value to create and select if none of the existing options match.',
}),
},
{ additionalProperties: false },
),
},
{ additionalProperties: false },
)
const properties: Record<string, TSchema> = {}
for (const f of fields) {
let schema: TSchema
const fieldName = f.config.name
const fieldDesc = f.description || fieldName
const baseDescription = `The field '${fieldName}' represents ${fieldDesc}.`
switch (f.config.type) {
case 'text':
case 'longText':
schema = Type.String()
schema = Type.String({ description: baseDescription })
break
case 'number':
case 'rating':
schema = Type.Number()
schema = Type.Number({ description: baseDescription })
break
case 'toggle':
schema = Type.Boolean()
schema = Type.Boolean({ description: baseDescription })
break
case 'select': {
const options = f.config.select?.options || []
schema = Type.String({
enum: options.map((o) => o.id),
})
const description =
options.length > 0
? `${baseDescription}\nSelect one existing option ID or provide {"newOption": {"value": "..."}} to create a new option.\n\nAvailable options:\n${options.map((o) => `- ${o.displayName} => ${o.id}`).join('\n')}`
: `${baseDescription}\nProvide {"newOption": {"value": "..."}} with the desired option name.`
if (options.length > 0) {
const optionLines = options.map((o) => `- ${o.displayName} => ${o.id}`).join('\n')
schema.description = `The field '${fieldName}' represents ${fieldDesc}.\nSelect one option and return the option ID as the value.\n\nAvailable options:\n${optionLines}`
schema = Type.Union(
[
Type.String({
enum: options.map((o) => o.id),
}),
newOptionSchema,
],
{ description },
)
} else {
schema = Type.Object(newOptionSchema.properties, {
additionalProperties: false,
description,
})
}
break
}
case 'selectMulti': {
const options = f.config.selectMulti?.options || []
schema = Type.Array(
Type.String({
enum: options.map((o) => o.id),
}),
)
if (options.length > 0) {
const optionLines = options.map((o) => `- ${o.displayName} => ${o.id}`).join('\n')
schema.description = `The field '${fieldName}' represents ${fieldDesc}.\nSelect applicable options and return the option IDs as the value.\n\nAvailable options:\n${optionLines}`
}
const description =
options.length > 0
? `${baseDescription}\nSelect applicable option IDs or provide {"newOption": {"value": "..."}} objects for new options (e.g. ["opt1", {"newOption": {"value": "new_name"}}]).\n\nAvailable options:\n${options.map((o) => `- ${o.displayName} => ${o.id}`).join('\n')}`
: `${baseDescription}\nProvide an array of {"newOption": {"value": "..."}} objects for new options.`
const itemSchema =
options.length > 0
? Type.Union([
Type.String({
enum: options.map((o) => o.id),
}),
newOptionSchema,
])
: newOptionSchema
schema = Type.Array(itemSchema, { description })
break
}
default:
schema = Type.String()
schema = Type.String({ description: baseDescription })
}
if (!schema.description) {
schema.description = `The field '${fieldName}' represents ${fieldDesc}.`
}
properties[f.id] = schema
// Every field is required but nullable: the agent must return each field key,
// using `null` for values it does not know. Nulls are skipped when persisting.
// This matches the OpenAI structured-outputs strict-mode convention (all
// properties required, optionality expressed via `null`).
properties[f.id] = Type.Union([schema, Type.Null()])
}
return Type.Object(properties)
return Type.Object(properties, { additionalProperties: false })
}
/**
* Builds the strict, all-optional metadata schema for the project's CREATION_CONTEXT
* fields (the same field set the agent may attach when creating files/versions).
* Returns undefined when no project is known or the project defines no fields, in
* which case the `metadata` parameter is omitted from the tools entirely.
*/
async function buildProjectMetadataSchema(projectId?: string): Promise<TSchema | undefined> {
if (!projectId) return undefined
const fields = await metadataService.listProjectFields('', projectId)
const creationFields: AutofillField[] = []
for (const f of fields) {
const config = f.field.config
if (!config || config.autofillSource !== 'CREATION_CONTEXT') continue
creationFields.push({
id: f.field.key,
config,
description: f.field.description,
})
}
if (creationFields.length === 0) return undefined
return fieldsToTypeBoxSchema(creationFields)
}
function escapeXml(str: string): string {
@@ -432,7 +629,6 @@ import * as aiActivities from './activities/ai'
import { agentAutofillMedia } from './workflows/agent-autofill'
import { agentChat } from './workflows/agent-chat'
import { agentEmbeddingMedia } from './workflows/agent-embedding'
import { agentToolCall } from './workflows/agent-tool-call'
import { queryEmbeddingForSearch } from './workflows/query-embedding-for-search'
export function initAgentWorkflows() {
@@ -440,7 +636,6 @@ export function initAgentWorkflows() {
registerWorkflow(WorkflowTaskType.query_embedding_for_search, queryEmbeddingForSearch)
registerWorkflow(WorkflowTaskType.ai_metadata_autofill, agentAutofillMedia)
registerWorkflow(WorkflowTaskType.chat, agentChat)
registerWorkflow(WorkflowTaskType.agent_tool_call, agentToolCall)
registerActivities(agentActivities)
registerActivities(aiActivities)
@@ -459,8 +654,8 @@ export {
type GenerateTextEmbeddingParams,
} from './activities/ai'
export * from './database-session-storage'
export * from './context/serialize-context'
export * from './workflows/agent-autofill'
export * from './workflows/agent-chat'
export * from './workflows/agent-embedding'
export * from './workflows/agent-tool-call'
export * from './workflows/query-embedding-for-search'
@@ -0,0 +1,117 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { validateToolArguments } from '@earendil-works/pi-ai'
import { createCreateFileTool } from './tools/create-file'
import { fieldsToTypeBoxSchema } from './index'
import { s3Service } from '@shumai/core/src/s3/s3'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
uploadFileToKey: vi.fn(),
putObject: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
Permission: { Edit: 'edit', Read: 'read' },
ResourceType: { Asset: 'asset' },
}))
vi.mock('@shumai/core/src/asset/asset', () => ({
assetService: {
createFile: vi
.fn()
.mockResolvedValue({ id: 'file-1', name: 'test', type: 'file', sizeByte: 100 }),
},
}))
/**
* Reproduce: pi-ai <= 0.82.1 runs its lenient coercion on typebox v1 schemas and
* converts `null` in a `[string, null]` union into `""` (see
* coerceWithUnionSchema in @earendil-works/pi-ai/dist/utils/validation.js).
*
* When the agent calls create_file with `path: null, data: {...}`, the harness
* validates first: the `null` path becomes `""`, so the tool's
* assertExactlyOneSource believes BOTH path and data were provided and throws
* "Provide exactly one of ..." even though the call is valid.
*
* Fixed upstream in pi-ai 0.84.0 (#7328): values that already match a union arm
* are preserved before coercion. This test should pass once @earendil-works/pi-ai
* is upgraded to >= 0.84.0.
*/
describe('reproduce null coercion in harness tool validation', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('keeps path: null when data is provided (no false "exactly one" error)', async () => {
const tool = createCreateFileTool('user-1')
const validated = validateToolArguments(
{
name: 'create_file',
description: '',
parameters: tool.parameters,
} as Parameters<typeof validateToolArguments>[0],
{
type: 'toolCall',
id: 'call-1',
name: 'create_file',
arguments: {
parent: '01KYKNP52YERGJ6Q4GTPF5FRQF',
path: null,
data: { name: 'random_text.txt', content: 'hello' },
},
} as Parameters<typeof validateToolArguments>[1],
) as { path?: unknown; data?: unknown }
// The `null` path must survive validation untouched; coercion to "" makes
// the tool think both path and data were provided.
expect(validated.path).toBeNull()
expect(validated.data).toEqual({ name: 'random_text.txt', content: 'hello' })
// execute must accept the valid call (data-only) without throwing
const result = await tool.execute('call-1', validated as never)
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
id: 'file-1',
name: 'test',
type: 'file',
size: 100,
})
expect(s3Service.putObject).toHaveBeenCalledTimes(1)
expect(authzService.hasPermission).toHaveBeenCalledTimes(1)
expect(assetService.createFile).toHaveBeenCalledTimes(1)
})
it('keeps null metadata values (the "unknown" sentinel) through validation', () => {
const tool = createCreateFileTool(
'user-1',
fieldsToTypeBoxSchema([{ id: 'prompt', config: { name: 'Prompt', type: 'text' } }]),
)
const validated = validateToolArguments(
{
name: 'create_file',
description: '',
parameters: tool.parameters,
} as Parameters<typeof validateToolArguments>[0],
{
type: 'toolCall',
id: 'call-1',
name: 'create_file',
arguments: {
parent: 'folder-1',
path: null,
data: { name: 'a.md', content: 'x' },
metadata: { prompt: null },
},
} as Parameters<typeof validateToolArguments>[1],
) as { metadata?: Record<string, unknown> }
expect(validated.metadata).toEqual({ prompt: null })
})
})
+104 -18
View File
@@ -1,6 +1,22 @@
import { describe, it, expect } from 'vitest'
import { type TSchema } from 'typebox'
import { Value } from 'typebox/value'
import { fieldsToTypeBoxSchema, type AutofillField } from './index'
interface AnyOfFieldSchema {
type?: string
enum?: string[]
const?: string
description?: string
items?: { enum?: string[]; type?: string }
anyOf?: AnyOfFieldSchema[]
}
function propsOf(schema: TSchema): Record<string, { anyOf?: AnyOfFieldSchema[] }> {
return (schema as unknown as { properties: Record<string, { anyOf?: AnyOfFieldSchema[] }> })
.properties
}
describe('fieldsToTypeBoxSchema', () => {
it('converts text fields', () => {
const fields: AutofillField[] = [
@@ -8,10 +24,12 @@ describe('fieldsToTypeBoxSchema', () => {
{ id: 'f2', config: { name: 'Desc', type: 'longText' } },
]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('string')
expect(schema.properties.f1.description).toBe("The field 'Title' represents The title.")
expect(schema.properties.f2.type).toBe('string')
expect(schema.properties.f2.description).toBe("The field 'Desc' represents Desc.")
const props = propsOf(schema)
expect(props.f1.anyOf![0].type).toBe('string')
expect(props.f1.anyOf![1].type).toBe('null')
expect(props.f1.anyOf![0].description).toBe("The field 'Title' represents The title.")
expect(props.f2.anyOf![0].type).toBe('string')
expect(props.f2.anyOf![0].description).toBe("The field 'Desc' represents Desc.")
})
it('converts numeric fields', () => {
@@ -20,17 +38,18 @@ describe('fieldsToTypeBoxSchema', () => {
{ id: 'f2', config: { name: 'Rating', type: 'rating' } },
]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('number')
expect(schema.properties.f2.type).toBe('number')
const props = propsOf(schema)
expect(props.f1.anyOf![0].type).toBe('number')
expect(props.f2.anyOf![0].type).toBe('number')
})
it('converts toggle fields', () => {
const fields: AutofillField[] = [{ id: 'f1', config: { name: 'Active', type: 'toggle' } }]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('boolean')
expect(propsOf(schema).f1.anyOf![0].type).toBe('boolean')
})
it('converts select fields with enums and option descriptions', () => {
it('converts select fields with enums, newOption support and option descriptions', () => {
const fields: AutofillField[] = [
{
id: 'f1',
@@ -48,14 +67,41 @@ describe('fieldsToTypeBoxSchema', () => {
},
]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('string')
expect(schema.properties.f1.enum).toEqual(['opt1', 'opt2'])
expect(schema.properties.f1.description).toBe(
"The field 'Species' represents Species of animal.\nSelect one option and return the option ID as the value.\n\nAvailable options:\n- Option 1 => opt1\n- Option 2 => opt2",
const selectSchema = propsOf(schema).f1.anyOf![0]
expect(selectSchema.anyOf![0].type).toBe('string')
expect(selectSchema.anyOf![0].enum).toEqual(['opt1', 'opt2'])
expect(selectSchema.anyOf![1].type).toBe('object')
expect(selectSchema.description).toBe(
'The field \'Species\' represents Species of animal.\nSelect one existing option ID or provide {"newOption": {"value": "..."}} to create a new option.\n\nAvailable options:\n- Option 1 => opt1\n- Option 2 => opt2',
)
// Declared option IDs, newOption objects, and null are valid
expect(Value.Check(schema, { f1: 'opt1' })).toBe(true)
expect(Value.Check(schema, { f1: 'opt2' })).toBe(true)
expect(Value.Check(schema, { f1: { newOption: { value: 'kling' } } })).toBe(true)
expect(Value.Check(schema, { f1: null })).toBe(true)
// Arbitrary strings or invalid objects are rejected
expect(Value.Check(schema, { f1: 'not-an-option' })).toBe(false)
expect(Value.Check(schema, { f1: { newOption: { foo: 'bar' } } })).toBe(false)
})
it('converts selectMulti fields with array enums and option descriptions', () => {
it('converts select fields with empty options to newOption schema only', () => {
const fields: AutofillField[] = [
{
id: 'f1',
config: {
name: 'Provider',
type: 'select',
select: { options: [] },
},
},
]
const schema = fieldsToTypeBoxSchema(fields)
expect(Value.Check(schema, { f1: { newOption: { value: 'kling' } } })).toBe(true)
expect(Value.Check(schema, { f1: null })).toBe(true)
expect(Value.Check(schema, { f1: 'any-string' })).toBe(false)
})
it('converts selectMulti fields with array enums, newOption support and option descriptions', () => {
const fields: AutofillField[] = [
{
id: 'f1',
@@ -72,11 +118,35 @@ describe('fieldsToTypeBoxSchema', () => {
},
]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('array')
expect(schema.properties.f1.items.enum).toEqual(['opt1', 'opt2'])
expect(schema.properties.f1.description).toBe(
"The field 'Tags' represents Tags.\nSelect applicable options and return the option IDs as the value.\n\nAvailable options:\n- Tag 1 => opt1\n- Tag 2 => opt2",
const selectMultiSchema = propsOf(schema).f1.anyOf![0]
expect(selectMultiSchema.type).toBe('array')
expect(selectMultiSchema.description).toBe(
'The field \'Tags\' represents Tags.\nSelect applicable option IDs or provide {"newOption": {"value": "..."}} objects for new options (e.g. ["opt1", {"newOption": {"value": "new_name"}}]).\n\nAvailable options:\n- Tag 1 => opt1\n- Tag 2 => opt2',
)
expect(Value.Check(schema, { f1: ['opt1', 'opt2'] })).toBe(true)
expect(Value.Check(schema, { f1: [{ newOption: { value: 'Tag 3' } }] })).toBe(true)
expect(Value.Check(schema, { f1: ['opt1', { newOption: { value: 'Tag 3' } }] })).toBe(true)
expect(Value.Check(schema, { f1: null })).toBe(true)
expect(Value.Check(schema, { f1: ['opt1', 'bad'] })).toBe(false)
expect(Value.Check(schema, { f1: ['opt1', { newOption: { bad: 'value' } }] })).toBe(false)
})
it('converts selectMulti fields with empty options to array of newOption schema', () => {
const fields: AutofillField[] = [
{
id: 'f1',
config: {
name: 'Tags',
type: 'selectMulti',
selectMulti: { options: [] },
},
},
]
const schema = fieldsToTypeBoxSchema(fields)
expect(Value.Check(schema, { f1: [{ newOption: { value: 'Tag 1' } }] })).toBe(true)
expect(Value.Check(schema, { f1: [] })).toBe(true)
expect(Value.Check(schema, { f1: null })).toBe(true)
expect(Value.Check(schema, { f1: ['tag1'] })).toBe(false)
})
it('defaults to string for unknown types', () => {
@@ -85,6 +155,22 @@ describe('fieldsToTypeBoxSchema', () => {
{ id: 'f1', config: { name: 'Unknown', type: 'invalid' } },
]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('string')
expect(propsOf(schema).f1.anyOf![0].type).toBe('string')
})
it('marks every property required (nullable) and rejects unknown keys', () => {
const fields: AutofillField[] = [
{ id: 'f1', config: { name: 'Title', type: 'text' } },
{ id: 'f2', config: { name: 'Rating', type: 'rating' } },
]
const schema = fieldsToTypeBoxSchema(fields)
const props = propsOf(schema)
// typebox v1 objects are strict by default: unknown keys are rejected
expect(props.f1.anyOf![0].type).toBe('string')
expect(props.f1.anyOf![1].type).toBe('null')
expect(Value.Check(schema, { f1: 'x', f2: null })).toBe(true)
expect(Value.Check(schema, { f1: null, f2: 3 })).toBe(true)
expect(Value.Check(schema, { f1: 'x' })).toBe(false)
expect(Value.Check(schema, { f1: 'x', f2: null, extra: 1 })).toBe(false)
})
})
@@ -1,175 +0,0 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createAnalyzeImageTool } from './analyze-image'
import {
prisma,
WorkflowTaskStatus,
type Asset,
type AssetComment,
type User,
type WorkflowTask,
} from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/db', async (importOriginal) => {
const actual = await importOriginal<typeof import('@shumai/db')>()
return {
...actual,
prisma: {
user: {
findUnique: vi.fn(),
},
asset: {
findUnique: vi.fn(),
},
assetComment: {
findUnique: vi.fn(),
},
workflowTask: {
create: vi.fn(),
},
},
}
})
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
getObject: vi.fn(),
},
}))
vi.mock('@shumai/workflow-core', () => ({
workflowService: {
executeWait: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn(),
},
Permission: { Read: 'Read' },
ResourceType: { Asset: 'asset' },
}))
describe('analyzeImageTool', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('should check user permissions and fetch the image from S3 when comment has no annotations', async () => {
vi.mocked(prisma.user.findUnique).mockResolvedValue({ id: 'user-1' } as User)
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
storageKey: { key: 'raw-image.png' },
media: null,
} as unknown as Asset)
vi.mocked(prisma.assetComment.findUnique).mockResolvedValue({
id: 'comment-1',
assetId: 'asset-1',
annotation: null,
} as unknown as AssetComment)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-image-bytes'),
contentType: 'image/png',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createAnalyzeImageTool('user-1', 'comment-1')
const result = await tool.execute('call-1', { assetId: 'asset-1' })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-1',
})
expect(result.content[0].type).toBe('image')
expect((result.content[0] as unknown as { data: string }).data).toBe(
Buffer.from('fake-image-bytes').toString('base64'),
)
expect(result.details.sourceKeys).toEqual(['raw-image.png'])
expect(prisma.workflowTask.create).not.toHaveBeenCalled()
})
it('should trigger overlay transcode workflow when comment has annotations', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
storageKey: { key: 'raw-image.png' },
media: {
imageTranscodes: [{ key: 'transcoded-image.webp' }],
},
} as unknown as Asset)
vi.mocked(prisma.assetComment.findUnique).mockResolvedValue({
id: 'comment-1',
assetId: 'asset-1',
annotation: [
{
type: 'box',
color: '#ff0000',
points: [
[0, 0],
[1, 1],
],
},
],
} as unknown as AssetComment)
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-1',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-1',
status: WorkflowTaskStatus.completed,
output: { key: 'annotated-image.webp' },
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-annotated-image-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createAnalyzeImageTool('user-1', 'comment-1')
const result = await tool.execute('call-1', { assetId: 'asset-1' })
expect(prisma.workflowTask.create).toHaveBeenCalledWith({
data: {
assetId: 'asset-1',
projectId: 'project-1',
type: 'transcode_image_annotation',
status: 'pending',
payload: {
projectId: 'project-1',
imageAnnotation: {
annotations: [
{
type: 'box',
color: '#ff0000',
points: [
[0, 0],
[1, 1],
],
},
],
},
},
},
})
expect(result.content[0].type).toBe('image')
expect((result.content[0] as unknown as { data: string }).data).toBe(
Buffer.from('fake-annotated-image-bytes').toString('base64'),
)
expect(result.details.sourceKeys).toEqual(['annotated-image.webp'])
})
})
-128
View File
@@ -1,128 +0,0 @@
import { Type } from '@sinclair/typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { type ImageContent } from '@earendil-works/pi-ai'
import { prisma, WorkflowTaskType, WorkflowTaskStatus, type User } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
const analyzeImageSchema = Type.Object({
assetId: Type.String({
description: 'The asset ID of the image to analyze. This parameter is required.',
}),
})
export function createAnalyzeImageTool(
userId: string,
userCommentId?: string | null,
): AgentTool<typeof analyzeImageSchema> {
return {
name: 'analyze_image',
label: 'Analyze Image',
description:
'Retrieves and views the image content. Call this tool if you need to analyze the image.',
parameters: analyzeImageSchema,
execute: async (_toolCallId, params) => {
const assetId = params.assetId
if (!userId) {
throw new Error('User ID is required for authorization.')
}
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: assetId,
})
const asset = await prisma.asset.findUnique({
where: { id: assetId },
include: { storageKey: true },
})
if (!asset) {
throw new Error(`Asset with ID ${assetId} not found.`)
}
// Get default media key
let mediaKey = asset.storageKey?.key
if (asset.media) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const mediaInfo = asset.media as any
if (mediaInfo.imageTranscodes && mediaInfo.imageTranscodes.length > 0) {
mediaKey = mediaInfo.imageTranscodes[0].key || mediaKey
}
}
if (!mediaKey) {
throw new Error('No media content found for this asset.')
}
// Check if trigger comment has draw annotations for this asset
let annotations: unknown = null
if (userCommentId) {
const comment = await prisma.assetComment.findUnique({
where: { id: userCommentId },
})
if (comment && comment.assetId === assetId && comment.annotation) {
const list = comment.annotation
if (Array.isArray(list) && list.length > 0) {
annotations = list
}
}
}
let keyToUse = mediaKey
if (annotations) {
// Trigger transcode workflow to draw annotation
const task = await prisma.workflowTask.create({
data: {
assetId,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_image_annotation,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
imageAnnotation: {
annotations: annotations as PrismaJson.AnnotationList,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const output = completedTask.output as any
if (output?.key) {
keyToUse = output.key
} else {
throw new Error('Image annotation transcode workflow failed to output S3 key.')
}
}
// Retrieve the image content from S3
const bucket = process.env.S3_BUCKET || 'shumai'
const { buffer, contentType } = await s3Service.getObject(bucket, keyToUse)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
const content: ImageContent[] = [
{
type: 'image',
data: buffer.toString('base64'),
mimeType,
},
]
return {
content,
details: {
sourceKeys: [keyToUse],
},
}
},
}
}
@@ -0,0 +1,270 @@
import { describe, it, expect, beforeEach } from 'vitest'
import { prisma } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import { resolveAnnotationsById } from './annotation-resolver'
import { DatabaseSessionStorage } from '../database-session-storage'
import type { SessionTreeEntry } from '@earendil-works/pi-agent-core'
describe('resolveAnnotationsById', () => {
setupTestDbHooks()
beforeEach(() => {
// DB hooks automatically manage transactions
})
it('returns null annotations and null timestamp when annotationId is undefined or empty', async () => {
const result1 = await resolveAnnotationsById('asset-1', undefined)
expect(result1).toEqual({ annotations: null, timestamp: null })
const result2 = await resolveAnnotationsById('asset-1', '')
expect(result2).toEqual({ annotations: null, timestamp: null })
})
it('resolves annotations and timestamp from AssetComment (Comments Mode)', async () => {
const user = await prisma.user.create({
data: { name: 'Comment Author', email: 'author@test.com', password: 'pw' },
})
const team = await prisma.team.create({
data: { name: 'Test Team' },
})
const project = await prisma.project.create({
data: { name: 'Test Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'video.mp4',
type: 'file',
projectId: project.id,
status: 'uploaded',
},
})
const comment = await prisma.assetComment.create({
data: {
assetId: asset.id,
creatorId: user.id,
message: 'Look at the highlighted area',
second: 18.5,
annotation: [
{
type: 'line',
color: '#ff0000',
points: [
[10, 10],
[50, 50],
],
},
],
},
})
const result = await resolveAnnotationsById(asset.id, comment.id)
expect(result.timestamp).toBe(18.5)
expect(result.annotations).toEqual([
{
type: 'line',
color: '#ff0000',
points: [
[10, 10],
[50, 50],
],
},
])
})
it('throws an error when AssetComment belongs to a different asset', async () => {
const user = await prisma.user.create({
data: { name: 'Author', email: 'author2@test.com', password: 'pw' },
})
const team = await prisma.team.create({
data: { name: 'Team' },
})
const project = await prisma.project.create({
data: { name: 'Project', teamId: team.id },
})
const assetA = await prisma.asset.create({
data: { name: 'videoA.mp4', type: 'file', projectId: project.id, status: 'uploaded' },
})
const assetB = await prisma.asset.create({
data: { name: 'videoB.mp4', type: 'file', projectId: project.id, status: 'uploaded' },
})
const commentB = await prisma.assetComment.create({
data: {
assetId: assetB.id,
creatorId: user.id,
message: 'Comment on B',
annotation: [
{
type: 'box',
color: '#ff0000',
points: [
[0, 0],
[10, 10],
],
},
],
},
})
await expect(resolveAnnotationsById(assetA.id, commentB.id)).rejects.toThrow(
`Annotation "${commentB.id}" belongs to asset "${assetB.id}", not target asset "${assetA.id}".`,
)
})
it('resolves annotations and timestamp from AgentSessionEntry (1-on-1 Chat Mode)', async () => {
const user = await prisma.user.create({
data: { name: 'Chat User', email: 'chatuser@test.com', password: 'pw' },
})
const team = await prisma.team.create({
data: { name: 'Chat Team' },
})
const project = await prisma.project.create({
data: { name: 'Chat Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'chat-video.mp4',
type: 'file',
projectId: project.id,
status: 'uploaded',
},
})
const agentUser = await prisma.user.create({
data: { name: 'Chat Bot User', email: 'bot@example.com', type: 'agent' },
})
const agent = await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'chat',
config: { provider: 'test-provider', model: 'test-model' },
},
})
// Create session storage
const storage = await DatabaseSessionStorage.create({
agentId: agent.id,
userId: user.id,
assetId: asset.id,
})
// Set active message context with markup annotations and position
storage.currentMessageContext = {
currentAsset: {
id: asset.id,
name: asset.name,
type: 'file',
path: asset.name,
},
position: { type: 'time', seconds: 12.34 },
annotation: true,
annotations: [
{
type: 'rectangle',
x: 100,
y: 200,
width: 50,
height: 50,
},
],
}
// User sends a message -> DatabaseSessionStorage.appendEntry is called
const userEntry: SessionTreeEntry = {
type: 'message',
id: '01JTESTENTRY1234567890',
parentId: null,
timestamp: new Date().toISOString(),
message: {
role: 'user',
content: 'Check this rectangle markup on video',
timestamp: Date.now(),
},
}
await storage.appendEntry(userEntry)
// Verify DatabaseSessionStorage persisted entry with details.id = entry.id
const dbEntry = await prisma.agentSessionEntry.findUnique({
where: { id: '01JTESTENTRY1234567890' },
})
expect(dbEntry).toBeDefined()
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const details = (dbEntry?.data as any)?.details
expect(details.id).toBe('01JTESTENTRY1234567890')
expect(details.position).toEqual({ type: 'time', seconds: 12.34 })
expect(details.annotations).toHaveLength(1)
// Now resolveAnnotationsById directly with the entry id and matching asset.id
const result = await resolveAnnotationsById(asset.id, '01JTESTENTRY1234567890')
expect(result.timestamp).toBe(12.34)
expect(result.annotations).toEqual([
{
type: 'rectangle',
x: 100,
y: 200,
width: 50,
height: 50,
},
])
})
it('throws an error when AgentSessionEntry belongs to a different asset', async () => {
const user = await prisma.user.create({
data: { name: 'User', email: 'user3@test.com', password: 'pw' },
})
const team = await prisma.team.create({
data: { name: 'Team' },
})
const project = await prisma.project.create({
data: { name: 'Project', teamId: team.id },
})
const assetA = await prisma.asset.create({
data: { name: 'chatA.mp4', type: 'file', projectId: project.id, status: 'uploaded' },
})
const assetB = await prisma.asset.create({
data: { name: 'chatB.mp4', type: 'file', projectId: project.id, status: 'uploaded' },
})
const agentUser = await prisma.user.create({
data: { name: 'Bot User', email: 'bot2@example.com', type: 'agent' },
})
const agent = await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'chat',
config: { provider: 'test-provider', model: 'test-model' },
},
})
const storage = await DatabaseSessionStorage.create({
agentId: agent.id,
userId: user.id,
assetId: assetB.id,
})
storage.currentMessageContext = {
currentAsset: { id: assetB.id, name: assetB.name, type: 'file', path: assetB.name },
annotation: true,
annotations: [{ type: 'box' }],
}
const userEntry: SessionTreeEntry = {
type: 'message',
id: '01JENTRYONASSETB123456',
parentId: null,
timestamp: new Date().toISOString(),
message: { role: 'user', content: 'Markup on B', timestamp: Date.now() },
}
await storage.appendEntry(userEntry)
await expect(resolveAnnotationsById(assetA.id, '01JENTRYONASSETB123456')).rejects.toThrow(
`Annotation "01JENTRYONASSETB123456" belongs to asset "${assetB.id}", not target asset "${assetA.id}".`,
)
})
it('throws an error if annotationId does not match any record', async () => {
await expect(resolveAnnotationsById('asset-1', 'non_existent_id')).rejects.toThrow(
'Annotation with ID "non_existent_id" not found.',
)
})
})
@@ -0,0 +1,52 @@
import { prisma } from '@shumai/db'
import type { ShumaiMessageContext } from '@shumai/dtos'
export async function resolveAnnotationsById(
assetId: string,
annotationId?: string | null,
): Promise<{ annotations: PrismaJson.AnnotationList | null; timestamp: number | null }> {
if (!annotationId) {
return { annotations: null, timestamp: null }
}
// 1. Check AssetComment table (Comments Mode)
const comment = await prisma.assetComment.findUnique({
where: { id: annotationId },
select: { assetId: true, annotation: true, second: true },
})
if (comment) {
if (comment.assetId !== assetId) {
throw new Error(
`Annotation "${annotationId}" belongs to asset "${comment.assetId}", not target asset "${assetId}".`,
)
}
const list = comment.annotation
const annotations =
Array.isArray(list) && list.length > 0 ? (list as PrismaJson.AnnotationList) : null
return { annotations, timestamp: comment.second }
}
// 2. Check AgentSessionEntry table (1-on-1 Chat Mode)
const entry = await prisma.agentSessionEntry.findUnique({
where: { id: annotationId },
select: { assetId: true, data: true },
})
if (entry) {
const details = (entry.data as Record<string, unknown> | null)?.details as
| ShumaiMessageContext
| undefined
const entryAssetId = entry.assetId || details?.currentAsset?.id
if (entryAssetId && entryAssetId !== assetId) {
throw new Error(
`Annotation "${annotationId}" belongs to asset "${entryAssetId}", not target asset "${assetId}".`,
)
}
const list = details?.annotations
const annotations =
Array.isArray(list) && list.length > 0 ? (list as unknown as PrismaJson.AnnotationList) : null
const timestamp = details?.position?.type === 'time' ? details.position.seconds : null
return { annotations, timestamp }
}
throw new Error(`Annotation with ID "${annotationId}" not found.`)
}
@@ -0,0 +1,341 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
import { createCreateFileTool } from './create-file'
import { fieldsToTypeBoxSchema } from '../index'
import { s3Service } from '@shumai/core/src/s3/s3'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService } from '@shumai/core/src/authz/authz'
import { Value } from 'typebox/value'
import * as fs from 'fs'
import * as os from 'os'
import * as path from 'path'
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
uploadFileToKey: vi.fn(),
putObject: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
Permission: { Edit: 'edit', Read: 'read' },
ResourceType: { Asset: 'asset' },
}))
vi.mock('@shumai/core/src/asset/asset', () => ({
assetService: {
createFile: vi
.fn()
.mockResolvedValue({ id: 'file-1', name: 'test', type: 'file', sizeByte: 100 }),
},
}))
describe('createCreateFileTool', () => {
const createdFiles: string[] = []
beforeEach(() => {
vi.clearAllMocks()
})
afterEach(() => {
for (const f of createdFiles) {
if (fs.existsSync(f)) {
fs.unlinkSync(f)
}
}
createdFiles.length = 0
})
const createTempFile = (content: string | Buffer, filename: string): string => {
const filePath = path.join(os.tmpdir(), filename)
fs.writeFileSync(filePath, content)
createdFiles.push(filePath)
return filePath
}
it('should upload a local file when "path" is provided', async () => {
const filePath = createTempFile('# Hello from disk', 'create-file-test.md')
const tool = createCreateFileTool('user-1')
const result = await tool.execute('call-1', { parent: 'folder-1', path: filePath, data: null })
expect(s3Service.uploadFileToKey).toHaveBeenCalledTimes(1)
expect(s3Service.putObject).not.toHaveBeenCalled()
const uploadArgs = vi.mocked(s3Service.uploadFileToKey).mock.calls[0]
const s3Key = uploadArgs[1]
expect(s3Key).toMatch(/^files\/.+\.md$/)
expect(uploadArgs[2]).toBe('text/markdown')
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'edit',
type: 'asset',
id: 'folder-1',
})
expect(assetService.createFile).toHaveBeenCalledWith({
parentId: 'folder-1',
name: 'create-file-test.md',
key: s3Key,
sizeByte: Buffer.byteLength('# Hello from disk', 'utf-8'),
contentType: 'text/markdown',
creatorId: 'user-1',
metadata: undefined,
})
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
id: 'file-1',
name: 'test',
type: 'file',
size: 100,
})
})
it('should forward the metadata to assetService.createFile when provided', async () => {
const filePath = createTempFile('# Hello from disk', 'create-file-metadata.md')
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
])
const tool = createCreateFileTool('user-1', metadataSchema)
await tool.execute('call-1', {
parent: 'folder-1',
path: filePath,
data: null,
metadata: { prompt: 'Generated using gemini' },
})
expect(assetService.createFile).toHaveBeenCalledWith(
expect.objectContaining({
metadata: { prompt: 'Generated using gemini' },
}),
)
})
it('should forward select and selectMulti newOption metadata', async () => {
const filePath = createTempFile('# Hello from disk', 'create-file-new-option.md')
const metadataSchema = fieldsToTypeBoxSchema([
{
id: 'provider',
config: {
name: 'Provider',
type: 'select',
select: { options: [{ id: 'openai', displayName: 'OpenAI', color: '#f43f5e' }] },
},
},
{
id: 'tags',
config: {
name: 'Tags',
type: 'selectMulti',
selectMulti: { options: [{ id: 'tag1', displayName: 'Tag 1', color: '#3b82f6' }] },
},
},
])
const tool = createCreateFileTool('user-1', metadataSchema)
await tool.execute('call-1', {
parent: 'folder-1',
path: filePath,
data: null,
metadata: {
provider: { newOption: { value: 'Kling' } },
tags: ['tag1', { newOption: { value: 'Tag 2' } }],
},
})
expect(assetService.createFile).toHaveBeenCalledWith(
expect.objectContaining({
metadata: {
provider: { newOption: { value: 'Kling' } },
tags: ['tag1', { newOption: { value: 'Tag 2' } }],
},
}),
)
})
it('should not include metadata when metadata is null', async () => {
const filePath = createTempFile('# Hello from disk', 'create-file-no-metadata.md')
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
])
const tool = createCreateFileTool('user-1', metadataSchema)
await tool.execute('call-1', {
parent: 'folder-1',
path: filePath,
data: null,
metadata: null,
})
const callArgs = vi.mocked(assetService.createFile).mock.calls[0][0]
expect(callArgs.metadata).toBeUndefined()
})
it('should throw when the local file does not exist', async () => {
const tool = createCreateFileTool('user-1')
await expect(
tool.execute('call-1', { parent: 'folder-1', path: '/nonexistent/file.md', data: null }),
).rejects.toThrow('Local file not found at path: /nonexistent/file.md')
expect(s3Service.uploadFileToKey).not.toHaveBeenCalled()
expect(assetService.createFile).not.toHaveBeenCalled()
})
it('should create a file directly from name and content', async () => {
const tool = createCreateFileTool('user-1')
const result = await tool.execute('call-1', {
parent: 'folder-1',
path: null,
data: { name: 'notes.md', content: '# Notes\n\nHello' },
})
expect(s3Service.putObject).toHaveBeenCalledTimes(1)
expect(s3Service.uploadFileToKey).not.toHaveBeenCalled()
const [bucket, s3Key, body, size, contentType] = vi.mocked(s3Service.putObject).mock
.calls[0] as unknown as [string, string, Buffer, number, string]
expect(bucket).toBe(process.env.S3_BUCKET || 'shumai')
expect(s3Key).toMatch(/^files\/.+notes\.md$/)
expect(body.toString('utf-8')).toBe('# Notes\n\nHello')
expect(size).toBe(Buffer.byteLength('# Notes\n\nHello', 'utf-8'))
expect(contentType).toBe('text/markdown')
expect(assetService.createFile).toHaveBeenCalledWith({
parentId: 'folder-1',
name: 'notes.md',
key: s3Key,
sizeByte: Buffer.byteLength('# Notes\n\nHello', 'utf-8'),
contentType: 'text/markdown',
creatorId: 'user-1',
metadata: undefined,
})
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
id: 'file-1',
name: 'test',
type: 'file',
size: 100,
})
})
it('should default unknown extensions to text/plain when creating from content', async () => {
const tool = createCreateFileTool('user-1')
await tool.execute('call-1', {
parent: 'folder-1',
path: null,
data: { name: 'script.ts', content: 'const x = 1' },
})
const [, , , , contentType] = vi.mocked(s3Service.putObject).mock.calls[0] as unknown as [
string,
string,
Buffer,
number,
string,
]
expect(contentType).toBe('text/plain')
})
it('should sanitize the name when creating from content', async () => {
const tool = createCreateFileTool('user-1')
await tool.execute('call-1', {
parent: 'folder-1',
path: null,
data: { name: 'my/notes.md', content: 'hello' },
})
const [, s3Key] = vi.mocked(s3Service.putObject).mock.calls[0] as unknown as [
string,
string,
Buffer,
number,
string,
]
expect(s3Key).toContain('my_notes.md')
expect(assetService.createFile).toHaveBeenCalledWith(
expect.objectContaining({ name: 'my_notes.md' }),
)
})
it('should throw when neither "path" nor "data" is provided', async () => {
const tool = createCreateFileTool('user-1')
await expect(
tool.execute('call-1', { parent: 'folder-1', path: null, data: null }),
).rejects.toThrow('Provide exactly one of "path" (a local file) or "data" (name and content).')
})
it('should throw when both "path" and "data" are provided', async () => {
const tool = createCreateFileTool('user-1')
await expect(
tool.execute('call-1', {
parent: 'folder-1',
path: '/some/file.md',
data: { name: 'notes.md', content: 'hello' },
}),
).rejects.toThrow('Provide exactly one of "path" (a local file) or "data" (name and content).')
expect(s3Service.uploadFileToKey).not.toHaveBeenCalled()
expect(s3Service.putObject).not.toHaveBeenCalled()
expect(assetService.createFile).not.toHaveBeenCalled()
})
it('should expose a strict, all-required (nullable) metadata schema when fields are provided', () => {
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
{
id: 'source',
config: {
name: 'Source',
type: 'select',
select: { options: [{ id: 'gemini', displayName: 'Gemini', color: '#ffffff' }] },
},
},
{ id: 'rating', config: { name: 'Rating', type: 'rating' } },
])
const tool = createCreateFileTool('user-1', metadataSchema)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
const meta = (tool.parameters as any).properties.metadata
// `metadata` is a nullable union: [strict object, null]
const metaObject = meta.anyOf[0]
expect(meta).toBeDefined()
expect(metaObject.required).toEqual(['prompt', 'source', 'rating'])
expect(metaObject.additionalProperties).toBe(false)
expect(metaObject.properties.prompt.anyOf[0].type).toBe('string')
expect(metaObject.properties.prompt.anyOf[1].type).toBe('null')
expect(metaObject.properties.source.anyOf[0].anyOf[0].enum).toEqual(['gemini'])
expect(metaObject.properties.rating.anyOf[0].type).toBe('number')
// Every field must be present (use null when unknown); unknown keys and bad enums are rejected
expect(Value.Check(meta, { prompt: 'hello', source: 'gemini', rating: 5 })).toBe(true)
expect(
Value.Check(meta, {
prompt: 'hello',
source: { newOption: { value: 'kling' } },
rating: 5,
}),
).toBe(true)
expect(Value.Check(meta, { prompt: null, source: null, rating: null })).toBe(true)
expect(Value.Check(meta, { prompt: 'hello', source: null, rating: null })).toBe(true)
expect(Value.Check(meta, { prompt: 'hello' })).toBe(false)
expect(
Value.Check(meta, { prompt: 'hello', source: 'gemini', rating: 5, unknownKey: 'x' }),
).toBe(false)
expect(Value.Check(meta, { prompt: 'hello', source: 'not-an-option', rating: null })).toBe(
false,
)
})
it('should omit the metadata parameter when no schema is provided', () => {
const tool = createCreateFileTool('user-1')
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
expect((tool.parameters as any).properties.metadata).toBeUndefined()
})
})
+120 -70
View File
@@ -1,91 +1,141 @@
import { Type } from '@sinclair/typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { executeAgentToolWorkflow } from './utils'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { s3Service } from '@shumai/core/src/s3/s3'
import { getFileMimeType, readFileMimeType } from '@shumai/core/src/utils/file-mime'
import { sanitizeFilename } from '@shumai/core/src/utils/filename'
import { type User } from '@shumai/db'
import { Type, type TSchema } from 'typebox'
import * as fs from 'fs'
import * as path from 'path'
import { s3Service } from '@shumai/core/src/s3/s3'
import { detectSupportedMimeType } from '@shumai/core/src/utils/mime'
import { ulid } from 'ulid'
import { sanitizeFilename } from '@shumai/core/src/utils/filename'
function getMimeType(filePath: string): string {
try {
const fd = fs.openSync(filePath, 'r')
try {
const buffer = Buffer.alloc(4100)
const bytesRead = fs.readSync(fd, buffer, 0, 4100, 0)
const detected = detectSupportedMimeType(new Uint8Array(buffer.subarray(0, bytesRead)))
if (detected) return detected
} finally {
fs.closeSync(fd)
}
} catch {
/* Ignore detection errors and fallback to extension mapping */
}
const ext = path.extname(filePath).toLowerCase()
const mimeTypes: Record<string, string> = {
'.png': 'image/png',
'.jpg': 'image/jpeg',
'.jpeg': 'image/jpeg',
'.gif': 'image/gif',
'.webp': 'image/webp',
'.svg': 'image/svg+xml',
'.mp4': 'video/mp4',
'.mov': 'video/quicktime',
'.pdf': 'application/pdf',
'.txt': 'text/plain',
'.md': 'text/markdown',
'.markdown': 'text/markdown',
}
return mimeTypes[ext] || 'application/octet-stream'
interface CreateFileToolParams {
parent: string
path: string | null
data: { name: string; content: string } | null
metadata?: Record<string, unknown> | null
}
const createFileSchema = Type.Object({
parent: Type.String({
description: 'The parent folder ID under which to create the file. This parameter is required.',
}),
path: Type.String({
description:
'The absolute or relative local path to the file on disk. This parameter is required.',
}),
})
function assertExactlyOneSource(params: CreateFileToolParams) {
const hasPath = params.path !== null
const hasData = params.data !== null
if (hasPath === hasData) {
throw new Error('Provide exactly one of "path" (a local file) or "data" (name and content).')
}
return hasPath ? params.path! : params.data!
}
export function createCreateFileTool(userId: string): AgentTool<typeof createFileSchema> {
return {
export function createCreateFileTool(userId: string, metadataSchema?: TSchema): AgentTool {
const createFileSchema = Type.Object(
{
parent: Type.String({
description:
'The parent folder ID under which to create the file. This parameter is required.',
}),
path: Type.Union([
Type.String({
description:
'The absolute or relative local path to an existing file on disk. Provide a value here OR in "data", but not both. Use null if you are providing "data".',
}),
Type.Null(),
]),
data: Type.Union([
Type.Object(
{
name: Type.String({
minLength: 1,
description:
'The desired name for the new file, including its extension (e.g. "notes.md").',
}),
content: Type.String({
description: 'The full content of the file as a plain string.',
}),
},
{ additionalProperties: false },
),
Type.Null(),
]),
...(metadataSchema ? { metadata: Type.Union([metadataSchema, Type.Null()]) } : {}),
},
{ additionalProperties: false },
)
const tool: AgentTool = {
name: 'create_file',
label: 'Create File',
description: 'Create a new file in a specified parent folder from a local file path.',
description:
'Create a new file under a folder. You MUST choose exactly ONE creation method: ' +
'(1) provide "path" to upload an existing local file, OR ' +
'(2) provide "data" with name and content to create a new text file. ' +
'Set the unused parameter to null. ' +
'The optional "metadata" parameter accepts only the CREATION_CONTEXT fields declared in its schema; never invent field keys, and use null for fields you do not know.',
parameters: createFileSchema,
execute: async (_toolCallId, params) => {
const absolutePath = path.resolve(process.cwd(), params.path)
if (!fs.existsSync(absolutePath)) {
throw new Error(`Local file not found at path: ${params.path}`)
// The harness validates tool args against `createFileSchema` before calling
// execute, so `params` is guaranteed to match this shape.
const p = params as CreateFileToolParams
const source = assertExactlyOneSource(p)
let name: string
let size: number
let mimeType: string
let s3Key: string
if (typeof source === 'string') {
// Branch A: upload an existing local file
const absolutePath = path.resolve(process.cwd(), source)
if (!fs.existsSync(absolutePath)) {
throw new Error(`Local file not found at path: ${source}`)
}
name = path.basename(absolutePath)
size = fs.statSync(absolutePath).size
mimeType = readFileMimeType(absolutePath)
// Generate compliant S3 key matching normal file upload format
s3Key = `files/${ulid()}/${sanitizeFilename(name)}`
await s3Service.uploadFileToKey(absolutePath, s3Key, mimeType)
} else {
// Branch B: create the file directly from provided name and content
name = sanitizeFilename(source.name)
const content = Buffer.from(source.content, 'utf-8')
size = content.byteLength
mimeType = getFileMimeType(null, source.name, 'text/plain')
s3Key = `files/${ulid()}/${name}`
await s3Service.putObject(process.env.S3_BUCKET || 'shumai', s3Key, content, size, mimeType)
}
const fileSize = fs.statSync(absolutePath).size
const mimeType = getMimeType(absolutePath)
// Generate compliant S3 key matching normal file upload format
const s3Key = `files/${ulid()}/${sanitizeFilename(path.basename(absolutePath))}`
await s3Service.uploadFileToKey(absolutePath, s3Key, mimeType)
const result = await executeAgentToolWorkflow({
toolName: 'create_file',
args: {
parent: params.parent,
s3Key,
name: path.basename(absolutePath),
size: fileSize,
contentType: mimeType,
},
userId,
assetId: params.parent,
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Edit,
type: ResourceType.Asset,
id: p.parent,
})
const asset = await assetService.createFile({
parentId: p.parent,
name,
key: s3Key,
sizeByte: size,
contentType: mimeType,
creatorId: userId,
metadata: p.metadata ?? undefined,
})
const result = {
id: asset.id,
name: asset.name,
type: asset.type,
size: Number(asset.sizeByte),
}
return {
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
details: result,
details: {},
}
},
}
return tool
}
@@ -0,0 +1,56 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createCreateFolderTool } from './create-folder'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
Permission: { Edit: 'edit', Read: 'read' },
ResourceType: { Asset: 'asset' },
}))
vi.mock('@shumai/core/src/asset/asset', () => ({
assetService: {
createAsset: vi.fn().mockResolvedValue({
id: 'folder-1',
name: 'New Folder',
type: 'folder',
sizeByte: 0,
}),
},
}))
describe('createCreateFolderTool', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('checks edit permission and calls assetService.createAsset with folder type', async () => {
const tool = createCreateFolderTool('user-1')
const result = await tool.execute('call-1', { parent: 'root-folder', name: 'New Folder' })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'edit',
type: 'asset',
id: 'root-folder',
})
expect(assetService.createAsset).toHaveBeenCalledWith({
name: 'New Folder',
parentId: 'root-folder',
type: 'folder',
creatorId: 'user-1',
})
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
id: 'folder-1',
name: 'New Folder',
type: 'folder',
size: 0,
})
})
})
+25 -8
View File
@@ -1,6 +1,8 @@
import { Type } from '@sinclair/typebox'
import { Type } from 'typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { executeAgentToolWorkflow } from './utils'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { type User } from '@shumai/db'
const createFolderSchema = Type.Object({
parent: Type.String({
@@ -19,15 +21,30 @@ export function createCreateFolderTool(userId: string): AgentTool<typeof createF
description: 'Create a new folder in a specified parent folder.',
parameters: createFolderSchema,
execute: async (_toolCallId, params) => {
const result = await executeAgentToolWorkflow({
toolName: 'create_folder',
args: params,
userId,
assetId: params.parent,
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Edit,
type: ResourceType.Asset,
id: params.parent,
})
const newFolder = await assetService.createAsset({
name: params.name,
parentId: params.parent,
type: 'folder',
creatorId: userId,
})
const result = {
id: newFolder.id,
name: newFolder.name,
type: newFolder.type,
size: Number(newFolder.sizeByte),
}
return {
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
details: result,
details: {},
}
},
}
@@ -0,0 +1,207 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
import { createCreateVersionTool } from './create-version'
import { fieldsToTypeBoxSchema } from '../index'
import { s3Service } from '@shumai/core/src/s3/s3'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService } from '@shumai/core/src/authz/authz'
import { Value } from 'typebox/value'
import * as fs from 'fs'
import * as os from 'os'
import * as path from 'path'
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
uploadFileToKey: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
Permission: { Edit: 'edit', Read: 'read' },
ResourceType: { Asset: 'asset' },
}))
vi.mock('@shumai/core/src/asset/asset', () => ({
assetService: {
createVersion: vi
.fn()
.mockResolvedValue({ id: 'file-1', name: 'v2', type: 'file', sizeByte: 100 }),
},
}))
describe('createCreateVersionTool', () => {
const createdFiles: string[] = []
beforeEach(() => {
vi.clearAllMocks()
})
afterEach(() => {
for (const f of createdFiles) {
if (fs.existsSync(f)) {
fs.unlinkSync(f)
}
}
createdFiles.length = 0
})
const createTempFile = (content: string | Buffer, filename: string): string => {
const filePath = path.join(os.tmpdir(), filename)
fs.writeFileSync(filePath, content)
createdFiles.push(filePath)
return filePath
}
it('should upload a local file and create a version via assetService.createVersion', async () => {
const filePath = createTempFile('# Hello from disk', 'create-version-test.md')
const tool = createCreateVersionTool('user-1')
const result = await tool.execute('call-1', { parent: 'file-1', path: filePath })
expect(s3Service.uploadFileToKey).toHaveBeenCalledTimes(1)
const uploadArgs = vi.mocked(s3Service.uploadFileToKey).mock.calls[0]
const s3Key = uploadArgs[1]
expect(s3Key).toMatch(/^files\/.+\.md$/)
expect(uploadArgs[2]).toBe('text/markdown')
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'edit',
type: 'asset',
id: 'file-1',
})
expect(assetService.createVersion).toHaveBeenCalledWith({
parentId: 'file-1',
name: 'create-version-test.md',
key: s3Key,
sizeByte: Buffer.byteLength('# Hello from disk', 'utf-8'),
contentType: 'text/markdown',
creatorId: 'user-1',
metadata: undefined,
})
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
id: 'file-1',
name: 'v2',
type: 'file',
size: 100,
})
})
it('should forward the metadata to assetService.createVersion when provided', async () => {
const filePath = createTempFile('# Hello from disk', 'create-version-metadata.md')
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
])
const tool = createCreateVersionTool('user-1', metadataSchema)
await tool.execute('call-1', {
parent: 'file-1',
path: filePath,
metadata: { prompt: 'Generated using gemini' },
})
expect(assetService.createVersion).toHaveBeenCalledWith(
expect.objectContaining({
metadata: { prompt: 'Generated using gemini' },
}),
)
})
it('should forward select and selectMulti newOption metadata', async () => {
const filePath = createTempFile('# Hello from disk', 'create-version-new-option.md')
const metadataSchema = fieldsToTypeBoxSchema([
{
id: 'provider',
config: {
name: 'Provider',
type: 'select',
select: { options: [{ id: 'openai', displayName: 'OpenAI', color: '#f43f5e' }] },
},
},
{
id: 'tags',
config: {
name: 'Tags',
type: 'selectMulti',
selectMulti: { options: [{ id: 'tag1', displayName: 'Tag 1', color: '#3b82f6' }] },
},
},
])
const tool = createCreateVersionTool('user-1', metadataSchema)
await tool.execute('call-1', {
parent: 'file-1',
path: filePath,
metadata: {
provider: { newOption: { value: 'Kling' } },
tags: ['tag1', { newOption: { value: 'Tag 2' } }],
},
})
expect(assetService.createVersion).toHaveBeenCalledWith(
expect.objectContaining({
metadata: {
provider: { newOption: { value: 'Kling' } },
tags: ['tag1', { newOption: { value: 'Tag 2' } }],
},
}),
)
})
it('should not include metadata when metadata is null', async () => {
const filePath = createTempFile('# Hello from disk', 'create-version-no-metadata.md')
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
])
const tool = createCreateVersionTool('user-1', metadataSchema)
await tool.execute('call-1', { parent: 'file-1', path: filePath, metadata: null })
const callArgs = vi.mocked(assetService.createVersion).mock.calls[0][0]
expect(callArgs.metadata).toBeUndefined()
})
it('should throw when the local file does not exist', async () => {
const tool = createCreateVersionTool('user-1')
await expect(
tool.execute('call-1', { parent: 'file-1', path: '/nonexistent/file.md' }),
).rejects.toThrow('Local file not found at path: /nonexistent/file.md')
expect(s3Service.uploadFileToKey).not.toHaveBeenCalled()
expect(assetService.createVersion).not.toHaveBeenCalled()
})
it('should expose a strict, all-required (nullable) metadata schema when fields are provided', () => {
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
{ id: 'toggle', config: { name: 'Active', type: 'toggle' } },
])
const tool = createCreateVersionTool('user-1', metadataSchema)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
const meta = (tool.parameters as any).properties.metadata
// `metadata` is a nullable union: [strict object, null]
const metaObject = meta.anyOf[0]
expect(meta).toBeDefined()
expect(metaObject.required).toEqual(['prompt', 'toggle'])
expect(metaObject.additionalProperties).toBe(false)
expect(metaObject.properties.prompt.anyOf[0].type).toBe('string')
expect(metaObject.properties.prompt.anyOf[1].type).toBe('null')
expect(metaObject.properties.toggle.anyOf[0].type).toBe('boolean')
expect(Value.Check(meta, { prompt: 'p', toggle: true })).toBe(true)
expect(Value.Check(meta, { prompt: null, toggle: null })).toBe(true)
expect(Value.Check(meta, { prompt: 'p', toggle: null, unknownKey: 'x' })).toBe(false)
})
it('should omit the metadata parameter when no schema is provided', () => {
const tool = createCreateVersionTool('user-1')
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
expect((tool.parameters as any).properties.metadata).toBeUndefined()
})
})
+63 -65
View File
@@ -1,92 +1,90 @@
import { Type } from '@sinclair/typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { executeAgentToolWorkflow } from './utils'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { s3Service } from '@shumai/core/src/s3/s3'
import { readFileMimeType } from '@shumai/core/src/utils/file-mime'
import { sanitizeFilename } from '@shumai/core/src/utils/filename'
import { type User } from '@shumai/db'
import { Type, type TSchema } from 'typebox'
import * as fs from 'fs'
import * as path from 'path'
import { s3Service } from '@shumai/core/src/s3/s3'
import { detectSupportedMimeType } from '@shumai/core/src/utils/mime'
import { ulid } from 'ulid'
import { sanitizeFilename } from '@shumai/core/src/utils/filename'
function getMimeType(filePath: string): string {
try {
const fd = fs.openSync(filePath, 'r')
try {
const buffer = Buffer.alloc(4100)
const bytesRead = fs.readSync(fd, buffer, 0, 4100, 0)
const detected = detectSupportedMimeType(new Uint8Array(buffer.subarray(0, bytesRead)))
if (detected) return detected
} finally {
fs.closeSync(fd)
}
} catch {
/* Ignore detection errors and fallback to extension mapping */
}
const ext = path.extname(filePath).toLowerCase()
const mimeTypes: Record<string, string> = {
'.png': 'image/png',
'.jpg': 'image/jpeg',
'.jpeg': 'image/jpeg',
'.gif': 'image/gif',
'.webp': 'image/webp',
'.svg': 'image/svg+xml',
'.mp4': 'video/mp4',
'.mov': 'video/quicktime',
'.pdf': 'application/pdf',
'.txt': 'text/plain',
'.md': 'text/markdown',
'.markdown': 'text/markdown',
}
return mimeTypes[ext] || 'application/octet-stream'
interface CreateVersionToolParams {
parent: string
path: string
metadata?: Record<string, unknown> | null
}
const createVersionSchema = Type.Object({
parent: Type.String({
description:
'The parent file ID for which to create a new version. This parameter is required.',
}),
path: Type.String({
description:
'The absolute or relative local path to the file on disk. This parameter is required.',
}),
})
export function createCreateVersionTool(userId: string, metadataSchema?: TSchema): AgentTool {
const createVersionSchema = Type.Object(
{
parent: Type.String({
description:
'The parent file ID for which to create a new version. This parameter is required.',
}),
path: Type.String({
description:
'The absolute or relative local path to the file on disk. This parameter is required.',
}),
...(metadataSchema ? { metadata: Type.Union([metadataSchema, Type.Null()]) } : {}),
},
{ additionalProperties: false },
)
export function createCreateVersionTool(userId: string): AgentTool<typeof createVersionSchema> {
return {
const tool: AgentTool = {
name: 'create_version',
label: 'Create Version',
description: 'Create a new version of an existing file from a local file path.',
description:
'Create a new version of an existing file from a local file path. ' +
'The optional "metadata" parameter accepts only the CREATION_CONTEXT fields declared in its schema; never invent field keys, and use null for fields you do not know.',
parameters: createVersionSchema,
execute: async (_toolCallId, params) => {
const absolutePath = path.resolve(process.cwd(), params.path)
// The harness validates tool args against `createVersionSchema` before calling
// execute, so `params` is guaranteed to match this shape.
const p = params as CreateVersionToolParams
const absolutePath = path.resolve(process.cwd(), p.path)
if (!fs.existsSync(absolutePath)) {
throw new Error(`Local file not found at path: ${params.path}`)
throw new Error(`Local file not found at path: ${p.path}`)
}
const fileSize = fs.statSync(absolutePath).size
const mimeType = getMimeType(absolutePath)
const mimeType = readFileMimeType(absolutePath)
const name = path.basename(absolutePath)
// Generate compliant S3 key matching normal file upload format
const s3Key = `files/${ulid()}/${sanitizeFilename(path.basename(absolutePath))}`
const s3Key = `files/${ulid()}/${sanitizeFilename(name)}`
await s3Service.uploadFileToKey(absolutePath, s3Key, mimeType)
const result = await executeAgentToolWorkflow({
toolName: 'create_version',
args: {
parent: params.parent,
s3Key,
name: path.basename(absolutePath),
size: fileSize,
contentType: mimeType,
},
userId,
assetId: params.parent,
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Edit,
type: ResourceType.Asset,
id: p.parent,
})
const asset = await assetService.createVersion({
parentId: p.parent,
name,
key: s3Key,
sizeByte: fileSize,
contentType: mimeType,
creatorId: userId,
metadata: p.metadata ?? undefined,
})
const result = {
id: asset.id,
name: asset.name,
type: asset.type,
size: Number(asset.sizeByte),
}
return {
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
details: result,
details: {},
}
},
}
return tool
}
+264 -90
View File
@@ -1,6 +1,6 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
import { createDownloadAssetTool } from './download-asset'
import { prisma, type Asset } from '@shumai/db'
import { createDownloadAssetTool, resolveAssetIdFromKey } from './download-asset'
import { prisma, type Asset, type StorageKey } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { authzService } from '@shumai/core/src/authz/authz'
import * as fs from 'fs'
@@ -13,6 +13,11 @@ vi.mock('@shumai/db', async (importOriginal) => {
prisma: {
asset: {
findUnique: vi.fn(),
findFirst: vi.fn(),
},
storageKey: {
findFirst: vi.fn(),
findUnique: vi.fn(),
},
},
}
@@ -51,119 +56,288 @@ describe('downloadAssetTool', () => {
it('should throw authorization error if user ID is empty', async () => {
const tool = createDownloadAssetTool('')
await expect(tool.execute('call-1', { assetId: 'asset-1' })).rejects.toThrow(
await expect(tool.execute('call-1', { assetId: 'asset-1', key: null })).rejects.toThrow(
'User ID is required for authorization.',
)
})
it('should throw error if asset not found', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
it('should throw error if both assetId and key are null', async () => {
const tool = createDownloadAssetTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1' })).rejects.toThrow(
'Asset with ID asset-1 not found.',
await expect(tool.execute('call-1', { assetId: null, key: null })).rejects.toThrow(
'Provide exactly one of "assetId" (to download a workspace asset) or "key" (to download a specific S3 storage key). Set the unused parameter to null.',
)
})
it('should download proxy image to .pi directory when imageTranscodes is present', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-img-1',
name: 'sample_photo.jpg',
storageKey: { key: 'raw/sample_photo.jpg' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/sample_photo.webp' }],
},
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-proxy-image-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
it('should throw error if both assetId and key are provided', async () => {
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'asset-img-1' })
await expect(
tool.execute('call-1', { assetId: 'asset-1', key: 'files/path/file.png' }),
).rejects.toThrow(
'Provide exactly one of "assetId" (to download a workspace asset) or "key" (to download a specific S3 storage key). Set the unused parameter to null.',
)
})
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-img-1',
describe('Download by assetId', () => {
it('should throw error if asset not found', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
const tool = createDownloadAssetTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', key: null })).rejects.toThrow(
'Asset with ID asset-1 not found.',
)
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/sample_photo.webp')
it('should download proxy image to .pi directory when downloading by assetId', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-img-1',
name: 'sample_photo.jpg',
storageKey: { key: 'raw/sample_photo.jpg' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/sample_photo.webp' }],
},
} as unknown as Asset)
const expectedPath = path.join(piDir, 'asset-img-1_sample_photo.jpg')
createdFiles.push(expectedPath)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-proxy-image-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
expect(fs.existsSync(expectedPath)).toBe(true)
expect(fs.readFileSync(expectedPath, 'utf-8')).toBe('fake-proxy-image-bytes')
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'asset-img-1', key: null })
expect(result.details.filePath).toBe(path.join('.pi', 'asset-img-1_sample_photo.jpg'))
expect(result.details.contentType).toBe('image/webp')
expect((result.content[0] as { type: 'text'; text: string }).text).toContain(
'Remember to delete this temporary file when finished.',
)
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-img-1',
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/sample_photo.webp')
const expectedPath = path.join(piDir, 'asset-img-1_sample_photo.jpg')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(fs.readFileSync(expectedPath, 'utf-8')).toBe('fake-proxy-image-bytes')
expect(result.details.filePath).toBe(path.join('.pi', 'asset-img-1_sample_photo.jpg'))
expect(result.details.contentType).toBe('image/webp')
expect((result.content[0] as { type: 'text'; text: string }).text).toContain(
'Remember to delete this temporary file when finished.',
)
})
it('should resolve version_stack asset to its latest version file', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'stack-1',
name: '',
type: 'version_stack',
storageKey: null,
media: null,
} as unknown as Asset)
vi.mocked(prisma.asset.findFirst).mockResolvedValue({
id: 'file-ver-2',
name: 'banana_pig.png',
type: 'file',
storageKey: { key: 'raw/banana_pig.png' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/banana_pig.webp' }],
},
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-version-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'stack-1', key: null })
expect(prisma.asset.findFirst).toHaveBeenCalledWith({
where: { parentId: 'stack-1', isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/banana_pig.webp')
const expectedPath = path.join(piDir, 'file-ver-2_banana_pig.png')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(result.details.name).toBe('banana_pig.png')
})
})
it('should download proxy video to .pi directory when videoTranscodes is present', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-vid-1',
name: 'video_test.mov',
storageKey: { key: 'raw/video_test.mov' },
media: {
proxyType: 'video',
videoTranscodes: [{ key: 'proxy/video_test.mp4' }],
},
} as unknown as Asset)
describe('Download by S3 key and Authorization', () => {
it('should reject keys that do not start with files/', async () => {
const tool = createDownloadAssetTool('user-1')
await expect(
tool.execute('call-1', { assetId: null, key: 'avatars/user-1.png' }),
).rejects.toThrow('Invalid storage key format: key must start with "files/"')
})
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-video-bytes'),
contentType: 'video/mp4',
} as unknown as { buffer: Buffer; contentType: string })
it('should reject keys with path traversal (..)', async () => {
const tool = createDownloadAssetTool('user-1')
await expect(
tool.execute('call-1', { assetId: null, key: 'files/../../etc/passwd' }),
).rejects.toThrow('cannot contain ".."')
})
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'asset-vid-1' })
it('should reject keys with insufficient path segments', async () => {
const tool = createDownloadAssetTool('user-1')
await expect(tool.execute('call-1', { assetId: null, key: 'files/invalid' })).rejects.toThrow(
'Invalid storage key format',
)
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/video_test.mp4')
it('should reject keys that do not belong to any valid asset in database', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findFirst).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findUnique).mockResolvedValue(null)
const expectedPath = path.join(piDir, 'asset-vid-1_video_test.mov')
createdFiles.push(expectedPath)
const tool = createDownloadAssetTool('user-1')
await expect(
tool.execute('call-1', { assetId: null, key: 'files/nonexistent-id/shot.webp' }),
).rejects.toThrow('does not belong to any valid asset or the asset has been deleted')
})
expect(fs.existsSync(expectedPath)).toBe(true)
expect(result.details.size).toBe(Buffer.from('fake-video-bytes').length)
it('should reject download when user lacks read permission on the owning asset (cross-project protection)', async () => {
// Key belongs to asset-in-other-project
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-in-other-project',
} as unknown as Asset)
// Authz rejects because user is in a different project/team
vi.mocked(authzService.hasPermission).mockRejectedValue(
new Error('User does not have Read permission on asset asset-in-other-project'),
)
const tool = createDownloadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: null,
key: 'files/asset-in-other-project/screenshots/secret_frame.webp',
}),
).rejects.toThrow('User does not have Read permission on asset asset-in-other-project')
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-in-other-project',
})
expect(s3Service.getObject).not.toHaveBeenCalled()
})
it('should authorize and download derived artifact key (files/<assetId>/...)', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'ast-123',
} as unknown as Asset)
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-screenshot-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: null,
key: 'files/ast-123/screenshots/shot_5.0s.webp',
})
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'ast-123',
})
expect(s3Service.getObject).toHaveBeenCalledWith(
'shumai',
'files/ast-123/screenshots/shot_5.0s.webp',
)
const expectedPath = path.join(piDir, 'shot_5.0s.webp')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(fs.readFileSync(expectedPath, 'utf-8')).toBe('fake-screenshot-bytes')
expect(result.details.filePath).toBe(path.join('.pi', 'shot_5.0s.webp'))
expect(result.details.key).toBe('files/ast-123/screenshots/shot_5.0s.webp')
expect(result.details.assetId).toBe('ast-123')
expect(result.details.size).toBe(Buffer.from('fake-screenshot-bytes').length)
})
it('should authorize and download original upload key (files/<uploadUlid>/...) via StorageKey', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findFirst).mockResolvedValue({
id: 'sk-1',
key: 'files/upload-ulid-456/photo.png',
assets: [{ id: 'asset-photo-1' }],
} as unknown as StorageKey & { assets: Asset[] })
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-original-photo-bytes'),
contentType: 'image/png',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: null,
key: 'files/upload-ulid-456/photo.png',
})
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-photo-1',
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'files/upload-ulid-456/photo.png')
const expectedPath = path.join(piDir, 'photo.png')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(result.details.assetId).toBe('asset-photo-1')
})
})
it('should download original file to .pi directory even when pdfTranscode is present', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-md-1',
name: 'doc.md',
storageKey: { key: 'raw/doc.md' },
media: {
proxyType: 'pdf',
pdfTranscode: { key: 'proxy/doc.pdf' },
original: { key: 'raw/doc.md' },
},
} as unknown as Asset)
describe('resolveAssetIdFromKey helper unit tests', () => {
it('resolves directly by asset ID', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue({ id: 'ast-999' } as unknown as Asset)
const assetId = await resolveAssetIdFromKey('files/ast-999/pdf_pages/page_1.webp')
expect(assetId).toBe('ast-999')
})
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('# fake markdown bytes'),
contentType: 'text/markdown',
} as unknown as { buffer: Buffer; contentType: string })
it('resolves via StorageKey prefix', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findFirst).mockResolvedValue({
assets: [{ id: 'ast-888' }],
} as unknown as StorageKey & { assets: Asset[] })
const assetId = await resolveAssetIdFromKey('files/ulid-777/document.pdf')
expect(assetId).toBe('ast-888')
})
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'asset-md-1' })
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'raw/doc.md')
const expectedPath = path.join(piDir, 'asset-md-1_doc.md')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(result.details.contentType).toBe('text/markdown')
it('resolves via exact StorageKey match', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findFirst).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findUnique).mockResolvedValue({
assets: [{ id: 'ast-777' }],
} as unknown as StorageKey & { assets: Asset[] })
const assetId = await resolveAssetIdFromKey('files/ulid-special/document.pdf')
expect(assetId).toBe('ast-777')
})
})
})
+209 -73
View File
@@ -1,4 +1,4 @@
import { Type } from '@sinclair/typebox'
import { Type } from 'typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { prisma, type User } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
@@ -7,71 +7,107 @@ import { sanitizeFilename } from '@shumai/core/src/utils/filename'
import * as fs from 'fs'
import * as path from 'path'
const downloadAssetSchema = Type.Object({
assetId: Type.String({
description:
'The asset ID of the workspace asset to download to local disk. This parameter is required.',
}),
})
export const downloadAssetSchema = Type.Object(
{
assetId: Type.Union([
Type.String({
description:
'The asset ID of the workspace asset to download to local disk, or null if providing "key".',
}),
Type.Null(),
]),
key: Type.Union([
Type.String({
description:
'The storage S3 key of a specific media artifact (e.g. a screenshot or transcode key from read_asset) to download to local disk, or null if providing "assetId".',
}),
Type.Null(),
]),
},
{ additionalProperties: false },
)
export async function resolveAssetIdFromKey(key: string): Promise<string> {
if (!key.startsWith('files/') || key.includes('..')) {
throw new Error(
'Invalid storage key format: key must start with "files/" and cannot contain "..".',
)
}
const segments = key.split('/')
if (segments.length < 3) {
throw new Error(
'Invalid storage key format: key must follow "files/<identifier>/<path>" structure.',
)
}
const identifier = segments[1]
// 1. Check if identifier is directly an Asset ID (Derived Artifacts: screenshots, pdf_pages, annotations, proxies)
const assetById = await prisma.asset.findUnique({
where: { id: identifier, isDeleted: false },
select: { id: true },
})
if (assetById) {
return assetById.id
}
// 2. Check if identifier is a StorageKey directory prefix (Original uploads / agent-created files)
const storageKeyByPrefix = await prisma.storageKey.findFirst({
where: { key: { startsWith: `files/${identifier}/` } },
include: {
assets: {
where: { isDeleted: false },
select: { id: true },
},
},
})
if (storageKeyByPrefix?.assets && storageKeyByPrefix.assets.length > 0) {
return storageKeyByPrefix.assets[0].id
}
// 3. Exact StorageKey fallback
const exactStorageKey = await prisma.storageKey.findUnique({
where: { key },
include: {
assets: {
where: { isDeleted: false },
select: { id: true },
},
},
})
if (exactStorageKey?.assets && exactStorageKey.assets.length > 0) {
return exactStorageKey.assets[0].id
}
throw new Error(
`Storage key "${key}" does not belong to any valid asset or the asset has been deleted.`,
)
}
export function createDownloadAssetTool(userId: string): AgentTool<typeof downloadAssetSchema> {
return {
name: 'download_asset',
label: 'Download Asset',
description:
'Downloads a workspace asset (proxy version if available) to the local .pi directory for processing with local scripts or bash commands. Remember to delete the temporary downloaded file when finished.',
'Downloads a workspace asset or specific storage S3 key to the local .pi directory for processing with local scripts or bash commands. ' +
'Provide exactly one of "assetId" (for full assets) or "key" (for intermediate artifacts like screenshots from read_asset). ' +
'Remember to delete the temporary downloaded file when finished.',
parameters: downloadAssetSchema,
execute: async (_toolCallId, params) => {
const assetId = params.assetId
const { assetId, key } = params
if (!userId) {
throw new Error('User ID is required for authorization.')
}
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: assetId,
})
const hasAssetId = assetId !== null
const hasKey = key !== null
const asset = await prisma.asset.findUnique({
where: { id: assetId },
include: { storageKey: true },
})
if (!asset) {
throw new Error(`Asset with ID ${assetId} not found.`)
}
let mediaKey = asset.storageKey?.key
if (asset.media) {
const mediaInfo = asset.media as unknown as PrismaJson.MediaInfo
if (
mediaInfo.proxyType === 'image' ||
(mediaInfo.imageTranscodes && mediaInfo.imageTranscodes.length > 0)
) {
if (mediaInfo.imageTranscodes && mediaInfo.imageTranscodes.length > 0) {
mediaKey = mediaInfo.imageTranscodes[0].key || mediaKey
}
} else if (
mediaInfo.proxyType === 'video' ||
(mediaInfo.videoTranscodes && mediaInfo.videoTranscodes.length > 0) ||
mediaInfo.videoPreview?.key
) {
if (mediaInfo.videoTranscodes && mediaInfo.videoTranscodes.length > 0) {
mediaKey = mediaInfo.videoTranscodes[0].key || mediaKey
} else if (mediaInfo.videoPreview?.key) {
mediaKey = mediaInfo.videoPreview.key
}
} else if (mediaInfo.original?.key) {
mediaKey = mediaInfo.original.key || mediaKey
}
}
if (!mediaKey) {
throw new Error(`No media content found for asset ${assetId}.`)
if (hasAssetId === hasKey) {
throw new Error(
'Provide exactly one of "assetId" (to download a workspace asset) or "key" (to download a specific S3 storage key). Set the unused parameter to null.',
)
}
const piDir = path.join(process.cwd(), '.pi')
@@ -79,32 +115,132 @@ export function createDownloadAssetTool(userId: string): AgentTool<typeof downlo
fs.mkdirSync(piDir, { recursive: true })
}
const safeName = sanitizeFilename(asset.name || 'file')
const filename = `${asset.id}_${safeName}`
const targetFilePath = path.join(piDir, filename)
const relativePath = path.join('.pi', filename)
const bucket = process.env.S3_BUCKET || 'shumai'
const { buffer, contentType } = await s3Service.getObject(bucket, mediaKey)
fs.writeFileSync(targetFilePath, buffer)
// Branch A: Download by assetId
if (assetId) {
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: assetId,
})
return {
content: [
{
type: 'text',
text: `Downloaded asset "${asset.name}" (ID: ${assetId}) to "${relativePath}". Remember to delete this temporary file when finished.`,
let asset = await prisma.asset.findUnique({
where: { id: assetId },
include: { storageKey: true },
})
if (!asset) {
throw new Error(`Asset with ID ${assetId} not found.`)
}
if (asset.type === 'version_stack') {
const latestVersion = await prisma.asset.findFirst({
where: { parentId: asset.id, isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
})
if (latestVersion) {
asset = latestVersion
}
}
let mediaKey = asset.storageKey?.key
if (asset.media) {
const mediaInfo = asset.media as unknown as PrismaJson.MediaInfo
if (
mediaInfo.proxyType === 'image' ||
(mediaInfo.imageTranscodes && mediaInfo.imageTranscodes.length > 0)
) {
if (mediaInfo.imageTranscodes && mediaInfo.imageTranscodes.length > 0) {
mediaKey = mediaInfo.imageTranscodes[0].key || mediaKey
}
} else if (
mediaInfo.proxyType === 'video' ||
(mediaInfo.videoTranscodes && mediaInfo.videoTranscodes.length > 0) ||
mediaInfo.videoPreview?.key
) {
if (mediaInfo.videoTranscodes && mediaInfo.videoTranscodes.length > 0) {
mediaKey = mediaInfo.videoTranscodes[0].key || mediaKey
} else if (mediaInfo.videoPreview?.key) {
mediaKey = mediaInfo.videoPreview.key
}
} else if (mediaInfo.original?.key) {
mediaKey = mediaInfo.original.key || mediaKey
}
}
if (!mediaKey) {
throw new Error(`No media content found for asset ${assetId}.`)
}
const safeName = sanitizeFilename(asset.name || 'file')
const filename = `${asset.id}_${safeName}`
const targetFilePath = path.join(piDir, filename)
const relativePath = path.join('.pi', filename)
const { buffer, contentType } = await s3Service.getObject(bucket, mediaKey)
fs.writeFileSync(targetFilePath, buffer)
return {
content: [
{
type: 'text',
text: `Downloaded asset "${asset.name}" (ID: ${assetId}) to "${relativePath}". Remember to delete this temporary file when finished.`,
},
],
details: {
assetId,
name: asset.name,
filePath: relativePath,
absolutePath: targetFilePath,
contentType: contentType || 'application/octet-stream',
size: buffer.length,
},
],
details: {
assetId,
name: asset.name,
filePath: relativePath,
absolutePath: targetFilePath,
contentType: contentType || 'application/octet-stream',
size: buffer.length,
},
}
}
// Branch B: Download by S3 storage key
if (key) {
const owningAssetId = await resolveAssetIdFromKey(key)
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: owningAssetId,
})
const rawBasename = path.basename(key)
const safeBasename = sanitizeFilename(rawBasename) || 'downloaded_file'
const filename = safeBasename
const targetFilePath = path.join(piDir, filename)
const relativePath = path.join('.pi', filename)
const { buffer, contentType } = await s3Service.getObject(bucket, key)
fs.writeFileSync(targetFilePath, buffer)
return {
content: [
{
type: 'text',
text: `Downloaded storage key "${key}" to "${relativePath}". Remember to delete this temporary file when finished.`,
},
],
details: {
key,
assetId: owningAssetId,
filePath: relativePath,
absolutePath: targetFilePath,
contentType: contentType || 'application/octet-stream',
size: buffer.length,
},
}
}
throw new Error('Invalid download_asset parameters.')
},
}
}
@@ -0,0 +1,79 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createListAssetsTool } from './list-assets'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
Permission: { Edit: 'edit', Read: 'read' },
ResourceType: { Asset: 'asset' },
}))
vi.mock('@shumai/core/src/asset/asset', () => ({
assetService: {
listChildren: vi.fn().mockResolvedValue({
data: [
{ id: '01M2FILE2', name: 'file2.txt', type: 'file', sizeByte: 200 },
{ id: '01M1FILE1', name: 'file1.txt', type: 'file', sizeByte: 100 },
],
pageInfo: { hasNextPage: false, hasPreviousPage: false, cursor: 'abc' },
}),
},
}))
describe('createListAssetsTool', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('checks read permissions and invokes assetService.listChildren with default all and id desc ordering', async () => {
const tool = createListAssetsTool('user-1')
const result = await tool.execute('call-1', { parent: 'folder-1' })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'read',
type: 'asset',
id: 'folder-1',
})
expect(assetService.listChildren).toHaveBeenCalledWith({
assetId: 'folder-1',
assetType: 'all',
sort: 'createdAt',
order: 'desc',
first: 20,
after: undefined,
})
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
assets: [
{ id: '01M2FILE2', name: 'file2.txt', type: 'file', size: 200 },
{ id: '01M1FILE1', name: 'file1.txt', type: 'file', size: 100 },
],
pageInfo: { hasNextPage: false, hasPreviousPage: false, cursor: 'abc' },
})
})
it('passes explicit pagination and type filter options to assetService.listChildren', async () => {
const tool = createListAssetsTool('user-1')
await tool.execute('call-1', {
parent: 'folder-1',
page: 2,
pageSize: 10,
type: 'folder',
})
expect(assetService.listChildren).toHaveBeenCalledWith({
assetId: 'folder-1',
assetType: 'folder',
sort: 'createdAt',
order: 'desc',
first: 10,
after: expect.any(String),
})
})
})
+39 -11
View File
@@ -1,6 +1,9 @@
import { Type } from '@sinclair/typebox'
import { Type } from 'typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { executeAgentToolWorkflow } from './utils'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { encodeCursor } from '@shumai/core/src/pagination'
import { type User } from '@shumai/db'
const listAssetsSchema = Type.Object({
parent: Type.String({
@@ -19,9 +22,8 @@ const listAssetsSchema = Type.Object({
}),
),
type: Type.Optional(
Type.String({
Type.Union([Type.Literal('file'), Type.Literal('folder'), Type.Literal('all')], {
description: "Filter assets by type: 'file', 'folder', or 'all'. Default is 'all'.",
enum: ['file', 'folder', 'all'],
default: 'all',
}),
),
@@ -34,15 +36,41 @@ export function createListAssetsTool(userId: string): AgentTool<typeof listAsset
description: 'List the assets (files and folders) inside a parent folder with pagination.',
parameters: listAssetsSchema,
execute: async (_toolCallId, params) => {
const result = await executeAgentToolWorkflow({
toolName: 'list_assets',
args: params,
userId,
assetId: params.parent,
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: params.parent,
})
const page = params.page || 1
const pageSize = params.pageSize || 20
const type = params.type || 'all'
const result = await assetService.listChildren({
assetId: params.parent,
assetType: type,
sort: 'createdAt',
order: 'desc',
first: pageSize,
after: page > 1 ? encodeCursor((page - 1) * pageSize) : undefined,
})
const assets = result.data.map((a) => ({
id: a.id,
name: a.name,
type: a.type,
size: Number(a.sizeByte),
}))
const output = {
assets,
pageInfo: result.pageInfo,
}
return {
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
details: result,
content: [{ type: 'text', text: JSON.stringify(output, null, 2) }],
details: {},
}
},
}
+544
View File
@@ -0,0 +1,544 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createReadAssetTool } from './read-asset'
import {
prisma,
WorkflowTaskType,
WorkflowTaskStatus,
type Asset,
type WorkflowTask,
} from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService } from '@shumai/core/src/authz/authz'
import * as annotationResolver from './annotation-resolver'
vi.mock('@shumai/db', async (importOriginal) => {
const actual = await importOriginal<typeof import('@shumai/db')>()
return {
...actual,
prisma: {
asset: {
findUnique: vi.fn(),
findFirst: vi.fn(),
},
workflowTask: {
create: vi.fn(),
},
},
}
})
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
getObject: vi.fn(),
presign: vi.fn().mockResolvedValue('https://mock-presigned-url.com'),
},
}))
vi.mock('@shumai/workflow-core', () => ({
workflowService: {
executeWait: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn(),
},
Permission: { Read: 'Read' },
ResourceType: { Asset: 'asset' },
}))
describe('readAssetTool', () => {
beforeEach(() => {
vi.clearAllMocks()
vi.spyOn(annotationResolver, 'resolveAnnotationsById').mockResolvedValue({
annotations: null,
timestamp: null,
})
})
it('should throw authorization error if user ID is empty', async () => {
const tool = createReadAssetTool('')
await expect(
tool.execute('call-1', {
assetId: 'asset-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
}),
).rejects.toThrow('User ID is required for authorization.')
})
it('should throw error if asset not found', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'asset-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
}),
).rejects.toThrow('Asset with ID asset-1 not found.')
})
describe('Image Assets', () => {
it('should read image and return ImageContent and S3 key', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'img-1',
name: 'photo.png',
mediaType: 'image/png',
storageKey: { key: 'raw/photo.png' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/photo.webp' }],
},
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-image-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'img-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
})
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'img-1',
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/photo.webp')
const textItem = result.content[0] as { type: 'text'; text: string }
const imageItem = result.content[1] as { type: 'image'; data: string; mimeType: string }
expect(textItem.text).toContain('proxy/photo.webp')
expect(imageItem.type).toBe('image')
expect(imageItem.data).toBe(Buffer.from('fake-image-bytes').toString('base64'))
expect(result.details.key).toBe('proxy/photo.webp')
})
it('should trigger image annotation transcode workflow when annotationId is provided', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'img-1',
name: 'photo.png',
mediaType: 'image/png',
projectId: 'project-1',
storageKey: { key: 'raw/photo.png' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/photo.webp' }],
},
} as unknown as Asset)
vi.spyOn(annotationResolver, 'resolveAnnotationsById').mockResolvedValue({
annotations: [
{
type: 'arrow',
color: '#ff0000',
points: [
[0, 0],
[10, 10],
],
},
],
timestamp: null,
})
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-anno',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-anno',
output: { key: 'annotated/photo.webp' },
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('annotated-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'img-1',
annotationId: 'comment-1',
imageConfig: null,
videoConfig: null,
docConfig: null,
})
expect(prisma.workflowTask.create).toHaveBeenCalledWith({
data: {
assetId: 'img-1',
projectId: 'project-1',
type: WorkflowTaskType.transcode_image_annotation,
status: WorkflowTaskStatus.pending,
payload: {
projectId: 'project-1',
imageAnnotation: {
annotations: [
{
type: 'arrow',
color: '#ff0000',
points: [
[0, 0],
[10, 10],
],
},
],
},
},
},
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'annotated/photo.webp')
expect(result.details.key).toBe('annotated/photo.webp')
})
})
describe('Video Assets', () => {
it('should throw error if videoConfig is missing for video assets', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'vid-1',
name: 'video.mp4',
mediaType: 'video/mp4',
media: { proxyType: 'video' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'vid-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
}),
).rejects.toThrow('videoConfig with start, end, and count is required')
})
it('should throw error on invalid video time range', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'vid-1',
name: 'video.mp4',
mediaType: 'video/mp4',
media: { proxyType: 'video' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'vid-1',
annotationId: null,
imageConfig: null,
videoConfig: { start: 10, end: 5, count: 1 },
docConfig: null,
}),
).rejects.toThrow(
'Invalid video time range: start (10) must be less than or equal to end (5).',
)
})
it('should extract frames and return ImageContent array and keys', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'vid-1',
name: 'video.mp4',
mediaType: 'video/mp4',
projectId: 'project-1',
media: { proxyType: 'video' },
} as unknown as Asset)
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-shot',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-shot',
output: {
screenshots: [
{ key: 'screenshots/shot1.webp', timestamp: 2.0 },
{ key: 'screenshots/shot2.webp', timestamp: 4.0 },
],
},
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockImplementation(async (_bucket, key) => ({
buffer: Buffer.from(`bytes-for-${key}`),
contentType: 'image/webp',
}))
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'vid-1',
annotationId: null,
imageConfig: null,
videoConfig: { start: 2.0, end: 4.0, count: 2 },
docConfig: null,
})
expect(prisma.workflowTask.create).toHaveBeenCalledWith({
data: {
assetId: 'vid-1',
projectId: 'project-1',
type: WorkflowTaskType.transcode_screenshot,
status: WorkflowTaskStatus.pending,
payload: {
projectId: 'project-1',
screenshot: {
start: 2.0,
end: 4.0,
count: 2,
commentTimestamp: null,
annotations: null,
},
},
},
})
expect(result.content.length).toBe(3) // 1 text + 2 images
expect(result.details.keys).toEqual(['screenshots/shot1.webp', 'screenshots/shot2.webp'])
})
})
describe('Document Assets', () => {
it('should throw error if docConfig is missing for document assets', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'doc.pdf',
mediaType: 'application/pdf',
media: { proxyType: 'pdf' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'doc-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
}),
).rejects.toThrow('docConfig with mode ("pages" or "text") is required')
})
it('should render PDF pages in mode: "pages"', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'doc.pdf',
mediaType: 'application/pdf',
projectId: 'project-1',
media: { proxyType: 'pdf' },
} as unknown as Asset)
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-pdf',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-pdf',
output: {
pages: [
{ key: 'pdf_pages/page1.webp', page: 1 },
{ key: 'pdf_pages/page2.webp', page: 2 },
],
},
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockImplementation(async (_bucket, key) => ({
buffer: Buffer.from(`bytes-for-${key}`),
contentType: 'image/webp',
}))
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'doc-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'pages', startPage: 1, endPage: 2 },
})
expect(result.content.length).toBe(3) // 1 text + 2 images
expect(result.details.keys).toEqual(['pdf_pages/page1.webp', 'pdf_pages/page2.webp'])
})
it('should throw error in mode: "pages" when page range exceeds 20 pages', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'doc.pdf',
mediaType: 'application/pdf',
media: { proxyType: 'pdf' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'doc-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'pages', startPage: 1, endPage: 25 },
}),
).rejects.toThrow('exceeds the maximum limit of 20 pages')
})
it('should throw error in mode: "text" when called on binary PDF', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'manual.pdf',
mediaType: 'application/pdf',
media: { proxyType: 'pdf' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'doc-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'text', startPage: null, endPage: null },
}),
).rejects.toThrow('is a binary PDF/document and cannot be read as raw text')
})
it('should throw error in mode: "text" when annotationId is provided', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'notes.md',
mediaType: 'text/markdown',
media: { proxyType: 'pdf' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'doc-1',
annotationId: 'comment-123',
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'text', startPage: null, endPage: null },
}),
).rejects.toThrow('annotationId cannot be used with docConfig mode "text"')
})
it('should throw error in mode: "text" when startPage or endPage is non-null', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'notes.md',
mediaType: 'text/markdown',
media: { proxyType: 'pdf' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'doc-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'text', startPage: 1, endPage: 2 },
}),
).rejects.toThrow('startPage and endPage must be null when docConfig mode is "text"')
})
it('should read raw text from S3 in mode: "text" for markdown files', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-md',
name: 'README.md',
mediaType: 'text/markdown',
storageKey: { key: 'raw/README.md' },
media: { proxyType: 'pdf' },
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('# Hello Shumai\nThis is markdown text.'),
contentType: 'text/markdown',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'doc-md',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'text', startPage: null, endPage: null },
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'raw/README.md')
expect(result.content[0].type).toBe('text')
expect((result.content[0] as { type: 'text'; text: string }).text).toContain(
'# Hello Shumai\nThis is markdown text.',
)
expect(result.details.key).toBe('raw/README.md')
})
})
describe('Version Stack Resolution', () => {
it('should resolve version_stack to latest version asset', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'stack-1',
name: '',
type: 'version_stack',
} as unknown as Asset)
vi.mocked(prisma.asset.findFirst).mockResolvedValue({
id: 'file-v2',
name: 'diagram_v2.png',
type: 'file',
mediaType: 'image/png',
storageKey: { key: 'raw/diagram_v2.png' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/diagram_v2.webp' }],
},
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('v2-image-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'stack-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
})
expect(prisma.asset.findFirst).toHaveBeenCalledWith({
where: { parentId: 'stack-1', isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
})
expect(result.details.name).toBe('diagram_v2.png')
expect(result.details.key).toBe('proxy/diagram_v2.webp')
})
})
})
+576
View File
@@ -0,0 +1,576 @@
import { Type } from 'typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { type ImageContent, type TextContent } from '@earendil-works/pi-ai'
import { prisma, WorkflowTaskType, WorkflowTaskStatus, type User } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { resolveAnnotationsById } from './annotation-resolver'
export const readAssetSchema = Type.Object(
{
assetId: Type.String({
description: 'The asset ID of the workspace asset to inspect. Required.',
}),
annotationId: Type.Union([
Type.String({
description:
'The ID of the comment or message entry from <annotation id="..." /> whose visual markup should be overlaid, or null.',
}),
Type.Null(),
]),
imageConfig: Type.Union([
Type.Object(
{},
{
additionalProperties: false,
description: 'Optional configuration for image assets, or null.',
},
),
Type.Null(),
]),
videoConfig: Type.Union([
Type.Object(
{
start: Type.Number({ description: 'Start time in seconds.' }),
end: Type.Number({ description: 'End time in seconds.' }),
count: Type.Number({ description: 'Number of frames to extract (e.g. 1 to 10).' }),
},
{
additionalProperties: false,
description:
'Configuration for video assets. Required when inspecting videos, or null for other asset types.',
},
),
Type.Null(),
]),
docConfig: Type.Union([
Type.Object(
{
mode: Type.Union([
Type.Literal('pages', {
description:
'Render specific page range as images for visual layout, charts, or visual annotations (required for binary PDFs).',
}),
Type.Literal('text', {
description:
'Read raw text content directly (ideal for .md, .txt, .csv, code files).',
}),
]),
startPage: Type.Union([
Type.Number({
description:
'Start page number (1-based index, required when mode="pages"), or null when mode="text".',
}),
Type.Null(),
]),
endPage: Type.Union([
Type.Number({
description:
'End page number (1-based index, max 20 pages per call, required when mode="pages"), or null when mode="text".',
}),
Type.Null(),
]),
},
{
additionalProperties: false,
description:
'Configuration for document/PDF/text assets, or null if inspecting another asset type.',
},
),
Type.Null(),
]),
},
{ additionalProperties: false },
)
export function isPlainTextAsset(mediaType: string, filename: string): boolean {
if (
mediaType.startsWith('text/') ||
mediaType === 'application/json' ||
mediaType === 'application/javascript' ||
mediaType === 'application/typescript' ||
mediaType === 'application/xml' ||
mediaType === 'application/x-yaml'
) {
return true
}
const textExtensions = [
'.txt',
'.md',
'.markdown',
'.json',
'.csv',
'.tsv',
'.js',
'.ts',
'.tsx',
'.jsx',
'.html',
'.htm',
'.css',
'.scss',
'.yaml',
'.yml',
'.xml',
'.py',
'.rb',
'.go',
'.rs',
'.c',
'.cpp',
'.h',
'.hpp',
'.java',
'.sh',
'.bash',
'.zsh',
'.sql',
]
return textExtensions.some((ext) => filename.endsWith(ext))
}
const MAX_TEXT_BYTES = 50 * 1024
const MAX_TEXT_LINES = 1000
export function createReadAssetTool(userId: string): AgentTool<typeof readAssetSchema> {
return {
name: 'read_asset',
label: 'Read Asset',
description:
'Inspects and reads the content of an asset in the workspace. ' +
'Supports visual analysis for images, video frame extraction, PDF page rendering, and direct text reading for text/markdown files. ' +
'Pass assetId, and provide the specific config (imageConfig, videoConfig, or docConfig) while setting unused configs to null. ' +
'Optionally provide annotationId to view visual markups drawn on the asset.',
parameters: readAssetSchema,
execute: async (_toolCallId, params) => {
const { assetId, annotationId, videoConfig, docConfig } = params
if (!userId) {
throw new Error('User ID is required for authorization.')
}
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: assetId,
})
let asset = await prisma.asset.findUnique({
where: { id: assetId },
include: { storageKey: true },
})
if (!asset) {
throw new Error(`Asset with ID ${assetId} not found.`)
}
if (asset.type === 'version_stack') {
const latestVersion = await prisma.asset.findFirst({
where: { parentId: asset.id, isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
})
if (latestVersion) {
asset = latestVersion
}
}
const mediaInfo = asset.media as unknown as PrismaJson.MediaInfo | null
const proxyType = mediaInfo?.proxyType
const mediaType = asset.mediaType?.toLowerCase() || ''
const filename = asset.name?.toLowerCase() || ''
const bucket = process.env.S3_BUCKET || 'shumai'
// ----------------------------------------------------------------------
// Branch 1: Image Assets
// ----------------------------------------------------------------------
if (proxyType === 'image' || mediaType.startsWith('image/') || filename.endsWith('.psd')) {
let mediaKey = asset.storageKey?.key
if (mediaInfo?.imageTranscodes && mediaInfo.imageTranscodes.length > 0) {
mediaKey = mediaInfo.imageTranscodes[0].key || mediaKey
}
if (!mediaKey) {
throw new Error('No media content found for this asset.')
}
const { annotations } = await resolveAnnotationsById(asset.id, annotationId)
let keyToUse = mediaKey
if (annotations) {
const task = await prisma.workflowTask.create({
data: {
assetId: asset.id,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_image_annotation,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
imageAnnotation: {
annotations,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- transcode image annotation task output
const output = completedTask.output as any
if (output?.key) {
keyToUse = output.key
} else {
throw new Error('Image annotation transcode workflow failed to output S3 key.')
}
}
const { buffer, contentType } = await s3Service.getObject(bucket, keyToUse)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
const downloadUrl = await s3Service.presign(
bucket,
keyToUse,
'GET',
true,
asset.name || undefined,
)
const content: Array<TextContent | ImageContent> = [
{
type: 'text',
text: `Image asset "${asset.name}" (ID: ${asset.id}):\n- S3 Key: "${keyToUse}"`,
},
{
type: 'image',
data: buffer.toString('base64'),
mimeType,
},
]
return {
content,
details: {
assetId: asset.id,
name: asset.name,
mediaType: asset.mediaType,
key: keyToUse,
downloadUrl,
sourceKeys: [keyToUse],
},
}
}
// ----------------------------------------------------------------------
// Branch 2: Video Assets
// ----------------------------------------------------------------------
if (proxyType === 'video' || mediaType.startsWith('video/')) {
if (!videoConfig) {
throw new Error(
'videoConfig with start, end, and count is required when inspecting video assets.',
)
}
const { start, end, count } = videoConfig
if (start < 0 || end < 0) {
throw new Error('Invalid video time range: start and end must be non-negative numbers.')
}
if (start > end) {
throw new Error(
`Invalid video time range: start (${start}) must be less than or equal to end (${end}).`,
)
}
if (count < 1) {
throw new Error(`Invalid frame count: count (${count}) must be at least 1.`)
}
const { annotations, timestamp: commentTimestamp } = await resolveAnnotationsById(
asset.id,
annotationId,
)
const task = await prisma.workflowTask.create({
data: {
assetId: asset.id,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_screenshot,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
screenshot: {
start,
end,
count,
commentTimestamp,
annotations,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- transcode screenshot output
const output = completedTask.output as any
const screenshots = output?.screenshots as
| Array<{ key: string; timestamp: number }>
| undefined
if (!screenshots || screenshots.length === 0) {
throw new Error('No screenshots were generated.')
}
const content: Array<TextContent | ImageContent> = []
const sourceKeys: string[] = []
const results: Array<{ key: string; timestamp: number; downloadUrl: string }> = []
const textLines: string[] = [
`Extracted ${screenshots.length} frame(s) from "${asset.name}" (${start}s - ${end}s):`,
]
for (const shot of screenshots) {
const { buffer, contentType } = await s3Service.getObject(bucket, shot.key)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
const downloadUrl = await s3Service.presign(
bucket,
shot.key,
'GET',
true,
asset.name || undefined,
)
textLines.push(`- Frame at ${shot.timestamp}s: S3 Key: "${shot.key}"`)
content.push({
type: 'image',
data: buffer.toString('base64'),
mimeType,
})
sourceKeys.push(shot.key)
results.push({
key: shot.key,
timestamp: shot.timestamp,
downloadUrl,
})
}
content.unshift({
type: 'text',
text: textLines.join('\n'),
})
return {
content,
details: {
assetId: asset.id,
name: asset.name,
mediaType: asset.mediaType,
keys: sourceKeys,
results,
sourceKeys,
},
}
}
// ----------------------------------------------------------------------
// Branch 3: Documents & PDF Assets
// ----------------------------------------------------------------------
if (
proxyType === 'pdf' ||
mediaType === 'application/pdf' ||
isPlainTextAsset(mediaType, filename)
) {
if (!docConfig) {
throw new Error(
'docConfig with mode ("pages" or "text") is required when inspecting document/PDF assets.',
)
}
if (docConfig.mode === 'text') {
if (annotationId !== null) {
throw new Error(
'annotationId cannot be used with docConfig mode "text" because visual markups can only be rendered on visual pages. Use docConfig: { mode: "pages", startPage: ..., endPage: ... } instead.',
)
}
if (docConfig.startPage !== null || docConfig.endPage !== null) {
throw new Error(
'startPage and endPage must be null when docConfig mode is "text". To inspect specific page numbers, use mode: "pages".',
)
}
if (!isPlainTextAsset(mediaType, filename)) {
throw new Error(
`Asset "${asset.name}" is a binary PDF/document and cannot be read as raw text. Please call read_asset with docConfig: { mode: "pages", startPage: 1, endPage: ... } to view its visual pages.`,
)
}
const mediaKey = asset.storageKey?.key
if (!mediaKey) {
throw new Error(`No media content found for asset ${asset.id}.`)
}
const { buffer } = await s3Service.getObject(bucket, mediaKey)
const rawText = buffer.toString('utf-8')
let text = rawText
let isTruncated = false
const lines = text.split('\n')
if (lines.length > MAX_TEXT_LINES) {
text = lines.slice(0, MAX_TEXT_LINES).join('\n')
isTruncated = true
}
if (Buffer.byteLength(text, 'utf-8') > MAX_TEXT_BYTES) {
text = Buffer.from(text, 'utf-8').subarray(0, MAX_TEXT_BYTES).toString('utf-8')
isTruncated = true
}
if (isTruncated) {
text += `\n\n[Content truncated to ${MAX_TEXT_LINES} lines / ${MAX_TEXT_BYTES / 1024}KB limit. Use download_asset to process the full file.]`
}
const downloadUrl = await s3Service.presign(
bucket,
mediaKey,
'GET',
true,
asset.name || undefined,
)
return {
content: [
{
type: 'text',
text: `Document "${asset.name}" (ID: ${asset.id}, S3 Key: "${mediaKey}"):\n\n${text}`,
},
],
details: {
assetId: asset.id,
name: asset.name,
mediaType: asset.mediaType,
key: mediaKey,
downloadUrl,
sourceKeys: [mediaKey],
size: buffer.length,
isTruncated,
},
}
}
// docConfig.mode === 'pages'
if (docConfig.startPage === null || docConfig.endPage === null) {
throw new Error('startPage and endPage are required when docConfig mode is "pages".')
}
const start = docConfig.startPage
const end = docConfig.endPage
if (start < 1) {
throw new Error(`Invalid page range: start page (${start}) must be at least 1.`)
}
if (start > end) {
throw new Error(
`Invalid page range: start page (${start}) must be less than or equal to end page (${end}).`,
)
}
const MAX_PAGE_RANGE = 20
const requestedPages = end - start + 1
if (requestedPages > MAX_PAGE_RANGE) {
throw new Error(
`Page range (${requestedPages} pages requested) exceeds the maximum limit of ${MAX_PAGE_RANGE} pages per request.`,
)
}
const { annotations, timestamp: commentTimestamp } = await resolveAnnotationsById(
asset.id,
annotationId,
)
const task = await prisma.workflowTask.create({
data: {
assetId: asset.id,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_pdf_pages,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
pdfPages: {
start,
end,
commentTimestamp,
annotations,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- transcode pdf pages output
const output = completedTask.output as any
const pages = output?.pages as Array<{ key: string; page: number }> | undefined
if (!pages || pages.length === 0) {
throw new Error('No PDF page images were generated.')
}
const content: Array<TextContent | ImageContent> = []
const sourceKeys: string[] = []
const results: Array<{ key: string; page: number; downloadUrl: string }> = []
const textLines: string[] = [
`Rendered ${pages.length} page(s) of "${asset.name}" (Pages ${start} to ${end}):`,
]
for (const pageItem of pages) {
const { buffer, contentType } = await s3Service.getObject(bucket, pageItem.key)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
const downloadUrl = await s3Service.presign(
bucket,
pageItem.key,
'GET',
true,
asset.name || undefined,
)
textLines.push(`- Page ${pageItem.page}: S3 Key: "${pageItem.key}"`)
content.push({
type: 'image',
data: buffer.toString('base64'),
mimeType,
})
sourceKeys.push(pageItem.key)
results.push({
key: pageItem.key,
page: pageItem.page,
downloadUrl,
})
}
content.unshift({
type: 'text',
text: textLines.join('\n'),
})
return {
content,
details: {
assetId: asset.id,
name: asset.name,
mediaType: asset.mediaType,
keys: sourceKeys,
results,
sourceKeys,
},
}
}
throw new Error(
`Unsupported asset type for read_asset. Asset ${asset.id} has media type "${mediaType || 'unknown'}". Use download_asset to download this file to disk.`,
)
},
}
}
@@ -1,209 +0,0 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createReadPdfPagesTool } from './read-pdf-pages'
import {
prisma,
WorkflowTaskStatus,
type Asset,
type AssetComment,
type User,
type WorkflowTask,
} from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/db', async (importOriginal) => {
const actual = await importOriginal<typeof import('@shumai/db')>()
return {
...actual,
prisma: {
user: {
findUnique: vi.fn(),
},
asset: {
findUnique: vi.fn(),
},
assetComment: {
findUnique: vi.fn(),
},
workflowTask: {
create: vi.fn(),
},
},
}
})
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
getObject: vi.fn(),
},
}))
vi.mock('@shumai/workflow-core', () => ({
workflowService: {
executeWait: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn(),
},
Permission: { Read: 'Read' },
ResourceType: { Asset: 'asset' },
}))
describe('readPdfPagesTool', () => {
beforeEach(() => {
vi.clearAllMocks()
vi.mocked(authzService.hasPermission).mockResolvedValue()
})
it('should trigger pdfPages transcode workflow and return image outputs', async () => {
vi.mocked(prisma.user.findUnique).mockResolvedValue({ id: 'user-1' } as User)
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
media: { proxyType: 'pdf' },
} as unknown as Asset)
vi.mocked(prisma.assetComment.findUnique).mockResolvedValue({
id: 'comment-1',
assetId: 'asset-1',
second: 2,
annotation: [
{
type: 'box',
color: '#ff0000',
points: [
[0, 0],
[1, 1],
],
},
],
} as unknown as AssetComment)
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-1',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-1',
status: WorkflowTaskStatus.completed,
output: {
pages: [
{ key: 'pdf_pages/page_1.webp', page: 1 },
{ key: 'pdf_pages/page_2.webp', page: 2 },
],
},
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockImplementation(async (_bucket: string, key: string) => {
return {
buffer: Buffer.from(`fake-bytes-for-${key}`),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string }
})
const tool = createReadPdfPagesTool('user-1', 'comment-1')
const result = await tool.execute('call-1', { assetId: 'asset-1', start: 1, end: 2 })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-1',
})
expect(prisma.workflowTask.create).toHaveBeenCalledWith({
data: {
assetId: 'asset-1',
projectId: 'project-1',
type: 'transcode_pdf_pages',
status: 'pending',
payload: {
projectId: 'project-1',
pdfPages: {
start: 1,
end: 2,
commentTimestamp: 2,
annotations: [
{
type: 'box',
color: '#ff0000',
points: [
[0, 0],
[1, 1],
],
},
],
},
},
},
})
expect(result.content.length).toBe(2)
expect(result.content[0].type).toBe('image')
expect((result.content[0] as unknown as { data: string }).data).toBe(
Buffer.from('fake-bytes-for-pdf_pages/page_1.webp').toString('base64'),
)
expect(result.details.sourceKeys).toEqual(['pdf_pages/page_1.webp', 'pdf_pages/page_2.webp'])
})
it('should throw error if start page is less than 1', async () => {
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 0, end: 5 })).rejects.toThrow(
'Invalid page range: start page (0) must be at least 1.',
)
})
it('should throw error if start page is greater than end page', async () => {
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 5, end: 2 })).rejects.toThrow(
'Invalid page range: start page (5) must be less than or equal to end page (2).',
)
})
it('should throw error if page range exceeds maximum limit of 20', async () => {
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 1, end: 25 })).rejects.toThrow(
'Page range (25 pages requested) exceeds the maximum limit of 20 pages per request.',
)
})
it('should throw error if asset not found', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 1, end: 3 })).rejects.toThrow(
'Asset with ID asset-1 not found.',
)
})
it('should throw error if proxyType is not pdf', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
media: { proxyType: 'image' },
} as unknown as Asset)
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 1, end: 3 })).rejects.toThrow(
'Asset asset-1 is not a PDF or document.',
)
})
it('should throw error if proxyType is null/undefined (unprocessed)', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
media: null,
} as unknown as Asset)
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 1, end: 3 })).rejects.toThrow(
'Asset asset-1 is not a PDF or document.',
)
})
})
-142
View File
@@ -1,142 +0,0 @@
import { Type } from '@sinclair/typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { type ImageContent } from '@earendil-works/pi-ai'
import { prisma, WorkflowTaskType, WorkflowTaskStatus, type User } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
const readPdfPagesSchema = Type.Object({
assetId: Type.String({
description: 'Asset ID of the PDF document. This parameter is required.',
}),
start: Type.Number({ description: 'Start page number (1-based index)' }),
end: Type.Number({ description: 'End page number (1-based index)' }),
})
export function createReadPdfPagesTool(
userId: string,
userCommentId?: string | null,
): AgentTool<typeof readPdfPagesSchema> {
return {
name: 'read_pdf_pages',
label: 'Read PDF Pages',
description:
'Converts specific page range of a PDF document into 1080p WebP images for visual analysis. Maximum 20 pages allowed per call.',
parameters: readPdfPagesSchema,
execute: async (_toolCallId, params) => {
if (params.start < 1) {
throw new Error(`Invalid page range: start page (${params.start}) must be at least 1.`)
}
if (params.start > params.end) {
throw new Error(
`Invalid page range: start page (${params.start}) must be less than or equal to end page (${params.end}).`,
)
}
const MAX_PAGE_RANGE = 20
const requestedPages = params.end - params.start + 1
if (requestedPages > MAX_PAGE_RANGE) {
throw new Error(
`Page range (${requestedPages} pages requested) exceeds the maximum limit of ${MAX_PAGE_RANGE} pages per request.`,
)
}
const targetAssetId = params.assetId
if (!userId) {
throw new Error('User ID is required for authorization.')
}
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: targetAssetId,
})
const asset = await prisma.asset.findUnique({
where: { id: targetAssetId },
})
if (!asset) {
throw new Error(`Asset with ID ${targetAssetId} not found.`)
}
const proxyType = (asset.media as PrismaJson.MediaInfo | null)?.proxyType
if (proxyType !== 'pdf') {
throw new Error(`Asset ${targetAssetId} is not a PDF or document.`)
}
// Fetch trigger comment details for comment page and annotations
let commentTimestamp: number | null = null
let annotations: unknown = null
if (userCommentId) {
const comment = await prisma.assetComment.findUnique({
where: { id: userCommentId },
})
if (comment && comment.assetId === targetAssetId) {
commentTimestamp = comment.second !== null ? comment.second : null
if (comment.annotation) {
const list = comment.annotation
if (Array.isArray(list) && list.length > 0) {
annotations = list
}
}
}
}
// Trigger transcode workflow to render PDF pages
const task = await prisma.workflowTask.create({
data: {
assetId: targetAssetId,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_pdf_pages,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
pdfPages: {
start: params.start,
end: params.end,
commentTimestamp,
annotations: annotations as PrismaJson.AnnotationList,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const output = completedTask.output as any
const pages = output?.pages as Array<{ key: string; page: number }> | undefined
if (!pages || pages.length === 0) {
throw new Error('No PDF page images were generated.')
}
const content: ImageContent[] = []
const sourceKeys: string[] = []
const bucket = process.env.S3_BUCKET || 'shumai'
for (const pageItem of pages) {
const { buffer, contentType } = await s3Service.getObject(bucket, pageItem.key)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
content.push({
type: 'image',
data: buffer.toString('base64'),
mimeType,
})
sourceKeys.push(pageItem.key)
}
return {
content,
details: {
sourceKeys,
},
}
},
}
}

Some files were not shown because too many files have changed in this diff Show More