Compare commits

..
76 Commits
Author SHA1 Message Date
yiling 52330869a2 Release v0.4.2 2026-09-03 21:49:37 +08:00
Yiling-J 0c561e9b39 refactor(webui): use semantic system colors in notification settings (#379) 2026-09-03 21:36:28 +08:00
Yiling-J a236cd3a97 feat(provider): add provider and model sync feature with vendored generator (#378)
* feat(provider): vendor and adapt pi model generator

- Vendor generate-models.ts, model-data.ts, and models-dev-reasoning-options.ts from @earendil-works/pi tag v0.82.1
- Adapt imports to use @earendil-works/pi-ai and expose generateModels programmatic function
- Add compare.test.ts to verify vendored generator against original built-in catalog
- Add packages/core/src/provider/generator to eslint ignores

* feat(provider): add provider and model sync api and ui

* feat(webui): add checkbox for existing providers and clean dialog description

* feat(webui): add expand all and collapse all to sync providers dialog

* refactor(provider): remove compare.test.ts and cli overhead from generate-models

* fix(provider): add 30s timeout and strict error handling to sync catalog fetching

* fix(e2e): eliminate drag-and-drop race condition on webkit

Allow requestAnimationFrame collision updates to settle in dragCard helper before releasing pointer, and scope fileCard locator via data-testid to avoid matching sidebar tree items.
2026-09-03 21:01:14 +08:00
Yiling-J c3ef769f51 feat(provider): optimize provider and model configuration management (#377)
* feat(provider): optimize provider and model configuration management

* style(provider): use default tabs variant in provider settings dialog

* style(provider): make tabs list full width

* feat(provider): make model card row clickable and remove pencil edit icon

* feat(settings): persist active settings tab in url hash

* feat(agent): add search box and alphabetical sorting for provider and model selectors

* refactor(providers): remove frontend pagination and display all models in scrollable list

* fix(providers): address code review findings for model editing, i18n, and api protocol propagation
2026-09-03 15:16:16 +08:00
Yiling-J e58dfaa71a feat(agent): consolidate media tools into read_asset and add S3 key support to download_asset (#376)
* feat(agent): consolidate media tools into read_asset and add S3 key support to download_asset

* fix(agent): refine parameter descriptions and remove download URL from read_asset text content

* fix(agent): resolve owning asset and enforce RBAC permissions when downloading by S3 key
2026-09-02 21:51:07 +08:00
Yiling-J 35b57cc0ab feat(agent): support dynamic annotationId resolution across chat and comments (#375)
* feat(agent): improve timestamp precision and proxy selection for video screenshots

* feat(agent): support dynamic annotationId resolution across chat and comments

* fix(agent): scope annotation resolution to asset and format has_markup consistently
2026-09-02 15:17:56 +08:00
Yiling-J 5a754bcb94 feat(agent): improve timestamp precision and proxy selection for video screenshots (#374) 2026-09-01 21:15:28 +08:00
Yiling-J 64b68e7d63 feat(agent): move thread metadata to structured context XML (#373)
* feat(agent): move thread metadata to structured context XML

* feat(core): dynamically inject thread metadata in getSessionEntries
2026-09-01 17:42:56 +08:00
Yiling-J c2f477a391 fix(agent): hide main comment sessions in dashboard sessions list (#372)
* fix(agent): hide main comment sessions in dashboard sessions list

* feat(webui): auto mention comment creator when replying
2026-09-01 16:20:58 +08:00
Yiling-J 9263971653 更新 README_CN.md 以包含 Docker Compose 部署指南
添加了使用 Docker Compose 部署 Shumai 的详细说明和提示词。
2026-09-01 10:25:57 +08:00
Yiling-J 79a479c87c fix(webui): strip styles and preserve multiline text on paste in chat input (#371)
* fix(webui): strip styles and preserve multiline text on paste in chat input

* fix(db): remove custom log wait strategy in test global setup to prevent postgres startup race condition
2026-08-31 21:29:00 +08:00
Yiling-J 134eb8b7fa fix(agent): set empty details in asset tool results to avoid data duplication (#370) 2026-08-31 20:43:05 +08:00
Yiling-J e06478df88 feat(chat): support markups, attachments, and timestamps in agent 1-to-1 chat (#368)
* feat(chat): support markups, attachments, and timestamps in agent 1-to-1 chat

* feat(chat): improve message card layout with inline badges and integrated assets

* fix(chat): enable image attachment thumbnails in 1-to-1 chat cards

* fix(chat): address code review feedback on input state, mentions, i18n, and session deletion

* fix(chat): replace ring with border color on selected message card
2026-08-31 20:11:27 +08:00
yiling 6e9cf8dfb6 Release v0.4.1 2026-08-30 20:26:57 +08:00
Yiling-J d2620f6911 fix(webui): prevent empty folder flash during initial loading (#367) 2026-08-30 18:43:23 +08:00
Yiling-J 4537baa6a2 feat(webui): redesign comment attachment display into compact rows (#366)
* feat(webui): redesign comment attachment display into compact rows

* feat(webui): support pure frontend image extension matching and self-contained preview dialog

* fix(webui): size image preview dialog to min(image size, 80% screen size)

* fix(webui): prevent horizontal overflow of comment attachment rows in scroll areas

* fix(webui): centralize attachment image preview ownership and localize download title
2026-08-30 17:16:20 +08:00
yiling 810e0c4cf5 fix(agent): only include duration_seconds for time-based media (video/audio) 2026-08-30 15:51:23 +08:00
yiling 07cb648183 fix(agent): use total_frames for video and total_pages for pdf in asset context 2026-08-30 15:44:59 +08:00
yiling c6602c9f1c docs(readme): add docker pulls, release, and license badges 2026-08-30 15:32:27 +08:00
Yiling-J 86032539ed feat(agent): refactor message context into unified shumai_message (#365)
* feat(agent): refactor message context into unified shumai_message

* fix(agent): persist custom_message entry type in DatabaseSessionStorage and handle message mapping defensively

* fix(agent): pass clean prompt to harness and replace optimistic messages on shumai_message stream

* fix(agent): align asset context DTO contract, sync comment attachments, and fix session stats
2026-08-30 15:25:13 +08:00
Yiling-J b2ec012e10 feat(webui): add manage versions option to file card context menu (#364) 2026-08-29 19:19:49 +08:00
Yiling-J 7fd8623684 fix(asset,webui): disallow dropping version stacks onto files and version stacks (#363)
* fix(asset,webui): disallow dropping version stacks onto files and version stacks

* fix(webui): derive isSingleFile from total drag selection and remove unused hasVersionStacks
2026-08-29 16:09:36 +08:00
Yiling-J 083753a96d refactor(agent): remove executeAgentToolWorkflow and invoke core services directly (#362)
* refactor(agent): remove executeAgentToolWorkflow and invoke core services directly

* fix(core): ensure creation atomicity, strict assetType filtering, and id-desc listing
2026-08-29 15:19:08 +08:00
yiling 4b62c31f29 Release v0.4.0 2026-08-28 20:40:33 +08:00
Yiling-J 2208d4e0e0 fix(webui): align mobile comment cards and thread vertical indicator (#361) 2026-08-28 20:26:26 +08:00
Yiling-J df7f4c3c2d Update README_CN with project management details
Added information about project management and media handling in Shumai.
2026-08-28 20:25:46 +08:00
Yiling-J 12323e1416 Enhance README with project management and media details
Added information about project management features and media processing in Shumai.
2026-08-28 20:23:38 +08:00
yiling 2edc5e1071 fix(e2e): enable ryuk container reaper for app e2e tests 2026-08-28 16:22:29 +08:00
Yiling-J 2162ed9ebe fix(webui): fix breadcrumb file dropdown trigger stretching and alignment (#360) 2026-08-28 16:07:27 +08:00
Yiling-J 6acebb73ec feat(webui): add mobile friendly file detail and public share view (#359)
* feat(webui): add mobile friendly file detail and public share view

* fix(webui): address review feedback for mobile file detail and public share view

* fix(webui): use dynamic viewport and flex column layout for mobile file viewer

* fix(webui): support iOS WebKit video fullscreen and guard requestFullscreen

* fix(webui): compact page indicator and layout in mobile PDF control bar
2026-08-28 15:11:33 +08:00
EICCIO Advisors 7a1d6c0ab5 docs: add SECURITY.md with a private disclosure path (#357)
The project has no security policy, so a researcher with a finding today has
only a public issue or Discord, both of which disclose before a fix exists.

Routes reports to GitHub private vulnerability reporting. Names deployment
manifests as in scope, since the documented quickstart is what most operators
actually run. Adds operator guidance for the three things a self-hoster owns:
the session secret, not publishing the database port, and treating the agent
as a code-execution surface.

Response timelines are left for the maintainer to set.
2026-08-28 09:21:21 +08:00
Yiling-J d16e31c0e5 feat(webui): add simplified mobile-friendly layout for file and share lists (#356)
* feat(webui): add simplified mobile-friendly layout for file and share lists

* refactor(webui): switch mobile file list layout to 1-column grid

* feat(webui): show foldertree by default in child sidebar and add close button

* feat(webui): add collapsible section headers and fix folder card full-width on mobile

* feat(webui): improve mobile breadcrumb bar with middle truncation and dual ellipsis

* fix(webui): unify breadcrumb DOM structure across desktop and mobile

* fix(webui): remove unused breadcrumbs variable in BreadcrumbNav

* fix(webui): complete and confirm file uploads on mobile and handle errors properly

* fix(webui): remove active scale animation on mobile FAB to stabilize dropdown menu anchor

* refactor(webui): use PresignedUrl type from dtos for uploadFiles and eliminate any cast

* fix(webui): scope folder card max width to desktop with responsive md:max-w-[340px]
2026-08-27 20:44:08 +08:00
Yiling-J d9a8b6a191 feat(webui): truncate long file names with 100ms hover tooltip (#355) 2026-08-27 10:54:35 +08:00
yiling 5ec4cd7fcd docs: update documentation link in README and README_CN 2026-08-26 21:44:50 +08:00
Yiling-J a7925e3c79 fix(share): hide and soft-delete symlinks when target assets are trashed (#354) 2026-08-26 21:04:38 +08:00
Yiling-J 472b420620 fix(transcode): apply team hardware acceleration and source bitrate capping to watermark transcoding (#353) 2026-08-26 17:00:11 +08:00
Yiling-J 8f55ba1d37 feat(transcode): add hardware-accelerated video transcoding support (#352)
* feat(transcode): add hardware-accelerated video transcoding support

* feat(transcode): use encoder quality modes and source-capped maxrate for video transcode

* fix(transcode): scale max bitrate for downsampled preview framerates

* fix(transcode): propagate videoBitRate in getMediaInfoActivity and update auto description
2026-08-26 16:28:14 +08:00
yiling 7b1428bf51 Release v0.3.11 2026-08-25 22:05:47 +08:00
yiling d5b3b1868e fix(webui): reload video player and reset state when switching videos in viewer 2026-08-25 21:52:15 +08:00
yiling 49f3fc0be5 fix(webui): restore full width bottom bar spanning across carousel in file viewer 2026-08-25 21:27:20 +08:00
Yiling-J 59e641ccb3 fix(webui): prevent left carousel remounting and scroll jump on file selection (#351) 2026-08-25 21:01:51 +08:00
Yiling-J 1d9268b224 feat(project): add recent viewed files section to project folder tree (#348)
* feat(project): add recent viewed files section to project folder tree

* feat(webui): disable toolbar buttons and context menu in recents page

* test(core): optimize recents cap test execution time

* feat(webui): show hint text in recents page top toolbar

* feat(webui): rename recents section to Recently Viewed

* refactor(core): consolidate recents methods into AssetService

* fix(api): forward pagination parameters in folder children endpoint (#349)

* fix(project-recents): scope views and disable recents mutations

* fix(api): allow recent views in demo mode

* fix(api): limit demo recent view exception to post
2026-08-25 18:51:24 +08:00
Yiling-J 470a4bd6f7 feat(auth): add admin password reset CLI command and reset password web UI (#347)
* feat(auth): add admin password reset CLI command and reset password web UI

* fix(auth): invalidate previous reset tokens and use AsyncLocalStorage for link capture
2026-08-25 14:35:33 +08:00
yiling 8801d59b2e docs: symlink CLAUDE.md to AGENTS.md 2026-08-25 09:50:54 +08:00
yiling 6a20e06982 Release v0.3.10 2026-08-23 21:19:45 +08:00
Yiling-J b17abce2c2 fix(kanban): improve drag-drop to column bottom, remove empty drop line, and fix timeline alignment (#346) 2026-08-23 21:11:33 +08:00
Yiling-J f67e334016 feat(kanban): update task status permissions, reporter defaults, and participant access (#345)
* feat(kanban): allow editors to change task status and default reporter to creator

* fix(kanban): hide create task button for reviewers and eliminate duplicate error toast
2026-08-23 20:38:56 +08:00
yiling c1de66520e update readme image 2026-08-23 16:53:44 +08:00
Yiling-J 37e086d822 fix(webui): sidebar selection styling, kanban active state, pointer cursors, and owner dashboard (#344)
* fix(webui): update sidebar selection styling, kanban active state, and pointer cursors

* feat(webui): move dashboard to left sidebar and prevent mount animation

* fix(e2e): update create-project test to click Home button in sidebar
2026-08-23 16:39:56 +08:00
Yiling-J d21f8ca8de fix(kanban): use true thumbnail and poster for task asset previews (#343) 2026-08-23 14:36:46 +08:00
Yiling-J 6af4e7c2ec feat(kanban): resolve latest version for linked version stack assets and update demo login text (#342) 2026-08-23 11:19:48 +08:00
Yiling-J 1f2bdc8c9e feat(agentic kanban): phase 1 not-agentic kanban board (#330)
* feat(kanban): implement AI-native kanban database models, services, and REST APIs

* feat(webui): implement team kanban board and task management

* style(webui): increase kanban dialog widths and form container

* fix(webui): ensure kanban dialogs override sm:max-w-lg base class

* style(webui): set edit task dialog max width to 5xl

* chore: untrack kanban plan markdown files

* feat(webui): update kanban page layout to 2 columns and refine task activity UI

* fix(webui): constrain kanban column and board heights for vertical scrolling

* feat(kanban): unify task status transition into updateTask API and allow unconstrained human task transitions

* fix(webui): add optimistic updates to kanban drag-and-drop to eliminate flash

* refactor(webui): remove client-side search box from kanban header

* feat(kanban): redesign task dialog comments area and add attachment support

* refactor(webui): remove top status action bar from task info dialog

* fix(webui): narrow query cache predicate and add defensive guards for kanban drag optimistic update

* feat(kanban): add drag-and-drop task reordering with fractional indexing (#331)

* feat(kanban): add drag-and-drop task reordering with fractional indexing

* fix(kanban): improve card drop zones, snap indicators, and column highlight

* fix(kanban): enable card droppables only during active drag

* fix(kanban): refine hover styles and drop indicators for column boundaries

* fix(kanban): unify bottom drop indicator and fix top indicator clipping

* fix(kanban): ensure full column highlight and full coverage drop target

* minor change

* feat(kanban): allow agent tasks free drag-and-drop and enforce status change permissions (#332)

* refactor(kanban): rename task_type_agentic translation keys to task_type_agent

* feat(kanban): refactor task type enum to isAgentTask boolean (#333)

* refactor(kanban): remove phase 2 agent execution logic and streamline phase 1 kanban

* feat(kanban): add notifications and audit logging for kanban (#334)

* feat(kanban): redesign config area layout and add parent tasks multi-selector (#335)

* feat(webui): redesign kanban task dialog config area

* feat(kanban): simplify task dependencies with parent tasks multi-selector

* refactor(kanban): remove unused links routes and legacy dependency methods

* feat(kanban): connect assets and kanban tasks with many-to-many relationship (#336)

* feat(kanban): connect assets and kanban tasks with many-to-many relationship

* feat(kanban): improve asset picker layout, infinite scrolling, and breadcrumb icon styling

* fix(kanban): ensure asset picker scrollarea is scrollable with min-h-0 and shrink-0 flex styling

* fix(kanban): hide recently deleted in folder tree picker and adjust header padding

* fix(kanban): use lucide files icon instead of paperclip for linked assets

* feat(kanban): improve task deletion, goal sidebar interaction, and card styling (#337)

* feat(kanban): improve task deletion, goal sidebar interaction, and card styling

* feat(kanban): move agent task badge after priority and update to orange style

* feat(kanban): add status Compare-And-Swap (CAS) to prevent drag-and-drop race conditions (#338)

* feat(kanban): add status Compare-And-Swap (CAS) to prevent drag-and-drop race conditions

* style(kanban): rotate kanban icon 180 degrees in sidebar and top nav

* style(kanban): remove rotate-180 from SquareKanban icons

* feat(kanban): add unassigned goal record and fix all tasks count (#339)

* feat(kanban): add unassigned goal record and fix all tasks count

* feat(kanban): toggle task count with three-dot action button on goal card hover

* fix(ci): pin bun version to 1.3.14 across CI and Docker (#340)

* fix(webui): truncate goal name to max 2 lines in kanban sidebar (#341)
2026-08-23 10:28:56 +08:00
yiling 4c0fd29b7f Release v0.3.9 2026-08-18 18:42:59 +08:00
Yiling-J ffef67be8f feat(quota): move usage monitoring to team dashboard (#327) 2026-08-18 15:57:36 +08:00
Yiling-J 4d100bf542 docs(team-settings): add resource quotas documentation and update readme features (#326)
* docs(team-settings): add resource quotas documentation and update readme features

* fix(webui): correct sandbox filesystem description
2026-08-18 14:48:12 +08:00
Yiling-J 8a75d015a1 fix(quota): exclude agents from quota rules and usage records (#325)
* fix(quota): exclude agents from quota rules and usage records

* refactor(quota): remove agent_skill_call_count resource type and related logic

* chore(db): replace manual migration with prisma-generated migration
2026-08-18 11:56:03 +08:00
Yiling-J f806272c06 feat(settings): reorganize settings into personal, team, and AI sections (#324) 2026-08-18 08:35:23 +08:00
Yiling-J 4935eeff28 fix(webui): preserve selected chatbot agent preference on reload (#323) 2026-08-17 20:47:57 +08:00
Yiling-J eed0264b9c feat(quota): add 2-level database-backed resource quota system (#318)
* feat(quota): add database-backed resource quota system

* feat(webui): add resource quota management in team settings (#319)

* feat(webui): add resource quota settings in team management

* refactor(webui): update quotas list to full row layout and restrict skill/mcp to select list

* feat(quota): refactor quota system to 2-level rules and records

* fix(webui): fix overflow and scroll area in quota usage records dialog

* fix(webui): ensure quota records scroll area has explicit max height

* feat(webui): improve quota cards interaction and restrict editable fields in edit mode

* fix(quota): serialize initial records, validate merged updates, and reset on dimension changes (#320)

* feat(quota): replace network quota with agent tools quota (#321)

* feat(webui): show remaining time until next refresh in quota usage records

* fix(webui): rename bash quota resource to Agent Bash Calls

* refactor(quota): remove unused reserved field from quota records

* refactor(quota): remove quota rule cache (#322)
2026-08-17 20:24:05 +08:00
Yiling-J dd82daf65c feat(webui): use project members for asset comment mentions (#315)
* feat(webui): use project members for asset comment mentions

* refactor(project): remove legacy unused bots endpoint and props
2026-08-16 16:44:50 +08:00
Yiling-J 55d9596835 feat(authz): enforce project-aware roles for agent/skill/mcp permissions (#314)
* feat(authz): enforce project-aware roles for agent/skill/mcp permissions

Introduce a shared resolveEffectiveRole helper that mirrors AuthzService
precedence rules (projectMember.role override -> team role for team-scoped
members -> restricted), and route all agent/skill/mcp runtime permission
checks through it. Previously these checks used the raw teamMember.role,
which let project-only members execute team-level agents/skills/MCP tools
and ignored per-project role overrides.

Changes:
- authz: add resolveEffectiveRole(teamId, projectId, userId, db) and
  refactor hasPermission to use it (behavior-preserving)
- chat/asset/agent-activity: gate agent usage by the effective project role
- read_skill: enforce skill permission via effective role (projectId
  threaded from the agent session)
- mcp-service: checkPermission and buildAgentTools filter by effective
  role; un-permitted servers' tools are no longer advertised to the model
- agent activity: only advertise skills the user's effective role permits
- listProjectMembers: filter included agent bots by the requester's
  effective project role
- new GET /projects/:projectId/chat-agents API returning chat agents
  scoped to the project and the user's effective role; WebUI chat selector
  now uses it (settings keeps the team-level API)
- tests: effective-role resolver, chat/mention/MCP/skill project-role
  matrices, listProjectChatAgents, listProjectMembers filter, API route

* fix(agent): forward projectId to getAgentChatContextActivity in agentChat workflow

The agentChat workflow called getAgentChatContextActivity without passing
projectId from the task payload, so project-scoped members (scope:
'project') resolved a null effective role in the activity and were denied
with 'Insufficient role to use agent' even though the chat selector and
chat.ts both authorized them. Forward payload.projectId so the activity
enforces the user's effective project role, and assert the forwarding in
the workflow unit test (fails without the fix).
2026-08-16 15:44:45 +08:00
Yiling-J cb1c1c502b feat(agent): add permission controls for chat agents (#313)
* feat(agent): add permission controls for chat agents

* refactor(agent): extract shared agent permission helpers

Consolidate the duplicated role-hierarchy logic into
packages/core/src/agent/permissions.ts:

- getRoleLevel(): numeric level for a team role (fail closed at 0)
- getAgentRequiredLevel(): required level for an agent permission
  (defaults to reviewer/1)
- getAllowedAgentRoles(): inclusive role set for filtering

The typed Record over TeamMemberRole makes the mapping exhaustive,
so adding a new role fails compilation until all sites are updated.
Used by ChatService.startOrContinueChat, AssetService comment
mentions, getAgentChatContextActivity, AgentService.listAgents, and
TeamService.getTeamMembers, replacing four inline copies of the
hierarchy and the allowed-roles conditionals.

Also fix the API permission-denial test to mock HTTPException(403)
and assert a 403 response instead of a generic Error/500, matching
real authzService behavior.

* test(agent): fix flaky comment DAG tests by spacing ULID id generation

ULID ids have 1ms timestamp granularity; back-to-back comment creates
frequently land in the same millisecond, in which case Prisma ULIDs sort
by their random suffix instead of creation order. The DAG spine in
initializeAgentSessionActivity chains top-level comments by id, so a
shuffled order produces paths that omit preceding comments, making the
path-to-root assertions flaky (~80% of full-file runs locally).

Add a 2ms sleep between consecutive comment creates in the affected
tests, guaranteeing strictly increasing ULID timestamps. A 2ms delay is
sufficient because the next create always fires at least 2ms after the
previous ULID was generated. Production is unaffected: real comments
arrive seconds apart via separate requests.
2026-08-16 11:16:14 +08:00
yiling 6d4f62d6f6 Release v0.3.8 2026-08-15 20:35:30 +08:00
Yiling-J 43808216e6 fix(core): rename latest version child record on version stack rename (#312) 2026-08-15 16:18:11 +08:00
Yiling-J f66bb7af73 feat(version-stack): add manage versions dialog and version removal (#311)
* feat(version-stack): add manage versions dialog and version removal

* fix(webui): remove ellipsis and add pointer cursor to manage versions button

* fix(version-stack): handle symlink deduplication and name sync for share links

* refactor(version-stack): extract shared symlink dedup helper

The 'repoint first symlink per share folder, delete duplicates with
fileCount decrement' logic was triplicated across createVersionStack,
reparentAssets, and reparentFileToFile with subtle variations. Extract
it into dedupeSymlinksToTarget() in packages/core/src/asset/symlink.ts
and call it from all three sites (behavior unchanged), with a dedicated
test suite covering the edge cases.
2026-08-15 15:37:17 +08:00
Yiling-J ae86d56a7d feat(webui): improve AGENTS.md editor placeholder and read-only notice (#310)
* feat(webui): improve AGENTS.md editor placeholder and read-only notice

* docs: add AGENTS.md documentation under Shumai Agent Settings
2026-08-15 11:00:30 +08:00
Yiling-J 1cee103b1c feat(agent): add nested agents.md support across backend and webui (#309)
* feat(agent): add nested agents.md support across backend and webui

* fix(webui): update agentsmd route paths and unnest folder agentsmd route

* fix(agent): flush agents.md autosave on close and address review feedback

- Flush pending debounced AGENTS.md autosave on close/unmount so edits
  made within 750ms of navigating away are no longer silently lost
- Serialize autosaves through a promise chain so an older in-flight
  request can never overwrite newer content on the server
- Internationalize the hardcoded "Failed to save" and
  "Loading AGENTS.md..." strings (en + zh)
- Validate that AGENTS.md is only stored on folder/root assets in
  AssetService, preventing bogus virtual paths for file assets
- Cap updateAgentsMd content at 100k chars via Zod validation
- Remove unused getAgentsMd/updateAgentsMd response DTO schemas
- Drop unused teamId prop from AgentsMdEditor
- Add tests: non-folder asset rejection, oversized content 400,
  and agent-chat workflow assetId passthrough

* refactor(db): isolate AGENTS.md content in dedicated 1:1 AssetAgentMd table

* style(webui): unify top bar button hover styles and add wand icon to agentsmd button

* style(webui): update agentsmd button state colors and remove presence dot

* fix(webui): fix close button overflow and stabilize debounced autosave in agentsmd editor

* fix(webui): update debounce to 1s and await pending save before closing agentsmd editor

* feat(webui): add terminal AGENTS.md breadcrumb on agentsmd editor page

* fix(webui): use ScrollArea for markdown editor scrolling
2026-08-15 10:25:46 +08:00
Yiling-J 3c3b6880e4 feat(webui): replace legacy markdown editor with new lexical wysiwyg editor (#308)
* feat(webui): replace legacy markdown editor with new lexical wysiwyg editor

* refactor(webui): organize markdown editor files into components/markdown-editor folder
2026-08-14 16:27:14 +08:00
yiling 0b84402c83 Release v0.3.7 2026-08-14 14:47:18 +08:00
Yiling-J 61a61abc7c fix(webui): fix select field height and multi-select overflow/interaction issues (#307)
* fix(webui): standardize single-select custom field height

* fix(webui): fix multi-select badge overflow calculation and simplify direct edit
2026-08-14 14:35:17 +08:00
Yiling-J fc878775c2 feat(agent): support dynamic newOption in select and selectMulti metadata fields (#306)
* feat(webui): redesign select and multi-select metadata fields with search box and dynamic option creation

* fix(webui): stabilize select popover placement and preserve field visibility in store

* refactor(webui): enforce 2-row collapsed height and whitespace-pre-wrap on LongTextField

* feat(agent): support dynamic newOption in select and selectMulti metadata fields

* fix(metadata): address code review feedback on selectMulti item validation and WebUI option deduplication
2026-08-14 10:56:18 +08:00
Yiling-J 7acf736300 feat(webui): redesign select metadata fields and improve long text field widget (#305)
* feat(webui): redesign select and multi-select metadata fields with search box and dynamic option creation

* fix(webui): stabilize select popover placement and preserve field visibility in store

* refactor(webui): enforce 2-row collapsed height and whitespace-pre-wrap on LongTextField
2026-08-13 22:46:40 +08:00
Yiling-J 9d50301a43 feat(sandbox): default Network Sandbox to off and add settings toggle switch (#304) 2026-08-13 20:50:34 +08:00
Yiling-J 7f185ca23c feat(agent): type-safe CREATION_CONTEXT metadata on create_file/create_version (#303)
* feat(agent): type-safe CREATION_CONTEXT metadata on create_file/create_version

Build the create_file and create_version tool metadata parameter from the
project's CREATION_CONTEXT fields at harness creation time, instead of the
schemaless key-value map that required the agent to call list_autofill_fields
first.

- Thread projectId from the chat workflow into createAgentSession and build
  a strict, all-required (nullable) metadata schema via fieldsToTypeBoxSchema
  (additionalProperties: false, option IDs enforced via enum).
- Remove the list_autofill_fields tool and its activity case.
- Reject metadata keys that are not CREATION_CONTEXT fields of the target
  project with a clear non-retryable error telling the agent to omit metadata.
- Skip null metadata values when persisting (create_file/create_version and
  the autofill workflow).
- Switch the touched schemas from @sinclair/typebox 0.34 to typebox v1 to
  match the harness validator.

* refactor(agent): migrate remaining tools from @sinclair/typebox 0.34 to typebox v1

All agent tool schemas now use the same typebox v1 package as the harness
validator; drop the @sinclair/typebox 0.34 dependency from @shumai/agent.

* test(agent): reproduce null coercion in harness tool validation

The pi-ai harness (<=0.82.1) coerces null values in [string, null] unions to
'' during tool argument validation for typebox v1 schemas. A valid create_file
call with path: null + data: {...} therefore arrives at execute with path: ''
and fails with 'Provide exactly one of path or data'. Null metadata values
(the 'unknown' sentinel) are likewise turned into ''.

This test reproduces both failures. It is expected to fail until
@earendil-works/pi-ai is upgraded to >=0.84.0 (#7328), which preserves values
that already match a union arm.

* fix(agent): patch pi-ai 0.82.1 to preserve null in nullable union tool args

Apply the upstream fix for earendil-works/pi#7328 (PR #7373) via a bun
patch instead of upgrading pi-ai. coerceWithUnionSchema now returns the
original value when it already validates against any union member,
instead of coercing null in [string, null] unions into "".

Fixes the create_file false "Provide exactly one of path or data" error
when the model passes path: null with data. Reproduced by
packages/agent/src/reproduce-tool-null-coercion.test.ts, which now passes.

* fix(docker): copy bun patch dir before frozen-lockfile install

The pi-ai patchedDependencies entry makes bun install --frozen-lockfile
fail in the builder stage because patches/ was not present in the image
yet. Copy it alongside package.json/bun.lock so the patch applies.

* refactor(agent): simplify metadata guidance in system prompt and error message

Shorten the create_file/create_version metadata instructions in the system
prompt and reword the invalid-metadata-key error to not reference
CREATION_CONTEXT internals. Update the matching test assertion.
2026-08-13 20:06:09 +08:00
yiling 94582c9001 try fixing aliyun docker push 2026-08-13 15:06:44 +08:00
Yiling-J 41b73feeab feat(metadata): replace aiAutofill boolean with autofillSource enum (#302)
* feat(metadata): replace aiAutofill boolean with autofillSource enum

* fix(agent): validate creation-context metadata and make writes transactional

Address code review feedback on the autofillSource migration:

- Update the agent system prompt to reference the 'metadata' parameter of
  create_file/create_version instead of the removed 'context' parameter,
  which was being silently discarded by the activity
- Write metadata inside the caller's transaction via the new
  metadataService.updateAssetMetadataInTx, keeping metadata rows atomic
  with asset creation instead of opening a separate non-atomic transaction
- Reject metadata keys that are not defined CREATION_CONTEXT fields of the
  project with a non-retryable ApplicationFailure before creating the asset
- Remove dead agentContext handling from autofillAiActivity and its tests,
  and drop the unused ai_autofill i18n keys (paraglide recompiled)
- Derive the DTO autofillSource enum from the AutofillSource const instead
  of a hardcoded z.enum to prevent drift
2026-08-13 13:41:46 +08:00
1104 changed files with 91557 additions and 7656 deletions
+8 -8
View File
@@ -22,9 +22,9 @@ jobs:
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
bun-version: 1.3.14
- name: Install dependencies
run: bun install
@@ -76,9 +76,9 @@ jobs:
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
bun-version: 1.3.14
- name: Install dependencies
run: bun install
@@ -118,9 +118,9 @@ jobs:
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
bun-version: 1.3.14
- name: Install dependencies
run: bun install
@@ -152,9 +152,9 @@ jobs:
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
bun-version: 1.3.14
- name: Install dependencies
run: bun install
+1 -1
View File
@@ -36,7 +36,7 @@ jobs:
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
bun-version: 1.3.14
- name: Setup Node.js
uses: actions/setup-node@v6
+1
View File
@@ -54,5 +54,6 @@ jobs:
platforms: linux/amd64,linux/arm64
context: .
push: true
provenance: false
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
+1
View File
@@ -11,6 +11,7 @@ data/
*.db
*.diff
*.orig
code_quality.md
.agents
# Playwright
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+8 -2
View File
@@ -1,6 +1,6 @@
#syntax=docker/dockerfile:1.7-labs
# Stage 1: Builder
FROM oven/bun:1 AS builder
FROM oven/bun:1.3.14 AS builder
WORKDIR /app
@@ -8,6 +8,7 @@ WORKDIR /app
COPY package.json bun.lock ./
COPY --parents packages/*/package.json ./
COPY --parents apps/*/package.json ./
COPY patches/ ./patches/
# Install dependencies (including devDependencies for build tools)
RUN bun install --frozen-lockfile
@@ -24,7 +25,7 @@ RUN bun run build.ts
RUN bun run scripts/extract-runtime-deps.ts
# Stage 2: Runner
FROM oven/bun:1 AS runner
FROM oven/bun:1.3.14 AS runner
# Install ffmpeg/ffprobe
COPY --from=mwader/static-ffmpeg:8.1 /ffmpeg /usr/local/bin/
@@ -79,6 +80,11 @@ USER root
COPY scripts/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
# Expose "bun index.js" as "shumai" CLI command (and alias "shuami")
RUN printf '#!/bin/sh\nexec bun /app/index.js "$@"\n' > /usr/local/bin/shumai \
&& chmod +x /usr/local/bin/shumai \
&& ln -s /usr/local/bin/shumai /usr/local/bin/shuami
ENTRYPOINT ["docker-entrypoint.sh"]
# Default command: run migrations and start API server
+14 -1
View File
@@ -9,7 +9,13 @@
</p>
<p align="center">
<a target="_blank" href="https://shumai.one/">Website</a> | <a target="_blank" href="https://docs.shumai.one/introduction">Docs</a> | <a target="_blank" href="https://staging.shumai.one">Demo</a> | <a target="_blank" href="https://discord.gg/sUzhujK6DD">Discord</a> | <a target="_blank" href="README_CN.md">中文</a>
<a href="https://hub.docker.com/r/shumaione/shumai"><img alt="Docker pulls" src="https://img.shields.io/docker/pulls/shumaione/shumai" /></a>
<a href="https://github.com/shumaiOne/shumai/releases"><img alt="GitHub latest release" src="https://img.shields.io/github/v/release/shumaiOne/shumai" /></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-MIT-blue" /></a>
</p>
<p align="center">
<a target="_blank" href="https://shumai.one/">Website</a> | <a target="_blank" href="https://docs.shumai.one/getting-started/overview">Docs</a> | <a target="_blank" href="https://staging.shumai.one">Demo</a> | <a target="_blank" href="https://discord.gg/sUzhujK6DD">Discord</a> | <a target="_blank" href="README_CN.md">中文</a>
</p>
![Shumai App Screenshot](docs/screenshot.webp)
@@ -37,6 +43,7 @@ But Shumai goes beyond being a Frame.io alternative. With modern models like GPT
* **Isolated Sandbox Execution**: Run agent-submitted scripts safely in a sandboxed environment.
* **AI Metadata Autofill**: Automatically fill in custom metadata for new assets.
* **Semantic Search**: Find assets by visual or conceptual meaning using vector embeddings (Gemini Embedding 2).
* **Resource Quotas**: Set usage limits on AI tokens, costs, and tool execution per member, role, or team-wide, with real-time tracking.
Here is a short demo video showing what the Shumai agent can do:
@@ -44,6 +51,12 @@ Here is a short demo video showing what the Shumai agent can do:
https://github.com/user-attachments/assets/1186067a-72c0-41b9-836f-987438f2332a
Shumai also brings project management into the creative workflow. A built-in Kanban board helps teams organize tasks, track progress, and collaborate in one place.
The ultimate goal is an **agentic Kanban**, where agents and team members can work together on tasks. This is still a **work in progress**: you can already create tasks for AI agents, but agents cannot act on those tasks yet.
<img width="2880" height="2160" alt="localhost_3000_teams_01M0HQR3EEBS0B46K4FK39X2C8_kanban(1080p-2) (1)" src="https://github.com/user-attachments/assets/cd0648d0-c69a-4398-958b-8d15a33378b0" />
> [!NOTE]
+32 -1
View File
@@ -9,8 +9,13 @@
</p>
<p align="center">
<a target="_blank" href="https://shumai.one/">官方网站</a> | <a target="_blank" href="https://docs.shumai.one/introduction">文档</a> | <a target="_blank" href="https://staging.shumai.one">在线演示</a> | <a target="_blank" href="README.md">English</a>
<a href="https://hub.docker.com/r/shumaione/shumai"><img alt="Docker pulls" src="https://img.shields.io/docker/pulls/shumaione/shumai" /></a>
<a href="https://github.com/shumaiOne/shumai/releases"><img alt="GitHub latest release" src="https://img.shields.io/github/v/release/shumaiOne/shumai" /></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-MIT-blue" /></a>
</p>
<p align="center">
<a target="_blank" href="https://shumai.one/">官方网站</a> | <a target="_blank" href="https://docs.shumai.one/getting-started/overview">文档</a> | <a target="_blank" href="https://staging.shumai.one">在线演示</a> | <a target="_blank" href="README.md">English</a>
</p>
![Shumai 应用截图](docs/screenshot.webp)
@@ -38,11 +43,19 @@ Shumai 是一个开源的 Frame.io 替代方案,专为创意团队打造,让
* **沙箱隔离执行**:Agent 生成的脚本会在隔离环境里运行,更安全。
* **AI 自动补全元数据**:新素材上传后,系统可以自动帮你补一些元数据。
* **语义搜索**:支持按画面内容或语义找素材,底层用的是向量 embedding(Gemini Embedding 2)。
* **资源配额**:支持按成员、角色或全团队对 AI Token、成本和工具调用设置使用上限与实时用量监控。
下面是一个简短的 demo 视频,展示了 Shumai Agent 能做什么:
https://github.com/user-attachments/assets/1186067a-72c0-41b9-836f-987438f2332a
Shumai 也将项目管理融入了创意工作流程。 内置的看板可以帮助团队整理任务以及跟踪进度。
最终目标是打造一个 Agentic看板,让 Agent 和团队成员能够围绕同一个目标协同工作。目前这一功能仍在开发中: 你已经可以在看板中为 Agent 创建任务,但 Agent 目前还无法自动执行这些任务。
<img width="2880" height="2160" alt="localhost_3000_teams_01M0HQR3EEBS0B46K4FK39X2C8_kanban(1080p-2) (1)" src="https://github.com/user-attachments/assets/29ff1db1-2d65-4e3c-b467-754b777b406f" />
> [!NOTE]
> **媒体处理与存储机制:** 上传媒体文件后,Shumai 始终会保留并存储原始文件。请注意,系统中显示的文件大小为原始文件的大小,不包含转码后生成的代理文件大小。无论图片还是视频,都会额外生成一张 300p 的 WebP 封面图,作为文件列表中的预览图。此外:
>
@@ -56,6 +69,24 @@ https://github.com/user-attachments/assets/1186067a-72c0-41b9-836f-987438f2332a
以下是使用本地存储运行 Shumai 的快速上手指南。如需了解高级配置(包括 S3 兼容存储配置、Temporal 工作流编排等),请参阅我们的[官方文档](https://docs.shumai.one/getting-started/overview)。
### 方案一:使用 Docker Compose(推荐)
<details>
<summary>复制这个提示词,让agent帮你安装</summary>
```
请帮我在本机使用 Docker Compose 一键部署 Shumai。
请严格参考官方文档:
https://gh-proxy.com/https://raw.githubusercontent.com/shumaiOne/shumai/main/README_CN.md
要求:
1. 只能使用 Docker Compose local方式部署,不要使用其他部署方式。
2. 部署前请先检查本机是否已安装并可正常使用 Docker 和 Docker Compose。
3. 如果本机未安装 Docker,请立即提醒用户先安装 Docker,并停止后续部署操作。不要自行安装 Docker,也不要采用其他方式部署。
4. 获取 `docker-compose-aliyun.yaml` 文件内容时,必须通过 `https://gh-proxy.com` 加速访问,不要直接访问 GitHub Raw 地址。
```
</details>
使用 Docker Compose 是运行 Shumai 最快的方式。您无需克隆仓库,也不需要手动安装依赖包。请确保电脑上已安装 [Docker](https://docs.docker.com/get-docker/) 和 [Docker Compose](https://docs.docker.com/compose/install/),然后按照以下步骤操作:
+58
View File
@@ -0,0 +1,58 @@
# Security Policy
## Reporting a Vulnerability
Please report security vulnerabilities **privately**. Do not open a public issue,
pull request or Discord message for a suspected vulnerability, as that discloses it
to everyone before a fix exists.
Use **[GitHub private vulnerability reporting](https://github.com/shumaiOne/shumai/security/advisories/new)**
(the "Report a vulnerability" button on the Security tab). If that is unavailable,
contact the maintainer directly rather than in a public channel.
Please include:
- What the issue is, and the impact you believe it has
- The version or commit you tested (`git rev-parse HEAD`, or the image digest)
- Steps to reproduce, ideally minimal
- Your deployment shape: storage backend, whether the agent feature is enabled,
and whether the instance is internet-reachable
## Supported Versions
Shumai is pre-1.0 and moves quickly. Security fixes are made against the **latest
release**. Users are encouraged to track releases rather than pin indefinitely.
## Scope
**In scope** — the application, its API, the shipped Docker images, and the
deployment manifests under `docker-compose/`.
Deployment manifests are explicitly in scope. A default that is unsafe when
followed as documented is a real finding, because the documented quickstart is what
most operators actually run.
**Out of scope**
- Vulnerabilities in third-party dependencies, unless Shumai's use of them is what
creates the exposure. Report those upstream.
- Findings that require an already-compromised host or database.
- Configurations that deviate from the documented deployment.
## Operator guidance
Shumai is self-hosted, so several controls belong to the operator rather than to the
project:
- **Set `BETTER_AUTH_SECRET` per deployment.** It signs sessions and, in local
storage mode, upload URLs. Never reuse a value from documentation or an example.
- **Do not expose the instance to the public internet without a reverse proxy**
terminating TLS and applying your own access policy.
- **Do not publish the database port.** Set a real database password.
- **Treat the agent feature as a code-execution surface.** If you enable it, run it
where a compromise is survivable, and review the sandbox network policy.
## Disclosure
Please allow a reasonable period for a fix before public disclosure. The maintainer
will acknowledge reports and keep reporters informed of progress.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent-app",
"version": "0.3.6",
"version": "0.4.2",
"private": true,
"type": "module",
"bin": {
-1
View File
@@ -1,5 +1,4 @@
export * from '@shumai/agent/src/workflows/agent-autofill'
export * from '@shumai/agent/src/workflows/agent-chat'
export * from '@shumai/agent/src/workflows/agent-embedding'
export * from '@shumai/agent/src/workflows/agent-tool-call'
export * from '@shumai/agent/src/workflows/query-embedding-for-search'
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/cli",
"version": "0.3.6",
"version": "0.4.2",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/transcode-app",
"version": "0.3.6",
"version": "0.4.2",
"private": true,
"type": "module",
"bin": {
+8
View File
@@ -56,6 +56,8 @@ apps/web/e2e/
|---|---|
| Sign up as the first user and become the team owner | `tests/auth/signup.spec.ts` |
| Log in with existing credentials | `tests/auth/login.spec.ts` |
| Show error when accessing reset-password page without token | `tests/auth/reset-password.spec.ts` |
| Reset password with valid token and allow logging in with new password | `tests/auth/reset-password.spec.ts` |
### team
| Test | File |
@@ -63,6 +65,7 @@ apps/web/e2e/
| Owner invites a member via invite link and the invitee joins | `tests/team/invite-member.spec.ts` |
| Owner removes a member via the members dialog; the membership is deleted from the DB | `tests/team/remove-member.spec.ts` |
| Owner changes a team member role in the members dialog and permissions update | `tests/team/change-member-role.spec.ts` |
| Owner manages resource quotas in settings and resets usage from the dashboard | `tests/team/quotas-settings.spec.ts` |
### project
| Test | File |
@@ -80,12 +83,17 @@ apps/web/e2e/
| Owner drags a file into a folder and a folder into another folder | `tests/project/move-folder-file.spec.ts` |
| Owner drags a file onto a file to create a version stack | `tests/project/version-stack-create.spec.ts` |
| Owner drags a file onto a version stack to add a new version | `tests/project/version-stack-add-version.spec.ts` |
| Owner manages version stack: removes version from 3-version stack | `tests/project/version-stack-manage.spec.ts` |
| Owner removes version from 2-version stack which dissolves into standalone files | `tests/project/version-stack-manage.spec.ts` |
| Dissolving a shared version stack renders the remaining standalone file name on the share page | `tests/project/version-stack-manage.spec.ts` |
| Moving a shared file into a shared version stack leaves only 1 stack card on the share page | `tests/project/version-stack-manage.spec.ts` |
| Owner searches the file list by an English name keyword | `tests/project/search-file-list.spec.ts` |
| Owner searches the file list by Chinese, Japanese, and Korean name keywords | `tests/project/search-file-list.spec.ts` |
| Owner searches the file list with a keyword and a filter condition | `tests/project/search-file-list.spec.ts` |
| Owner saves a search result as a collection | `tests/project/save-search-collection.spec.ts` |
| Image share with watermark + download disabled: no download buttons on public list/detail pages, watermarked proxy served, download-url rejected | `tests/project/share-watermark-no-download.spec.ts` |
| Video share with watermark + download disabled: no download buttons on public list/detail pages, watermarked proxy served, download-url rejected | `tests/project/share-watermark-no-download.spec.ts` |
| Owner views a file and it appears in recents | `tests/project/recents.spec.ts` |
### file
| Test | File |
+16 -11
View File
@@ -21,19 +21,23 @@ const currentDir = path.dirname(fileURLToPath(import.meta.url))
const envPath = path.resolve(currentDir, '.env.e2e')
const fixturesDir = path.resolve(currentDir, '../../../packages/e2e/fixtures')
// Load E2E environment variables if present
if (fs.existsSync(envPath)) {
const content = fs.readFileSync(envPath, 'utf8')
content.split('\n').forEach((line) => {
const parts = line.split('=')
if (parts.length >= 2) {
const key = parts[0].trim()
const value = parts.slice(1).join('=').trim()
process.env[key] = value
}
})
function loadEnv() {
if (fs.existsSync(envPath)) {
const content = fs.readFileSync(envPath, 'utf8')
content.split('\n').forEach((line) => {
const parts = line.split('=')
if (parts.length >= 2) {
const key = parts[0].trim()
const value = parts.slice(1).join('=').trim()
process.env[key] = value
}
})
}
}
// Load E2E environment variables if present
loadEnv()
export interface OwnerFixture {
/** Page opened in a browser context logged in as the team owner. */
page: Page
@@ -104,6 +108,7 @@ export const test = base.extend<{
prisma: [
// eslint-disable-next-line no-empty-pattern
async ({}, use) => {
loadEnv()
const connectionString = process.env.DATABASE_URL
const pool = new Pool({ connectionString })
const adapter = new PrismaPg(pool)
+22 -4
View File
@@ -24,14 +24,32 @@ export async function dragCard(page: Page, source: Locator, target: Locator): Pr
await page.mouse.move(fromX, fromY)
await page.mouse.down()
// Move in several chunks so the dnd-kit sensor activates mid-drag
const dx = toX - fromX
const dy = toY - fromY
const dist = Math.hypot(dx, dy)
// Step past the 10px activation constraint to trigger drag activation
// and give the browser a brief moment for status to become 'dragging'
const activationRatio = dist > 0 ? Math.min(1, 15 / dist) : 1
await page.mouse.move(fromX + dx * activationRatio, fromY + dy * activationRatio)
await page.waitForTimeout(50)
// Move in several chunks so the intermediate dragmove handlers run
const chunks = 8
for (let i = 1; i <= chunks; i++) {
const x = fromX + ((toX - fromX) * i) / chunks
const y = fromY + ((toY - fromY) * i) / chunks
const x = fromX + (dx * i) / chunks
const y = fromY + (dy * i) / chunks
await page.mouse.move(x, y)
await page.waitForTimeout(15)
await page.waitForTimeout(20)
}
// Ensure pointer is directly over target center and allow requestAnimationFrame
// and collision microtasks to settle before triggering drop
await page.mouse.move(toX, toY)
await page.evaluate(
() => new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve))),
)
await page.waitForTimeout(100)
await page.mouse.up()
}
+7 -6
View File
@@ -7,15 +7,16 @@ interface CreateShareResponse {
}
/**
* Locates a file/folder card by its name. File names are rendered as
* `EditableText` inputs (not text nodes), so `getByText` does not match them.
* The returned locator points at the card wrapper div (right-clickable to open
* the card context menu).
* Locates a file/folder card by its name. Matches both read-only text nodes
* and active EditableText inputs. The returned locator points at the card
* wrapper div (right-clickable to open the card context menu).
*/
export function fileCard(page: Page, name: string): Locator {
return page
.locator('div.group')
.filter({ has: page.locator(`input[value="${name}"]`) })
.locator('[data-testid="file-card"]')
.filter({
has: page.locator(`input[value="${name}"]`).or(page.getByText(name, { exact: true })),
})
.first()
}
+4 -1
View File
@@ -32,4 +32,7 @@ async function start() {
await import('../src/index.ts')
}
start().catch(console.error)
start().catch((err) => {
console.error('Failed to start E2E server:', err)
process.exit(1)
})
@@ -0,0 +1,63 @@
import { expect, test } from '../../fixtures'
import { apiSignup, E2E_PASSWORD, uniqueEmail } from '../../helpers/auth'
import { loginViaUi } from '../../helpers/ui'
test('should show error when accessing reset-password page without token', async ({ page }) => {
await page.goto('/reset-password')
await expect(page.getByText('Invalid or missing reset token.')).toBeVisible()
await expect(page.getByRole('button', { name: 'Go to Login' })).toBeVisible()
})
test('should reset password with valid token and allow logging in with new password', async ({
page,
request,
prisma,
}) => {
const email = uniqueEmail('reset-pw')
const oldPassword = E2E_PASSWORD
const newPassword = 'newSecretPassword123'
// 1. Seed user
await apiSignup(request, email, oldPassword)
// 2. Request password reset via API
const resetRes = await request.post('/api/auth/request-password-reset', {
data: {
email,
redirectTo: '/reset-password',
},
})
expect(resetRes.ok()).toBe(true)
// Retrieve the generated verification token from DB
const verification = await prisma.verification.findFirst({
where: {
identifier: { startsWith: 'reset-password:' },
},
orderBy: { createdAt: 'desc' },
})
expect(verification).not.toBeNull()
const token = verification!.identifier.replace('reset-password:', '')
// 3. Open reset password page with token
await page.goto(`/reset-password?token=${token}`)
// 4. Fill in new password and confirmation
await page.locator('input#password').fill(newPassword)
await page.locator('input#confirmPassword').fill(newPassword)
// 5. Submit form
await page.getByRole('button', { name: 'Reset Password' }).click()
// 6. Verify success state
await expect(page.getByText('Password reset successfully!')).toBeVisible()
// 7. Click Go to Login
await page.getByRole('button', { name: 'Go to Login' }).click()
await expect(page).toHaveURL('/login')
// 8. Log in with the new password
await loginViaUi(page, email, newPassword)
await expect(page).toHaveURL(/\/teams\/[^/]+/)
await expect(page.getByRole('heading', { name: 'Projects' })).toBeVisible()
})
@@ -40,8 +40,8 @@ test('owner should create a project with a cover image', async ({ owner, prisma
await expect(page).toHaveURL(/\/projects\/[^/]+/)
await expect(page.locator(`text=${projectName}`).first()).toBeVisible()
// Back on the dashboard the new project card is visible with its cover
await page.click('button[aria-label="Dashboard"]')
// Back on the home page the new project card is visible with its cover
await page.click('button[aria-label="Home"]')
await expect(page).toHaveURL(/\/teams\/[^/]+/)
await expect(page.locator(`img[alt="${projectName}"]`)).toBeVisible()
@@ -0,0 +1,35 @@
import { expect, test } from '../../fixtures'
import { fileCard } from '../../helpers/files'
test('owner views a file and it appears in recents', async ({ file }) => {
const { page, projectId, fileName, fileId } = file
// Open the file details page to record view
await page.goto(`/projects/${projectId}/files/${fileId}`)
await expect(page.getByText(fileName).first()).toBeVisible()
// Navigate to Recents page via sidebar or direct URL
await page.goto(`/projects/${projectId}`)
await page.getByText('Recently Viewed').first().click()
await expect(page).toHaveURL(new RegExp(`/projects/${projectId}/recents`))
// The viewed file should be visible in recents
const card = fileCard(page, fileName)
await expect(card).toBeVisible()
// Toolbar should show hint text instead of action buttons
await expect(
page.getByText('This page displays the 100 most recent files you viewed in this project.'),
).toBeVisible()
await expect(page.getByRole('button', { name: 'Field' })).not.toBeVisible()
await expect(page.getByRole('button', { name: 'Search' })).not.toBeVisible()
await expect(page.getByText('AGENTS.md')).not.toBeVisible()
// Right-clicking the file should NOT show the context menu
await card.click({ button: 'right' })
await expect(page.getByRole('menuitem', { name: 'Delete' })).not.toBeVisible()
await expect(page.getByRole('menuitem', { name: 'Download' })).not.toBeVisible()
// The per-item overflow menu must also be unavailable on the read-only Recents page.
await expect(card.getByRole('button')).toHaveCount(0)
})
@@ -0,0 +1,233 @@
import { expect, test } from '../../fixtures'
import { seedFile } from '../../helpers/assets'
import { dragCard } from '../../helpers/dnd'
import { apiAddAssetsToShare, apiCreateShare, fileCard } from '../../helpers/files'
test('owner manages versions: removes a version from a 3-version stack', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const fileA = await seedFile(prisma, projectId, rootFolderId, `e2e-mng-a-${Date.now()}`)
const fileB = await seedFile(prisma, projectId, rootFolderId, `e2e-mng-b-${Date.now()}`)
const fileC = await seedFile(prisma, projectId, rootFolderId, `e2e-mng-c-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
await expect(fileCard(page, fileA.name)).toBeVisible()
await expect(fileCard(page, fileB.name)).toBeVisible()
await expect(fileCard(page, fileC.name)).toBeVisible()
// 1. Create a 3-version stack: B on A -> stack (v2), then C on stack -> stack (v3)
await dragCard(page, fileCard(page, fileB.name), fileCard(page, fileA.name))
await expect(page.getByText('v2', { exact: true })).toBeVisible()
await dragCard(page, fileCard(page, fileC.name), fileCard(page, fileB.name))
await expect(page.getByText('v3', { exact: true })).toBeVisible()
const stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
expect(stack.fileCount).toBe(3)
// 2. Navigate to file viewer for the stack
await page.goto(`/projects/${projectId}/files/${stack.id}`)
// 3. Open breadcrumb menu and click "Manage versions..."
// Click on the asset name / badge dropdown trigger
await page.getByRole('button', { name: new RegExp(fileC.name) }).click()
// Hover or click on Versions sub-menu trigger
await page.getByRole('menuitem', { name: 'Versions', exact: true }).hover()
// Click "Manage versions..."
await page.getByRole('button', { name: /manage versions$/i }).click()
// 4. Verify Manage versions dialog is visible with all 3 versions
const dialog = page.getByRole('dialog')
await expect(dialog).toBeVisible()
await expect(dialog.getByText('Manage versions')).toBeVisible()
await expect(dialog.getByText(fileA.name)).toBeVisible()
await expect(dialog.getByText(fileB.name)).toBeVisible()
await expect(dialog.getByText(fileC.name)).toBeVisible()
// 5. Remove fileB from stack: hover its row and click Remove from stack
const rowB = dialog.locator('div.group', { hasText: fileB.name })
await rowB.hover()
const moreButton = rowB.getByRole('button', { name: 'More options' })
await moreButton.click()
await page.getByRole('menuitem', { name: /remove from stack/i }).click()
// 6. Verify toast and that fileB is removed from dialog
await expect(page.getByText(/version removed from stack/i)).toBeVisible()
await expect(dialog.getByText(fileB.name)).not.toBeVisible()
await expect(dialog.getByText(fileA.name)).toBeVisible()
await expect(dialog.getByText(fileC.name)).toBeVisible()
// DB: stack now has 2 versions (fileA and fileC), and fileB is standalone in root folder
await expect
.poll(async () => {
const s = await prisma.asset.findUnique({ where: { id: stack.id } })
return s?.fileCount
})
.toBe(2)
const updatedFileB = await prisma.asset.findUnique({ where: { id: fileB.id } })
expect(updatedFileB?.parentId).toBe(rootFolderId)
})
test('owner removes version from 2-version stack dissolving the stack into standalone files', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const fileA = await seedFile(prisma, projectId, rootFolderId, `e2e-dis-a-${Date.now()}`)
const fileB = await seedFile(prisma, projectId, rootFolderId, `e2e-dis-b-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
await dragCard(page, fileCard(page, fileB.name), fileCard(page, fileA.name))
await expect(page.getByText('v2', { exact: true })).toBeVisible()
const stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
// Open file view
await page.goto(`/projects/${projectId}/files/${stack.id}`)
// Open breadcrumb menu and click "Manage versions..."
await page.getByRole('button', { name: new RegExp(fileB.name) }).click()
await page.getByRole('menuitem', { name: 'Versions', exact: true }).hover()
await page.getByRole('button', { name: /manage versions$/i }).click()
const dialog = page.getByRole('dialog')
await expect(dialog).toBeVisible()
// Remove fileB -> stack dissolves -> only fileA remains
const rowB = dialog.locator('div.group', { hasText: fileB.name })
await rowB.hover()
const moreButton = rowB.getByRole('button', { name: 'More options' })
await moreButton.click()
await page.getByRole('menuitem', { name: /remove from stack/i }).click()
// Dialog should close automatically when stack dissolves
await expect(dialog).not.toBeVisible()
// DB: stack asset is deleted
await expect
.poll(async () => {
const s = await prisma.asset.findUnique({ where: { id: stack.id } })
return s === null
})
.toBe(true)
// Both fileA and fileB are now standalone files in rootFolder
const updatedFileA = await prisma.asset.findUnique({ where: { id: fileA.id } })
const updatedFileB = await prisma.asset.findUnique({ where: { id: fileB.id } })
expect(updatedFileA?.parentId).toBe(rootFolderId)
expect(updatedFileB?.parentId).toBe(rootFolderId)
})
test('dissolving a shared version stack renders the remaining standalone file name on the share page', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const fileA = await seedFile(prisma, projectId, rootFolderId, `e2e-shd-a-${Date.now()}`)
const fileB = await seedFile(prisma, projectId, rootFolderId, `e2e-shd-b-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
await dragCard(page, fileCard(page, fileB.name), fileCard(page, fileA.name))
await expect(page.getByText('v2', { exact: true })).toBeVisible()
const stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
// Share the version stack
await fileCard(page, fileB.name).click({ button: 'right' })
await page.getByRole('menuitem', { name: 'Create Share Link' }).click()
const toast = page.locator('[data-sonner-toast]').filter({ hasText: 'Share link created' })
await expect(toast).toBeVisible()
const share = await prisma.shareLink.findFirstOrThrow({
where: { projectId },
orderBy: { createdAt: 'desc' },
})
// Open file viewer for the stack
await page.goto(`/projects/${projectId}/files/${stack.id}`)
// Open breadcrumb menu and click "Manage versions"
await page.getByRole('button', { name: new RegExp(fileB.name) }).click()
await page.getByRole('menuitem', { name: 'Versions', exact: true }).hover()
await page.getByRole('button', { name: /manage versions$/i }).click()
const dialog = page.getByRole('dialog')
await expect(dialog).toBeVisible()
// Remove fileB -> stack dissolves -> only fileA remains
const rowB = dialog.locator('div.group', { hasText: fileB.name })
await rowB.hover()
const moreButton = rowB.getByRole('button', { name: 'More options' })
await moreButton.click()
await page.getByRole('menuitem', { name: /remove from stack/i }).click()
await expect(dialog).not.toBeVisible()
// Navigate to public share page: verify fileA is displayed with its non-empty name
await page.goto(`/share/${share.id}`)
await expect(fileCard(page, fileA.name)).toBeVisible()
await expect(fileCard(page, fileB.name)).not.toBeVisible()
})
test('moving a shared file into a shared version stack leaves only 1 stack card on the share page', async ({
project,
prisma,
}) => {
const { page, projectId, rootFolderId } = project
const fileA = await seedFile(prisma, projectId, rootFolderId, `e2e-mvsh-a-${Date.now()}`)
const fileB = await seedFile(prisma, projectId, rootFolderId, `e2e-mvsh-b-${Date.now()}`)
const fileC = await seedFile(prisma, projectId, rootFolderId, `e2e-mvsh-c-${Date.now()}`)
await page.goto(`/projects/${projectId}`)
// 1. Create a version stack from file B and file A (stack has v2, displays file B)
await dragCard(page, fileCard(page, fileB.name), fileCard(page, fileA.name))
await expect(page.getByText('v2', { exact: true })).toBeVisible()
const stack = await prisma.asset.findFirstOrThrow({
where: { projectId, parentId: rootFolderId, type: 'version_stack' },
})
// 2. Create a share link and add both stack AB and file C to it
const share = await apiCreateShare(project.context.request, projectId, `Test Share ${Date.now()}`)
await apiAddAssetsToShare(project.context.request, share.id, [stack.id, fileC.id])
// Verify initial share page has both stack and file C
await page.goto(`/share/${share.id}`)
await expect(fileCard(page, fileB.name)).toBeVisible()
await expect(fileCard(page, fileC.name)).toBeVisible()
// 3. In project page, drop file C onto stack card (displays file B)
await page.goto(`/projects/${projectId}`)
await dragCard(page, fileCard(page, fileC.name), fileCard(page, fileB.name))
await expect(page.getByText('v3', { exact: true })).toBeVisible()
// 4. Return to public share page: now only 1 card exists (the stack showing file C with v3)
await page.goto(`/share/${share.id}`)
await expect(page.getByText('v3', { exact: true })).toBeVisible()
await expect(fileCard(page, fileC.name)).toBeVisible()
await expect(fileCard(page, fileB.name)).not.toBeVisible()
// DB: share rootFolder fileCount is 1 and exactly 1 symlink exists
const symlinks = await prisma.asset.findMany({
where: { parentId: share.rootFolderId, type: 'symlink' },
})
expect(symlinks).toHaveLength(1)
expect(symlinks[0].targetId).toBe(stack.id)
const updatedShareRoot = await prisma.asset.findUnique({ where: { id: share.rootFolderId } })
expect(updatedShareRoot?.fileCount).toBe(1)
})
@@ -0,0 +1,126 @@
import { expect, test } from '../../fixtures'
test('owner manages resource quotas in settings and resets usage from the dashboard', async ({
owner,
prisma,
}) => {
const { page, teamId, email } = owner
// 1. Navigate to team settings
await page.goto(`/teams/${teamId}/settings`)
// 2. Click the Quotas sidebar tab
await page.getByRole('button', { name: /Quotas|配额/i }).click()
// Verify URL contains #quotas and remains after reload
expect(page.url()).toContain('#quotas')
await page.reload()
expect(page.url()).toContain('#quotas')
// 3. Verify Quotas settings card is displayed with empty state
await expect(page.getByRole('heading', { name: /^(Resource Quotas|资源配额)$/i })).toBeVisible()
await expect(page.getByText(/No resource quotas configured|暂未配置资源配额/i)).toBeVisible()
// 4. Open Create Quota Dialog
await page
.getByRole('button', { name: /Add Quota|添加配额/i })
.first()
.click()
await expect(page.getByText(/Create Quota Rule|创建配额规则/i)).toBeVisible()
// 5. Fill and submit new quota rule
await page.fill('#quota-limit', '50000')
await page.getByRole('button', { name: /^(Create|创建)$/i }).click()
// 6. Verify quota rule card appears in the list
await expect(page.getByText(/50,000 tokens|50000 tokens/i)).toBeVisible()
await expect(page.getByText(/Each Member|每个成员/i)).toBeVisible()
// Verify in database
const createdRule = await prisma.quotaRule.findFirst({
where: { teamId, limit: 50000 },
})
expect(createdRule).not.toBeNull()
expect(createdRule?.enabled).toBe(true)
// 7. Usage is now monitored from the dashboard, not the settings rule card.
await expect(page.getByRole('button', { name: /View Usage|查看使用量/i })).not.toBeVisible()
// 8. Toggle quota rule switch
await page.getByRole('switch').click()
await expect(page.getByRole('switch')).not.toBeChecked()
const disabledRule = await prisma.quotaRule.findFirst({
where: { id: createdRule?.id },
})
expect(disabledRule?.enabled).toBe(false)
// Toggle back on
await page.getByRole('switch').click()
await expect(page.getByRole('switch')).toBeChecked()
// 9. Open Edit Dialog by clicking the quota card
await page.getByText(/50,000 tokens|50000 tokens/i).click()
await expect(page.getByText(/Edit Quota Rule|编辑配额规则/i)).toBeVisible()
await expect(page.getByRole('button', { name: /Each Member|每个成员/i })).toBeDisabled()
await expect(page.locator('#quota-resource')).toBeDisabled()
// 10. Update limit
await page.fill('#quota-limit', '75000')
await page.getByRole('button', { name: /^(Save|保存)$/i }).click()
// Verify updated limit in UI and DB
await expect(page.getByText(/75,000 tokens|75000 tokens/i)).toBeVisible()
const updatedRule = await prisma.quotaRule.findFirst({
where: { id: createdRule?.id },
})
expect(updatedRule?.limit).toBe(75000)
// 11. Verify quota usage and reset it from the dashboard
const ownerUser = await prisma.user.findUnique({ where: { email } })
expect(ownerUser).not.toBeNull()
await prisma.quotaRecord.create({
data: {
ruleId: createdRule?.id as string,
teamId,
userId: ownerUser?.id,
periodStart: new Date(),
periodEnd: new Date(Date.now() + 24 * 60 * 60 * 1000),
consumed: 100,
},
})
await page.goto(`/teams/${teamId}/dashboard`)
await page.getByRole('button', { name: /Quotas|配额/i }).click()
await expect(
page.getByText(/Monitor live quota usage|在仪表盘中监控实时配额使用情况/),
).toBeVisible()
const quotaRuleButton = page.getByRole('button', { name: /Expand quota rule|展开配额规则/i })
await quotaRuleButton.click()
await expect(page.getByText(/^100 \/ 75,000 tokens$|^100 \/ 75000 tokens$/i)).toBeVisible()
await page.getByRole('button', { name: /Reset Usage|重置使用量/i }).click()
await expect(page.getByRole('alertdialog')).toBeVisible()
await page.getByRole('button', { name: /Confirm|确认/i }).click()
await expect(page.getByText(/^0 \/ 75,000 tokens$|^0 \/ 75000 tokens$/i)).toBeVisible()
const resetRecord = await prisma.quotaRecord.findFirst({
where: { ruleId: createdRule?.id, userId: ownerUser?.id },
})
expect(resetRecord?.consumed).toBe(0)
// 12. Delete quota rule from settings
await page.goto(`/teams/${teamId}/settings`)
await page.getByRole('button', { name: /Quotas|配额/i }).click()
await page.getByRole('button', { name: 'Quota actions' }).first().click()
await page.getByRole('menuitem', { name: /Delete|删除/i }).click()
await page
.getByRole('button', { name: /Delete|删除/i })
.last()
.click()
// Verify empty state returns
await expect(page.getByText(/No resource quotas configured|暂未配置资源配额/i)).toBeVisible()
const deletedRule = await prisma.quotaRule.findFirst({
where: { id: createdRule?.id },
})
expect(deletedRule).toBeNull()
})
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/web-app",
"version": "0.3.6",
"version": "0.4.2",
"private": true,
"type": "module",
"dependencies": {
+25
View File
@@ -13,12 +13,37 @@ import { initTranscodeWorkflows } from '@shumai/transcode'
import { workflowService } from '@shumai/workflow-core'
import { handleDaemonCommands } from '@shumai/core/src/utils/daemon'
import { authService } from '@shumai/core/src/auth/auth'
if (process.argv.includes('--check')) {
console.log('✅ Web app evaluated successfully!')
process.exit(0)
}
const cliArgs = process.argv.slice(2)
const resetCmdIndex = cliArgs.findIndex(
(arg) => arg === 'reset-password' || arg === 'reset-passowrd',
)
if (resetCmdIndex !== -1) {
const email = cliArgs[resetCmdIndex + 1]
if (!email || email.startsWith('-')) {
console.error('Error: Please provide a valid email address.')
console.error('Usage: shumai reset-password <email>')
process.exit(1)
}
try {
const link = await authService.generatePasswordResetLink(email)
console.log(link)
process.exit(0)
} catch (err: unknown) {
const message = err instanceof Error ? err.message : 'Unknown error'
console.error(`Error: ${message}`)
process.exit(1)
}
}
async function run() {
// Initialize workflows and activities for local executor mode
initAgentWorkflows()
+18 -15
View File
@@ -33,7 +33,7 @@
},
"apps/agent": {
"name": "@shumai/agent-app",
"version": "0.3.5",
"version": "0.3.7",
"bin": {
"shumai-agent": "./src/index.ts",
},
@@ -45,7 +45,7 @@
},
"apps/cli": {
"name": "@shumai/cli",
"version": "0.3.5",
"version": "0.3.7",
"bin": {
"shumai-cli": "./src/index.ts",
},
@@ -60,7 +60,7 @@
},
"apps/transcode": {
"name": "@shumai/transcode-app",
"version": "0.3.5",
"version": "0.3.7",
"bin": {
"shumai-transcode": "./src/index.ts",
},
@@ -72,7 +72,7 @@
},
"apps/web": {
"name": "@shumai/web-app",
"version": "0.3.5",
"version": "0.3.7",
"dependencies": {
"@shumai/agent": "workspace:*",
"@shumai/api": "workspace:*",
@@ -92,7 +92,7 @@
},
"packages/agent": {
"name": "@shumai/agent",
"version": "0.3.5",
"version": "0.3.7",
"dependencies": {
"@anthropic-ai/sandbox-runtime": "0.0.66",
"@earendil-works/pi-agent-core": "0.82.1",
@@ -102,12 +102,12 @@
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@shumai/workflow-core": "workspace:*",
"@sinclair/typebox": "0.34.49",
"@temporalio/activity": "1.17.2",
"@temporalio/worker": "1.17.2",
"@temporalio/workflow": "1.17.2",
"adm-zip": "0.5.17",
"jittered-fractional-indexing": "1.0.1",
"typebox": "1.1.38",
"ulid": "3.0.2",
"zod": "4.4.3",
},
@@ -117,7 +117,7 @@
},
"packages/api": {
"name": "@shumai/api",
"version": "0.3.5",
"version": "0.3.7",
"dependencies": {
"@hono/zod-validator": "0.7.6",
"@shumai/core": "workspace:*",
@@ -131,7 +131,7 @@
},
"packages/core": {
"name": "@shumai/core",
"version": "0.3.5",
"version": "0.3.7",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"@earendil-works/pi-ai": "0.82.1",
@@ -166,7 +166,7 @@
},
"packages/db": {
"name": "@shumai/db",
"version": "0.3.5",
"version": "0.3.7",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"@prisma/adapter-pg": "7.8.0",
@@ -182,7 +182,7 @@
},
"packages/dtos": {
"name": "@shumai/dtos",
"version": "0.3.5",
"version": "0.3.7",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"zod": "4.4.3",
@@ -190,7 +190,7 @@
},
"packages/e2e": {
"name": "@shumai/e2e",
"version": "0.3.5",
"version": "0.3.7",
"dependencies": {
"@earendil-works/pi-agent-core": "0.82.1",
"@google/genai": "1.52.0",
@@ -205,11 +205,11 @@
},
"packages/tableprinter": {
"name": "@shumai/tableprinter",
"version": "0.3.5",
"version": "0.3.7",
},
"packages/transcode": {
"name": "@shumai/transcode",
"version": "0.3.5",
"version": "0.3.7",
"dependencies": {
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
@@ -223,7 +223,7 @@
},
"packages/webui": {
"name": "@shumai/webui",
"version": "0.3.5",
"version": "0.3.7",
"dependencies": {
"@base-ui/react": "1.5.0",
"@dnd-kit/abstract": "0.4.0",
@@ -321,7 +321,7 @@
},
"packages/workflow-core": {
"name": "@shumai/workflow-core",
"version": "0.3.5",
"version": "0.3.7",
"dependencies": {
"@shumai/core": "workspace:*",
"@shumai/db": "workspace:*",
@@ -333,6 +333,9 @@
},
},
},
"patchedDependencies": {
"@earendil-works/pi-ai@0.82.1": "patches/@earendil-works%2Fpi-ai@0.82.1.patch",
},
"packages": {
"@alloc/quick-lru": ["@alloc/quick-lru@5.2.0", "", {}, "sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw=="],
+7 -2
View File
@@ -43,6 +43,7 @@
"pages": [
"using-shumai/agent-providers",
"using-shumai/agents",
"using-shumai/agents-md",
"using-shumai/agent-skills",
"using-shumai/agent-tools",
"using-shumai/mcp-servers",
@@ -50,8 +51,12 @@
]
},
{
"group": "Media Transcoding Settings",
"pages": ["using-shumai/transcode-settings"]
"group": "Team Settings",
"pages": [
"using-shumai/transcode-settings",
"using-shumai/quotas-settings",
"using-shumai/password-reset"
]
},
{
"group": "Configuration",
Binary file not shown.

Before

Width:  |  Height:  |  Size: 309 KiB

After

Width:  |  Height:  |  Size: 131 KiB

+1 -1
View File
@@ -25,7 +25,7 @@ To protect your workspace environments and sensitive data, the sandbox enforces
## Domain Whitelisting
By default, sandboxed scripts cannot make network requests to the internet. Team Owners must explicitly authorize external APIs by managing domain access:
By default, Network Sandbox is **disabled** (allowing all outbound network requests). Team Owners can toggle Network Sandbox **ON** in Sandbox Settings to enforce strict domain whitelisting:
### Allowed Domains
+12 -12
View File
@@ -30,36 +30,36 @@ The following tools can be individually toggled in the agent configuration dialo
- **Description:** Allows the agent to load and read skill definitions to perform specialized actions.
- **Why it matters:** Enables the agent to inspect the instructions/configurations of custom skills registered in your workspace before executing them.
### 3. Analyze Image (`analyze_image`)
### 3. Read Asset (`read_asset`)
- **Description:** Allows the agent to view and analyze the visual contents of an image file.
- **Why it matters:** Enables multimodal agents to "see" image assets uploaded to Shumai, allowing them to answer comments or write metadata about images.
- **Description:** Allows the agent to inspect images, extract still frames from video files, render PDF document pages, and read text/markdown documents.
- **Why it matters:** Enables multimodal agents to view visual media assets uploaded to Shumai, extract frame-by-frame video screenshots, inspect document layouts, and answer questions about uploaded creative assets.
### 4. Video Frame Extraction (`screenshot`)
- **Description:** Allows the agent to extract still image frames from video files for analysis.
- **Why it matters:** Enables the agent to perform frame-by-frame analysis on videos (e.g. for comments, annotation, or search).
### 5. Browse Shumai (`list_assets`)
### 4. Browse Shumai (`list_assets`)
- **Description:** Allows the agent to list folders and browse files in Shumai.
- **Why it matters:** Gives the agent visual mapping of the project structure so it can search for other files or locate related assets.
### 6. Create Folder (`create_folder`)
### 5. Create Folder (`create_folder`)
- **Description:** Allows the agent to create new folders in Shumai.
- **Why it matters:** Needed if the agent organizes outputs or places files in custom subdirectory folders.
### 7. Create File (`create_file`)
### 6. Create File (`create_file`)
- **Description:** Allows the agent to create and upload new files to Shumai.
- **Why it matters:** Allows the agent to generate new assets (e.g. text logs, transcoded clips, reports) and save them directly back into your project.
### 8. Create Version (`create_version`)
### 7. Create Version (`create_version`)
- **Description:** Allows the agent to upload and stack a new version of an existing file in Shumai.
- **Why it matters:** Allows the agent to update files in a clean, non-destructive way by stacking revisions rather than creating duplicate files.
### 8. Download Asset (`download_asset`)
- **Description:** Allows the agent to download a workspace asset or specific storage S3 key to the local `.pi` directory for processing with local scripts or bash commands.
- **Why it matters:** Enables agents executing custom skill scripts to fetch files or intermediate transcode artifacts to local disk.
---
## Restricting Tools
+53
View File
@@ -0,0 +1,53 @@
---
title: AGENTS.md Context & Instructions
description: Provide hierarchical, folder-specific instructions and context to AI agents.
---
In addition to global agent system prompts and skills, Shumai allows you to define project-level and folder-level instructions using **`AGENTS.md`**.
When an AI agent (such as a Chat Agent or Autofill Agent) operates within a project or folder, Shumai automatically loads the corresponding `AGENTS.md` files to provide context, operational rules, and project-specific knowledge.
---
## Hierarchical Traversal & Inheritance
Shumai uses a hierarchical context model for `AGENTS.md`:
- **Upward Traversal**: When an agent works on a file or folder, Shumai traverses the folder tree upwards from the current folder all the way to the **project root level**.
- **Combined Context**: All `AGENTS.md` files found along this path are loaded into the agent's context in order (from the project root down to the current folder).
- **Layered Rules**: This lets you define broad, project-wide rules and guidelines at the project root `AGENTS.md`, while adding specific rules, vocabulary, or autofill hints in nested subfolder `AGENTS.md` files.
---
## What to Include in AGENTS.md
You can use `AGENTS.md` to define any instructions you want the agent to follow when working in that folder or project:
- **Project & Folder Overview**: Describe what assets are stored in the folder and how they relate to the broader project.
- **Agent Behavioral Rules**: Specify guardrails, tone, required formats, or things the agent should and shouldn't do when answering queries or leaving comments.
- **Terminology & Domain Knowledge**: Define specialized vocabulary, acronyms, brand naming conventions, or project-specific jargon.
- **Metadata & Autofill Instructions**: Guide the **Autofill Agent** on how to summarize assets, generate tags, or populate specific custom metadata fields for files in this folder.
---
## Accessing and Editing AGENTS.md
### Opening the Editor
To view or edit `AGENTS.md`:
1. Open any project or folder in the Shumai web application.
2. Click the **`AGENTS.md`** button in the top navigation toolbar, or right-click any folder in the file browser and choose **AGENTS.md**.
3. A dedicated WYSIWYG Markdown editor opens with full formatting support (headings, lists, code blocks, bold/italics, and links).
### Auto-Save
As you make changes in the editor, Shumai automatically saves your edits in the background with visual save status indicators (`Saving...`, `Saved`).
---
## Permissions & Read-Only Mode
- **Viewing**: All team members who have read access to the project can view `AGENTS.md`.
- **Editing**: **Only Team Owners** are permitted to create, edit, or delete `AGENTS.md` files.
- **Read-Only State**: Non-owner members will see a clear read-only indicator in the toolbar explaining that only team owners can edit instructions for the file.
+60
View File
@@ -0,0 +1,60 @@
---
title: Password Reset
description: How administrators generate secure password reset links for team members.
---
Because Shumai does not require an external email delivery service (SMTP) to be configured for standard self-hosted deployments, password resets are initiated securely by an administrator using the Shumai CLI.
---
## Overview
When a user forgets their password, an administrator can generate a one-time password reset link directly from the command line.
- **Link Expiration**: The generated password reset link is valid for **1 hour** (`3600 seconds`).
- **One-Time Use**: Once used, the token is permanently invalidated.
- **Session Revocation**: When a user resets their password, all existing active sessions for that account are automatically revoked across all devices.
- **Latest Link Validity**: Generating a new reset link for a user immediately supersedes any previously generated pending links.
---
## Generating a Password Reset Link
### Running on Local Host / Server
Run the `reset-password` subcommand from your Shumai installation directory:
```bash
shumai reset-password <user-email>
```
_(Alternatively, `bun apps/web/src/index.ts reset-password <user-email>`)_
**Example Output:**
```text
http://localhost:3000/reset-password?token=bb3xg4uO3cwN0FQv4cgUK6Hh
```
### Running inside Docker
If Shumai is running inside a Docker container, execute the command using `docker exec`:
```bash
docker exec -it <container-name-or-id> shumai reset-password <user-email>
```
> [!TIP]
> Both `shumai` and the common alias `shuami` (e.g. `shuami reset-password <user-email>`) are supported.
> [!NOTE]
> For remote servers or production deployments, ensure `BETTER_AUTH_URL` is set in your environment (e.g., `BETTER_AUTH_URL=https://shumai.example.com`) so the CLI generates the reset link with your public domain instead of `localhost`.
---
## Member Reset Experience
1. The administrator sends the generated URL to the team member via their preferred secure communication channel.
2. The user opens the link in their browser.
3. The user inputs their **New Password** and **Confirm Password** (minimum 3 characters).
4. Upon clicking **Reset Password**, the password is securely updated, all previous active sessions are invalidated, and the user is redirected to `/login` to sign in.
+121
View File
@@ -0,0 +1,121 @@
---
title: Resource Quotas
description: Configure team resource quotas, monitor live usage, and reset quota windows from the dashboard.
---
AI workflows and media processing tools can consume significant compute and API credits. Shumai provides a **Resource Quotas** system that allows Team Owners to set granular rate limits, govern budgets, and monitor real-time usage across team members and roles.
---
## Overview
Resource Quotas allow you to define maximum usage limits across specific time windows (e.g., hourly, daily, or weekly). When a member or team reaches their assigned quota, further actions for that resource are safely blocked until the quota period resets.
Quota configuration and usage monitoring are separate parts of the team experience:
- To create and manage quota rules, open **Team Settings** and navigate to the **Quotas** tab.
- To monitor live usage, open the **Team Dashboard** and select **Quotas** under the **TEAM** section.
The quota dashboard is available to Team Owners.
---
## Scope Modes
When creating or editing a quota rule, you can define how the limit is applied:
### 1. Each Member
- **Behavior**: Every member in the scope receives their own independent limit. Usage by one member does not affect the remaining quota of others.
- **Optional Role Filter**: You can restrict the quota to members with a specific role (e.g. only **Editors** or **Reviewers**), or apply it to the entire team.
- **Use Case**: Give each editor a daily budget of 100,000 tokens.
### 2. All Members (Shared Pool)
- **Behavior**: The quota limit is shared as a collective pool across all matching members. Every member's consumption draws from the same total.
- **Optional Role Filter**: Apply the shared pool to all team members or restrict it to a specific role.
- **Use Case**: Cap the total team spending on AI models to $50 per day.
### 3. Selected Members
- **Behavior**: Assign individual limits to specific chosen team members.
- **Use Case**: Grant higher limits to senior artists or technical directors while keeping standard limits for others.
---
## Supported Resource Types
Shumai supports quota enforcement across multiple layers of the AI and agent architecture:
| Resource Type | Description | Target Specification |
| :-------------------------- | :-------------------------------------------------------------------------------------- | :----------------------------------- |
| **Total Tokens** | Total LLM input and output tokens consumed in chat, comments, and autofill. | Team, Role, or Members |
| **AI Cost ($)** | Dollar expenditure accrued from AI model providers. | Team, Role, or Members |
| **MCP Server Calls** | Number of tool invocations directed to a connected Model Context Protocol (MCP) server. | Target MCP Server ID |
| **Sandboxed Bash Commands** | Shell commands executed by the agent inside the isolated sandbox. | Wildcard pattern (e.g. `npm *`, `*`) |
| **Agent Tool Calls** | Calls to specific native tools (e.g. `read_asset`, `download_asset`, `create_file`). | Tool name |
### Bash Command Wildcard Matching
For sandboxed bash execution, you can restrict specific command patterns using wildcards (`*` and `?`):
- `*`: Matches any bash command execution.
- `npm *`: Matches any npm command (e.g., `npm install`, `npm test`).
- `git commit*`: Matches git commit invocations.
---
## Time Periods & Rollover
Quotas can be configured with the following reset periods:
- **1 Hour** (`1hour`)
- **5 Hours** (`5hour`)
- **1 Day** (`1day`)
- **7 Days** (`7day`)
### Lazy Reset Mechanism
Shumai tracks usage windows automatically. When a period window expires, the system resets the quota upon the member's next request, seamlessly starting a fresh window without requiring background maintenance tasks.
---
## Monitoring Live Usage
Team Owners can inspect live consumption from the team dashboard:
1. Open the **Team Dashboard**.
2. Select **Quotas** under the **TEAM** section.
3. Quota rules are shown as first-level collapsible entries.
4. Expand a rule to view its usage records.
Each usage record displays:
- **Consumption Status**: Progress bar showing consumed units versus the total limit.
- **Remaining Allowance**: Percentage and numerical amount remaining.
- **Refresh Countdown**: Remaining time until the active window resets (e.g., _"Refreshes in 45m"_).
- **Target**: The team shared pool, role pool, or individual member associated with the record.
- **Member Search**: For member-scoped rules, search the usage records by member name or email.
Usage records are loaded when a rule is expanded and refresh automatically while the rule is open.
### Manually Resetting a Usage Record
Each usage record includes a **Reset Usage** action. After confirming the action:
- Consumption is set to zero.
- The quota period starts immediately.
- The period ends after the configured quota duration.
- For example, resetting a one-day quota at 10:00 starts a new window from 10:00 until 10:00 the following day.
A reset can also initialize a zero-consumption record for a member or shared pool that has not received usage yet. Resets are recorded in the team audit log.
---
## Managing Quota Rules
- **Enable / Disable**: Toggle the switch on any quota card in Team Settings to temporarily pause or resume enforcement without deleting the rule.
- **Edit Rules**: Click any quota card in Team Settings to update limits, change reset periods, or adjust member selections.
- **Delete Rules**: Open the card action menu (`⋮`) in Team Settings and select **Delete** to remove the quota rule.
Quota usage is monitored and reset from the Team Dashboard rather than from the settings rule cards.
+1
View File
@@ -74,6 +74,7 @@ export default tseslint.config(
'**/packages/db/src/generated/**',
'**/packages/core/src/generated/**',
'**/data/**',
'**/packages/core/src/provider/generator/**',
'**/packages/webui/components/ui/**',
'**/packages/webui/paraglide/**',
'**/packages/webui/public/**',
+4 -1
View File
@@ -1,6 +1,6 @@
{
"name": "shumai",
"version": "0.3.6",
"version": "0.4.2",
"private": true,
"type": "module",
"repository": {
@@ -65,5 +65,8 @@
"typescript-eslint": "8.60.1",
"vite-tsconfig-paths": "6.1.1",
"vitest": "4.1.8"
},
"patchedDependencies": {
"@earendil-works/pi-ai@0.82.1": "patches/@earendil-works%2Fpi-ai@0.82.1.patch"
}
}
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent",
"version": "0.3.6",
"version": "0.4.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
@@ -14,12 +14,12 @@
"@shumai/db": "workspace:*",
"@shumai/dtos": "workspace:*",
"@shumai/workflow-core": "workspace:*",
"@sinclair/typebox": "0.34.49",
"@temporalio/activity": "1.17.2",
"@temporalio/worker": "1.17.2",
"@temporalio/workflow": "1.17.2",
"adm-zip": "0.5.17",
"jittered-fractional-indexing": "1.0.1",
"typebox": "1.1.38",
"ulid": "3.0.2",
"zod": "4.4.3"
},
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,354 @@
import { describe, it, expect } from 'vitest'
import { escapeXmlAttr, serializeContextToXml } from './serialize-context'
import type { ShumaiMessageContext } from '@shumai/dtos'
describe('serializeContextToXml', () => {
describe('escapeXmlAttr', () => {
it('escapes &, <, >, ", and \' characters correctly', () => {
expect(escapeXmlAttr('Alice & Bob <Admin> "v1" \'test\'')).toBe(
'Alice &amp; Bob &lt;Admin&gt; &quot;v1&quot; &apos;test&apos;',
)
})
it('handles clean strings without modification', () => {
expect(escapeXmlAttr('normal-string_123.mp4')).toBe('normal-string_123.mp4')
})
})
describe('Tag Serialization', () => {
it('returns empty string for null, undefined, or empty context', () => {
expect(serializeContextToXml(null)).toBe('')
expect(serializeContextToXml(undefined)).toBe('')
expect(serializeContextToXml({})).toBe('')
})
it('serializes minimal context with only user', () => {
const context: ShumaiMessageContext = {
user: { id: 'usr_1', name: 'Alice', role: 'editor' },
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <user id="usr_1" name="Alice" role="editor" />\n</context>',
)
})
it('serializes context with user and thread tag with reply_count', () => {
const context: ShumaiMessageContext = {
user: { id: 'usr_1', name: 'Alice', role: 'editor' },
thread: { id: 'th_01JXYZ', replyCount: 3 },
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <user id="usr_1" name="Alice" role="editor" />\n <thread id="th_01JXYZ" reply_count="3" />\n</context>',
)
})
it('serializes context with thread tag without reply_count', () => {
const context: ShumaiMessageContext = {
thread: { id: 'th_01JXYZ' },
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <thread id="th_01JXYZ" />\n</context>',
)
})
it('serializes minimal context with only currentAsset (without ancestors)', () => {
const context: ShumaiMessageContext = {
currentAsset: {
id: 'ast_1',
name: 'clip.mp4',
type: 'file',
mediaType: 'video',
path: 'clip.mp4',
},
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <current_asset id="ast_1" name="clip.mp4" type="file" media_type="video" path="clip.mp4" />\n</context>',
)
})
it('serializes full video context with duration, timestamp, annotation, attached files, and referenced assets', () => {
const context: ShumaiMessageContext = {
user: { id: 'usr_01', name: 'Alice Chen', role: 'editor' },
currentAsset: {
id: 'ast_01JC04_CURRENT',
name: 'product_launch_v2.mp4',
type: 'file',
mediaType: 'video',
parentId: 'ast_01JC03_PARENT',
path: 'Marketing/2026_Launch/Footage/product_launch_v2.mp4',
durationSeconds: 85.4,
ancestors: [
{ id: 'ast_01JC01_ROOT', name: 'Marketing' },
{ id: 'ast_01JC02_SUB', name: '2026_Launch' },
{ id: 'ast_01JC03_PARENT', name: 'Footage' },
],
},
position: { type: 'time', seconds: 14.25 },
annotation: true,
attachedFiles: [
{
id: 'ast_01JC05_ATT1',
name: 'brand_guidelines_2026.pdf',
type: 'file',
mediaType: 'pdf',
path: 'Brand/brand_guidelines_2026.pdf',
},
],
referencedAssets: [
{
id: 'ast_01JC07_REF2',
name: 'Color_LUTs',
type: 'folder',
path: 'Assets/Color_LUTs',
},
{
id: 'ast_01JC06_REF1',
name: 'logo_white.png',
type: 'file',
mediaType: 'image',
path: 'Brand/Logos/logo_white.png',
},
],
}
const expected = `<context>
<user id="usr_01" name="Alice Chen" role="editor" />
<current_asset id="ast_01JC04_CURRENT" name="product_launch_v2.mp4" type="file" media_type="video" parent_id="ast_01JC03_PARENT" path="Marketing/2026_Launch/Footage/product_launch_v2.mp4" duration_seconds="85.40">
<ancestors>
<folder id="ast_01JC01_ROOT" name="Marketing" />
<folder id="ast_01JC02_SUB" name="2026_Launch" />
<folder id="ast_01JC03_PARENT" name="Footage" />
</ancestors>
</current_asset>
<position type="time" seconds="14.2500" />
<annotation />
<attached_files>
<file id="ast_01JC05_ATT1" name="brand_guidelines_2026.pdf" type="file" media_type="pdf" path="Brand/brand_guidelines_2026.pdf" />
</attached_files>
<referenced_assets>
<asset id="ast_01JC06_REF1" name="logo_white.png" type="file" media_type="image" path="Brand/Logos/logo_white.png" />
<folder id="ast_01JC07_REF2" name="Color_LUTs" path="Assets/Color_LUTs" />
</referenced_assets>
</context>`
expect(serializeContextToXml(context)).toBe(expected)
})
it('serializes PDF document context with total_pages and position.type === "page"', () => {
const context: ShumaiMessageContext = {
currentAsset: {
id: 'pdf_1',
name: 'report.pdf',
type: 'file',
mediaType: 'pdf',
totalPages: 12,
},
position: { type: 'page', page: 3 },
}
const expected = `<context>
<current_asset id="pdf_1" name="report.pdf" type="file" media_type="pdf" total_pages="12" />
<position type="page" page="3" />
</context>`
expect(serializeContextToXml(context)).toBe(expected)
})
it('serializes Video context with total_frames and duration_seconds', () => {
const context: ShumaiMessageContext = {
currentAsset: {
id: 'vid_1',
name: 'clip.mp4',
type: 'file',
mediaType: 'video',
mimeType: 'video/mp4',
durationSeconds: 9,
totalFrames: 268,
},
}
const expected = `<context>
<current_asset id="vid_1" name="clip.mp4" type="file" media_type="video" mime_type="video/mp4" duration_seconds="9.00" total_frames="268" />
</context>`
expect(serializeContextToXml(context)).toBe(expected)
})
it('serializes navigation context with navigated="true" and ancestors hierarchy', () => {
const context: ShumaiMessageContext = {
currentAsset: {
id: 'f_sub',
name: 'Subfolder',
type: 'folder',
navigated: true,
ancestors: [{ id: 'f_root', name: 'Root' }],
},
}
const expected = `<context>
<current_asset id="f_sub" name="Subfolder" type="folder" navigated="true">
<ancestors>
<folder id="f_root" name="Root" />
</ancestors>
</current_asset>
</context>`
expect(serializeContextToXml(context)).toBe(expected)
})
it('serializes mime_type alongside media_type for current asset and attached files', () => {
const context: ShumaiMessageContext = {
currentAsset: {
id: 'img_1',
name: 'photo.png',
type: 'file',
mediaType: 'image',
mimeType: 'image/png',
},
attachedFiles: [
{
id: 'doc_1',
name: 'spec.pdf',
type: 'file',
mediaType: 'pdf',
mimeType: 'application/pdf',
},
],
}
const expected = `<context>
<current_asset id="img_1" name="photo.png" type="file" media_type="image" mime_type="image/png" />
<attached_files>
<file id="doc_1" name="spec.pdf" type="file" media_type="pdf" mime_type="application/pdf" />
</attached_files>
</context>`
expect(serializeContextToXml(context)).toBe(expected)
})
})
describe('XML Escaping & Security', () => {
it('escapes user names, asset names, and paths with special characters', () => {
const context: ShumaiMessageContext = {
user: {
id: 'u_1',
name: 'Alice & Bob <Admin>',
role: 'reviewer "lead"',
},
currentAsset: {
id: 'ast_1',
name: 'Final <Cut> "v2" & Test.mp4',
type: 'file',
path: 'Folder A > Folder B & C/clip.mp4',
ancestors: [{ id: 'anc_1', name: 'Folder B & C' }],
},
attachedFiles: [
{
id: 'att_1',
name: "Don't <Touch> & Go.pdf",
type: 'file',
path: "Path 'A' & 'B'/doc.pdf",
},
],
}
const xml = serializeContextToXml(context)
expect(xml).toContain('name="Alice &amp; Bob &lt;Admin&gt;"')
expect(xml).toContain('role="reviewer &quot;lead&quot;"')
expect(xml).toContain('name="Final &lt;Cut&gt; &quot;v2&quot; &amp; Test.mp4"')
expect(xml).toContain('path="Folder A &gt; Folder B &amp; C/clip.mp4"')
expect(xml).toContain('name="Folder B &amp; C"')
expect(xml).toContain('name="Don&apos;t &lt;Touch&gt; &amp; Go.pdf"')
expect(xml).toContain('path="Path &apos;A&apos; &amp; &apos;B&apos;/doc.pdf"')
// Verify no unescaped dangerous characters exist inside attribute quotes
const attrMatch = xml.match(/\w+="([^"]*)"/g)
expect(attrMatch).not.toBeNull()
})
})
describe('Order Stability & Prompt Caching Invariants', () => {
it('produces identical XML regardless of object key order', () => {
const contextA: ShumaiMessageContext = {
user: { role: 'editor', name: 'Alice', id: 'u_1' },
currentAsset: {
name: 'video.mp4',
type: 'file',
id: 'ast_1',
mediaType: 'video',
},
position: { seconds: 12.345, type: 'time' },
}
const contextB: ShumaiMessageContext = {
position: { type: 'time', seconds: 12.345 },
currentAsset: {
id: 'ast_1',
mediaType: 'video',
type: 'file',
name: 'video.mp4',
},
user: { id: 'u_1', name: 'Alice', role: 'editor' },
}
expect(serializeContextToXml(contextA)).toBe(serializeContextToXml(contextB))
})
it('sorts attachedFiles and referencedAssets by id ascending', () => {
const context: ShumaiMessageContext = {
attachedFiles: [
{ id: 'z_file', name: 'Z', type: 'file' },
{ id: 'a_file', name: 'A', type: 'file' },
{ id: 'm_file', name: 'M', type: 'file' },
],
referencedAssets: [
{ id: 'z_asset', name: 'Z', type: 'file' },
{ id: 'a_asset', name: 'A', type: 'folder' },
{ id: 'm_asset', name: 'M', type: 'file' },
],
}
const xml = serializeContextToXml(context)
const fileOrder = [...xml.matchAll(/<file id="([^"]+)"/g)].map((m) => m[1])
const refOrder = [...xml.matchAll(/<(?:asset|folder) id="([^"]+)"/g)].map((m) => m[1])
expect(fileOrder).toEqual(['a_file', 'm_file', 'z_file'])
expect(refOrder).toEqual(['a_asset', 'm_asset', 'z_asset'])
})
it('normalizes float formatting with fixed precision (toFixed(4) for position, toFixed(2) for duration)', () => {
const context1: ShumaiMessageContext = {
currentAsset: { id: '1', name: 'a', type: 'file', durationSeconds: 14 },
position: { type: 'time', seconds: 5.5 },
}
const context2: ShumaiMessageContext = {
currentAsset: { id: '1', name: 'a', type: 'file', durationSeconds: 14.0001 },
position: { type: 'time', seconds: 5.50002 },
}
expect(serializeContextToXml(context1)).toContain('duration_seconds="14.00"')
expect(serializeContextToXml(context1)).toContain('seconds="5.5000"')
expect(serializeContextToXml(context2)).toContain('duration_seconds="14.00"')
expect(serializeContextToXml(context2)).toContain('seconds="5.5000"')
})
it('serializes annotation tag with id when context.id is provided', () => {
const context: ShumaiMessageContext = {
id: '01JABCDEF1234567890',
annotation: true,
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <annotation id="01JABCDEF1234567890" />\n</context>',
)
})
it('escapes special XML characters in annotation id', () => {
const context: ShumaiMessageContext = {
id: 'test<id>&"123"',
annotation: true,
}
expect(serializeContextToXml(context)).toBe(
'<context>\n <annotation id="test&lt;id&gt;&amp;&quot;123&quot;" />\n</context>',
)
})
})
})
@@ -0,0 +1 @@
export { escapeXmlAttr, serializeContextToXml } from '@shumai/dtos'
@@ -7,8 +7,7 @@ import { DatabaseSessionStorage } from './database-session-storage'
import { agentService } from '@shumai/core/src/agent/agent'
import { createAgentSession, type DbProviderInfo } from './index'
import * as sandboxedBashModule from './tools/sandboxed-bash'
import * as analyzeImageModule from './tools/analyze-image'
import * as screenshotModule from './tools/screenshot'
import * as readAssetModule from './tools/read-asset'
describe('DatabaseSessionStorage', () => {
setupTestDbHooks()
@@ -451,9 +450,8 @@ describe('DatabaseSessionStorage', () => {
})
const sessionId = initialStorage.sessionId
// 2. Spy on media tools constructors
const createAnalyzeImageToolSpy = vi.spyOn(analyzeImageModule, 'createAnalyzeImageTool')
const createScreenshotToolSpy = vi.spyOn(screenshotModule, 'createScreenshotTool')
// 2. Spy on media tool constructor
const createReadAssetToolSpy = vi.spyOn(readAssetModule, 'createReadAssetTool')
// 3. Resume the session via createAgentSession with a NEW comment ID
await createAgentSession({
@@ -470,12 +468,11 @@ describe('DatabaseSessionStorage', () => {
providers: mockProviders,
})
// 4. Assert that the tools were instantiated using the new comment ID and user ID
expect(createAnalyzeImageToolSpy).toHaveBeenCalledWith(user.id, 'new-comment-456')
expect(createScreenshotToolSpy).toHaveBeenCalledWith(user.id, 'new-comment-456')
// 4. Assert that the tool was instantiated using user ID
expect(createReadAssetToolSpy).toHaveBeenCalledWith(user.id)
})
it('should use the current comment ID for media tools in subsequent turns (video)', async () => {
it('should instantiate media tools with user ID in subsequent turns (video)', async () => {
const { agent, user, team } = await setupTestData()
// Create a project and asset to satisfy FK constraints
@@ -505,9 +502,8 @@ describe('DatabaseSessionStorage', () => {
})
const sessionId = initialStorage.sessionId
// 2. Spy on media tools constructors
const createAnalyzeImageToolSpy = vi.spyOn(analyzeImageModule, 'createAnalyzeImageTool')
const createScreenshotToolSpy = vi.spyOn(screenshotModule, 'createScreenshotTool')
// 2. Spy on media tool constructor
const createReadAssetToolSpy = vi.spyOn(readAssetModule, 'createReadAssetTool')
// 3. Resume the session via createAgentSession with a NEW comment ID
await createAgentSession({
@@ -524,9 +520,8 @@ describe('DatabaseSessionStorage', () => {
providers: mockProviders,
})
// 4. Assert that the tools were instantiated using the new comment ID and user ID
expect(createScreenshotToolSpy).toHaveBeenCalledWith(user.id, 'new-comment-456')
expect(createAnalyzeImageToolSpy).toHaveBeenCalledWith(user.id, 'new-comment-456')
// 4. Assert that the tool was instantiated using user ID
expect(createReadAssetToolSpy).toHaveBeenCalledWith(user.id)
})
describe('P1 Bug Reproductions', () => {
@@ -662,5 +657,158 @@ describe('DatabaseSessionStorage', () => {
expect(stats.totalTokens).toBe(170)
expect(stats.costTotal).toBeCloseTo(0.032)
})
it('should save user message turn as custom_message (shumai_message) in DB when currentMessageContext is set', async () => {
const { agent, user } = await setupTestData()
const storage = await DatabaseSessionStorage.create({ agentId: agent.id, userId: user.id })
storage.currentMessageContext = {
user: { id: user.id, name: user.name, role: 'owner' },
currentAsset: { id: 'asset-123', name: 'video.mp4', type: 'file' },
}
const userEntry: SessionTreeEntry = {
type: 'message',
id: 'msg-user-1',
parentId: null,
timestamp: new Date().toISOString(),
message: {
role: 'user',
content: 'Hello, help me analyze this video',
timestamp: Date.now(),
},
}
await storage.appendEntry(userEntry)
// 1. Verify DB record type is 'custom_message', not 'message'
const record = await prisma.agentSessionEntry.findUnique({
where: { id: 'msg-user-1' },
})
expect(record).toBeDefined()
expect(record?.type).toBe('custom_message')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const data = record?.data as any
expect(data.customType).toBe('shumai_message')
expect(data.content).toBe('Hello, help me analyze this video')
expect(data.details?.user?.id).toBe(user.id)
expect(data.details?.currentAsset?.id).toBe('asset-123')
// 2. Verify getEntries() and getPathToRoot() do not throw and return proper custom_message
const entries = await storage.getEntries()
expect(entries).toHaveLength(1)
expect(entries[0].type).toBe('custom_message')
const pathEntries = await storage.getPathToRoot('msg-user-1')
expect(pathEntries).toHaveLength(1)
expect(pathEntries[0].type).toBe('custom_message')
const stats = await storage.getSessionStats()
expect(stats.messageCount).toBe(1)
})
it('should safely handle legacy records where type was message but data was shumai_message', async () => {
const { agent } = await setupTestData()
const storage = await DatabaseSessionStorage.create({ agentId: agent.id })
await prisma.agentSessionEntry.create({
data: {
id: 'legacy-corrupted-entry',
sessionId: storage.sessionId,
type: 'message',
data: {
customType: 'shumai_message',
content: 'Legacy prompt',
display: true,
details: { user: { id: 'u1', name: 'User 1' } },
},
},
})
// Must not throw in getEntries(), getPathToRoot(), or getSessionStats()
const entries = await storage.getEntries()
expect(entries).toHaveLength(1)
expect(entries[0].type).toBe('custom_message')
const path = await storage.getPathToRoot('legacy-corrupted-entry')
expect(path).toHaveLength(1)
expect(path[0].type).toBe('custom_message')
const stats = await storage.getSessionStats()
expect(stats.messageCount).toBe(1)
})
it('should dynamically inject thread metadata into details.thread in getPathToRoot when replies exist', async () => {
const { agent, user } = await setupTestData()
const project = await prisma.project.create({
data: { name: 'Test Project', teamId: agent.teamId },
})
const asset = await prisma.asset.create({
data: {
name: 'Test Asset',
type: 'file',
status: 'uploaded',
projectId: project.id,
},
})
const storage = await DatabaseSessionStorage.create({ agentId: agent.id, assetId: asset.id })
const rootComment = await prisma.assetComment.create({
data: {
id: 'comment-root-1',
assetId: asset.id,
message: 'Top-level question',
creatorId: user.id,
},
})
// Add a reply
await prisma.assetComment.create({
data: {
id: 'comment-reply-1',
assetId: asset.id,
message: 'Reply 1',
creatorId: user.id,
replyToId: rootComment.id,
},
})
// Create session entry for root comment
await prisma.agentSessionEntry.create({
data: {
id: rootComment.id,
sessionId: storage.sessionId,
assetId: asset.id,
type: 'custom_message',
data: {
customType: 'shumai_message',
content: 'Top-level question',
display: true,
details: { user: { id: user.id, name: user.name } },
},
},
})
const path = await storage.getPathToRoot(rootComment.id)
expect(path).toHaveLength(1)
expect(path[0].type).toBe('custom_message')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const details = (path[0] as any).details
expect(details.thread).toEqual({
id: rootComment.id,
replyCount: 1,
})
// eslint-disable-next-line @typescript-eslint/no-explicit-any
expect((path[0] as any).content).toBe('Top-level question')
const entries = await storage.getEntries()
expect(entries).toHaveLength(1)
expect(entries[0].type).toBe('custom_message')
// eslint-disable-next-line @typescript-eslint/no-explicit-any
expect((entries[0] as any).details?.thread).toEqual({
id: rootComment.id,
replyCount: 1,
})
})
})
})
+109 -54
View File
@@ -8,6 +8,7 @@ import {
type SessionTreeEntry,
} from '@earendil-works/pi-agent-core'
import { agentService } from '@shumai/core/src/agent/agent'
import type { ShumaiMessageContext } from '@shumai/dtos'
export interface DatabaseSessionMetadata extends SessionMetadata {
agentId: string
@@ -72,6 +73,7 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
}
nextEntryId?: string | null
currentMessageContext?: PrismaJson.ShumaiMessageContext
async createEntryId(): Promise<string> {
if (this.nextEntryId) {
@@ -90,9 +92,13 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
data: unknown
}): SessionTreeEntry {
const payload = (record.data as Record<string, unknown>) || {}
let type = (record.type || 'message') as SessionTreeEntry['type']
if (type === 'message' && payload.customType === 'shumai_message') {
type = 'custom_message'
}
return {
id: record.id,
type: (record.type || 'message') as SessionTreeEntry['type'],
type,
parentId: record.parentId,
timestamp: record.createdAt.toISOString(),
...payload,
@@ -100,10 +106,44 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
}
async appendEntry(entry: SessionTreeEntry): Promise<void> {
const strippedEntry = structuredClone(entry)
let entryToProcess = entry
if (
entryToProcess.type === 'message' &&
entryToProcess.message &&
entryToProcess.message.role === 'user' &&
this.currentMessageContext
) {
let contentText = ''
if (typeof entryToProcess.message.content === 'string') {
contentText = entryToProcess.message.content
} else if (Array.isArray(entryToProcess.message.content)) {
contentText = entryToProcess.message.content
.map((c) =>
typeof c === 'object' && c && 'text' in c && typeof c.text === 'string' ? c.text : '',
)
.join('')
}
entryToProcess = {
type: 'custom_message',
id: entryToProcess.id,
parentId: entryToProcess.parentId,
timestamp: entryToProcess.timestamp,
customType: 'shumai_message',
content: contentText,
display: true,
details: {
...this.currentMessageContext,
id: entryToProcess.id,
},
} as unknown as SessionTreeEntry
this.currentMessageContext = undefined
}
const strippedEntry = structuredClone(entryToProcess)
// Strip image data and skill content before saving to DB
if (strippedEntry.type === 'message') {
if (strippedEntry.type === 'message' && strippedEntry.message) {
const msg = strippedEntry.message
if (msg.role === 'toolResult') {
const details = msg.details as { sourceKeys?: string[]; skillId?: string } | undefined
@@ -139,7 +179,10 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
}
}
const targetLeafId = entry.type === 'leaf' ? entry.targetId : entry.id
const targetLeafId =
strippedEntry.type === 'leaf'
? (strippedEntry as unknown as { targetId?: string }).targetId || strippedEntry.id
: strippedEntry.id
// Extract base fields and store type-specific properties in data payload
const payload = { ...strippedEntry } as Record<string, unknown>
@@ -155,17 +198,18 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
await prisma.$transaction([
prisma.agentSessionEntry.upsert({
where: { id: entry.id },
where: { id: strippedEntry.id },
create: {
id: entry.id,
id: strippedEntry.id,
sessionId: this.sessionId,
assetId: session?.assetId || null,
type: entry.type,
parentId: entry.parentId || null,
createdAt: entry.timestamp ? new Date(entry.timestamp) : new Date(),
type: strippedEntry.type,
parentId: strippedEntry.parentId || null,
createdAt: strippedEntry.timestamp ? new Date(strippedEntry.timestamp) : new Date(),
data: payload as PrismaJson.PiSessionEntryData,
},
update: {
type: strippedEntry.type,
data: payload as PrismaJson.PiSessionEntryData,
},
}),
@@ -236,7 +280,9 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
let costTotal = 0
for (const entry of entries) {
if (entry.type === 'message') {
if (entry.type === 'custom_message' && entry.customType === 'shumai_message') {
messageCount++
} else if (entry.type === 'message' && entry.message) {
const msg = entry.message
if (msg.role === 'user' || msg.role === 'assistant') {
messageCount++
@@ -324,48 +370,7 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
await this.reinjectSkillContentAsync(entry)
}
// Dynamically tag comments that currently have reply threads
const entryIds = pathEntries.map((e) => e.id)
if (entryIds.length > 0) {
const threadCounts = await prisma.assetComment.groupBy({
by: ['replyToId'],
where: {
replyToId: { in: entryIds },
message: { not: '__CHAT__' },
},
_count: {
id: true,
},
})
const threadReplyCountMap = new Map<string, number>()
for (const tc of threadCounts) {
if (tc.replyToId) {
threadReplyCountMap.set(tc.replyToId, tc._count.id)
}
}
if (threadReplyCountMap.size > 0) {
for (let i = 0; i < pathEntries.length; i++) {
const entry = pathEntries[i]
const count = threadReplyCountMap.get(entry.id)
if (count !== undefined && count > 0 && entry.type === 'message') {
const cloned = structuredClone(entry)
const msg = cloned.message as unknown as {
content?: Array<{ type: string; text: string }>
}
if (Array.isArray(msg.content) && msg.content[0] && msg.content[0].type === 'text') {
const threadTag = `[Thread ID: ${entry.id}] [Replies: ${count}]`
if (!msg.content[0].text.includes(`[Thread ID: ${entry.id}]`)) {
msg.content[0].text = `${threadTag} ${msg.content[0].text}`
}
}
pathEntries[i] = cloned
}
}
}
}
return pathEntries
return this.injectThreadMetadataAsync(pathEntries)
}
async getEntries(): Promise<SessionTreeEntry[]> {
@@ -378,11 +383,61 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
await this.reinjectImageDataAsync(entry)
await this.reinjectSkillContentAsync(entry)
}
return this.injectThreadMetadataAsync(entries)
}
private async injectThreadMetadataAsync(
entries: SessionTreeEntry[],
): Promise<SessionTreeEntry[]> {
const entryIds = entries.map((e) => e.id)
if (entryIds.length === 0) {
return entries
}
const threadCounts = await prisma.assetComment.groupBy({
by: ['replyToId'],
where: {
replyToId: { in: entryIds },
message: { not: '__CHAT__' },
},
_count: {
id: true,
},
})
const threadReplyCountMap = new Map<string, number>()
for (const tc of threadCounts) {
if (tc.replyToId) {
threadReplyCountMap.set(tc.replyToId, tc._count.id)
}
}
if (threadReplyCountMap.size > 0) {
for (let i = 0; i < entries.length; i++) {
const entry = entries[i]
const count = threadReplyCountMap.get(entry.id)
if (count !== undefined && count > 0) {
if (entry.type === 'custom_message') {
const cloned = structuredClone(entry)
const existingDetails = ((cloned as unknown as { details?: ShumaiMessageContext })
.details || {}) as ShumaiMessageContext
;(cloned as unknown as { details: ShumaiMessageContext }).details = {
...existingDetails,
thread: {
id: entry.id,
replyCount: count,
},
}
entries[i] = cloned
}
}
}
}
return entries
}
private async reinjectImageDataAsync(entry: SessionTreeEntry) {
if (entry.type === 'message') {
if (entry.type === 'message' && entry.message) {
const msg = entry.message as AgentMessage
if (msg.role === 'toolResult') {
const details = msg.details as { sourceKeys?: string[] } | undefined
@@ -407,7 +462,7 @@ export class DatabaseSessionStorage implements SessionStorage<DatabaseSessionMet
}
private async reinjectSkillContentAsync(entry: SessionTreeEntry) {
if (entry.type === 'message') {
if (entry.type === 'message' && entry.message) {
const msg = entry.message as AgentMessage
if (msg.role === 'toolResult' && msg.toolName === 'read_skill') {
const details = msg.details as { skillId?: string } | undefined
+327 -1
View File
@@ -8,8 +8,11 @@ import {
} from './index'
import { createSandboxedBashTool } from './tools/sandboxed-bash'
import { SandboxManager, type SandboxAskCallback } from '@anthropic-ai/sandbox-runtime'
import { quotaService } from '@shumai/core/src/quota/quota-service'
import { prisma } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import { type TSchema } from 'typebox'
import { Value } from 'typebox/value'
import * as path from 'path'
import * as childProcess from 'node:child_process'
import * as fs from 'fs'
@@ -89,10 +92,16 @@ describe('Sandbox Network isolation integration', () => {
setupTestDbHooks()
it('should register callback that pushes blocked hosts to database and propagates to bash tool', async () => {
// 1. Create a team
// 1. Create a team and enabled sandbox
const team = await prisma.team.create({
data: { name: 'Test Sandbox Team' },
})
await prisma.sandbox.create({
data: {
teamId: team.id,
networkSandboxEnabled: true,
},
})
// Ensure agent user and agent exist in database to avoid foreign key violation
await prisma.user.create({
@@ -198,6 +207,12 @@ describe('Sandbox Network isolation integration', () => {
const team2 = await prisma.team.create({
data: { name: 'Team 2' },
})
await prisma.sandbox.create({
data: { teamId: team1.id, networkSandboxEnabled: true },
})
await prisma.sandbox.create({
data: { teamId: team2.id, networkSandboxEnabled: true },
})
// Ensure agent user and agents exist in database
await prisma.user.create({
@@ -290,6 +305,64 @@ describe('Sandbox Network isolation integration', () => {
}
})
it('should bypass domain blocking and pending domain recording when networkSandboxEnabled is false', async () => {
const team = await prisma.team.create({
data: { name: 'Disabled Sandbox Team' },
})
await prisma.sandbox.create({
data: { teamId: team.id, networkSandboxEnabled: false },
})
await prisma.user.create({
data: {
id: 'agent-disabled',
name: 'Ai Agent Disabled',
email: 'disabled@shumai.ai',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'agent-disabled',
teamId: team.id,
type: 'chat',
config: { provider: 'openai', model: 'gpt-4' },
},
})
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
vi.spyOn(SandboxManager, 'wrapWithSandbox').mockImplementation(async (cmd) => cmd)
await createAgentSession({
teamId: team.id,
agentId: 'agent-disabled',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: ['*'],
providers: mockProviders,
})
expect(initializeSpy).toHaveBeenCalledWith(
expect.objectContaining({
network: expect.objectContaining({
allowedDomains: ['*'],
}),
}),
expect.any(Function),
)
const callback = initializeSpy.mock.calls[0][1]
const res = await callback!({ host: 'any-external-domain.com', port: 443 })
expect(res).toBe(true)
const sandbox = await prisma.sandbox.findUnique({
where: { teamId: team.id },
})
expect(sandbox?.pendingDomains).toEqual([])
})
it('should pass thinkingLevel to AgentHarness and default to off', async () => {
const team = await prisma.team.create({
data: { name: 'Test Team' },
@@ -814,6 +887,149 @@ describe('createAgentSession bash restriction for non-owner users', () => {
})
})
describe('createAgentSession typed metadata schema', () => {
setupTestDbHooks()
afterEach(() => {
vi.restoreAllMocks()
})
async function setupSessionWithProject(opts: { withCreationFields: boolean }) {
const team = await prisma.team.create({ data: { name: 'Typed Metadata Team' } })
const project = await prisma.project.create({
data: { name: 'Typed Metadata Project', teamId: team.id },
})
const user = await prisma.user.create({
data: { name: 'Typed Metadata User', email: 'typed-metadata@shumai.ai' },
})
await prisma.teamMember.create({
data: { teamId: team.id, userId: user.id, role: 'owner' },
})
await prisma.user.create({
data: {
id: 'typed-metadata-agent',
name: 'Ai Agent Typed Metadata',
email: 'typed-metadata-agent@shumai.ai',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'typed-metadata-agent',
teamId: team.id,
type: 'chat',
config: { provider: 'google', model: 'gemini' },
},
})
if (opts.withCreationFields) {
await prisma.metadataField.create({
data: {
key: 'prompt',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Prompt', type: 'text', autofillSource: 'CREATION_CONTEXT' },
},
})
await prisma.metadataField.create({
data: {
key: 'source',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: {
name: 'Source',
type: 'select',
autofillSource: 'CREATION_CONTEXT',
select: {
options: [{ id: 'gemini', displayName: 'Gemini', color: '#ffffff' }],
},
},
},
})
await prisma.metadataField.create({
data: {
key: 'manualNotes',
scope: 'PROJECT',
projectId: project.id,
teamId: team.id,
config: { name: 'Manual Notes', type: 'text', autofillSource: 'NONE' },
},
})
}
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
try {
const { harness } = await createAgentSession({
teamId: team.id,
agentId: 'typed-metadata-agent',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [],
allowedDomains: [],
providers: mockProviders,
userId: user.id,
projectId: project.id,
})
return { harness }
} finally {
initializeSpy.mockRestore()
}
}
it('should build create_file/create_version with the project CREATION_CONTEXT fields', async () => {
const { harness } = await setupSessionWithProject({ withCreationFields: true })
const tools = harness.getTools()
expect(tools.some((t) => t.name === 'list_autofill_fields')).toBe(false)
const createFile = tools.find((t) => t.name === 'create_file')
const createVersion = tools.find((t) => t.name === 'create_version')
expect(createFile).toBeDefined()
expect(createVersion).toBeDefined()
const checkFile = (args: Record<string, unknown>) =>
Value.Check(createFile!.parameters as unknown as TSchema, {
parent: 'f1',
path: '/tmp/x.md',
data: null,
...args,
})
const checkVersion = (args: Record<string, unknown>) =>
Value.Check(createVersion!.parameters as unknown as TSchema, {
parent: 'f1',
path: '/tmp/x.md',
...args,
})
for (const check of [checkFile, checkVersion]) {
// Declared CREATION_CONTEXT fields (all-required, nullable) validate; other keys do not
expect(check({ metadata: { prompt: 'hello', source: null } })).toBe(true)
expect(check({ metadata: { prompt: null, source: 'gemini' } })).toBe(true)
expect(check({ metadata: null })).toBe(true)
expect(check({ metadata: { prompt: null, source: null, manualNotes: 'no' } })).toBe(false)
expect(check({ metadata: { prompt: null, source: null, bogus: 1 } })).toBe(false)
expect(check({ metadata: { prompt: 'hello' } })).toBe(false)
}
})
it('should omit the metadata parameter when the project has no CREATION_CONTEXT fields', async () => {
const { harness } = await setupSessionWithProject({ withCreationFields: false })
const createFile = harness.getTools().find((t) => t.name === 'create_file')
const createVersion = harness.getTools().find((t) => t.name === 'create_version')
expect(createFile).toBeDefined()
expect(createVersion).toBeDefined()
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
const fileProps = (createFile!.parameters as any).properties
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
const versionProps = (createVersion!.parameters as any).properties
expect('metadata' in fileProps).toBe(false)
expect('metadata' in versionProps).toBe(false)
})
})
describe('createAgentSession enabled skill filtering', () => {
setupTestDbHooks()
@@ -958,4 +1174,114 @@ describe('createAgentSession enabled skill filtering', () => {
initializeSpy.mockRestore()
}
})
it('enforces agent tool call count quota on tool execution', async () => {
const team = await prisma.team.create({ data: { name: 'Tool Quota Enforcement Team' } })
const user = await prisma.user.create({
data: { name: 'Member User', email: 'member-tool-quota@shumai.ai', password: 'pw' },
})
await prisma.teamMember.create({
data: { teamId: team.id, userId: user.id, role: 'editor' },
})
await prisma.user.create({
data: {
id: 'tool-quota-agent',
name: 'Tool Quota Agent',
email: 'tool-quota-agent@shumai.ai',
type: 'agent',
},
})
await prisma.agent.create({
data: {
id: 'tool-quota-agent',
teamId: team.id,
type: 'chat',
config: { provider: 'google', model: 'gemini' },
},
})
await quotaService.createRule(team.id, {
scopeMode: 'all_members',
resource: 'agent_tool_call_count',
resourceData: { name: 'read_thread' },
limit: 1,
period: '1hour',
enabled: true,
})
const initializeSpy = vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
try {
const { harness } = await createAgentSession({
teamId: team.id,
userId: user.id,
agentId: 'tool-quota-agent',
providerName: 'google',
modelId: 'gemini',
systemPrompt: 'prompt',
teamSkills: [],
enabledSkillIds: [],
allowedDomains: [],
providers: mockProviders,
})
const readThreadTool = harness.getTools().find((t) => t.name === 'read_thread')
expect(readThreadTool).toBeDefined()
// First call executes and consumes quota
const res1 = await readThreadTool!.execute(
'call-1',
{ threadId: 'non-existent' },
undefined,
undefined,
undefined,
)
expect(res1).toBeDefined()
// Second call exceeds quota and returns error
const res2 = await readThreadTool!.execute(
'call-2',
{ threadId: 'non-existent' },
undefined,
undefined,
undefined,
)
expect(res2).toBeDefined()
const textContent = res2.content[0]
if (textContent && textContent.type === 'text') {
expect(textContent.text).toContain('Quota exceeded')
} else {
expect.unreachable('Expected text content')
}
} finally {
initializeSpy.mockRestore()
}
})
})
describe('getModelFromDb', () => {
it('should prioritize provider active api protocol over model config api', () => {
const providers: DbProviderInfo[] = [
{
name: 'custom-provider',
config: { api: 'anthropic-messages', apiKey: 'KEY' },
models: [
{
modelId: 'my-model',
name: 'My Model',
config: {
api: 'openai-responses', // stale model config
reasoning: false,
input: ['text'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 128000,
maxTokens: 4096,
},
},
],
},
]
const model = getModelFromDb(providers, 'custom-provider', 'my-model')
expect(model.api).toBe('anthropic-messages')
})
})
+184 -67
View File
@@ -9,24 +9,27 @@ import type { Api, Model } from '@earendil-works/pi-ai'
import { createModels, InMemoryCredentialStore } from '@earendil-works/pi-ai'
import { builtinProviders } from '@earendil-works/pi-ai/providers/all'
import { agentService } from '@shumai/core/src/agent/agent'
import { prisma, type TeamMemberRole } from '@shumai/db'
import { Type, type TSchema } from '@sinclair/typebox'
import { resolveEffectiveRole } from '@shumai/core/src/authz/authz'
import { quotaService, QuotaExceededError } from '@shumai/core/src/quota/quota-service'
import { logger } from '@shumai/core/src/logger'
import { prisma } from '@shumai/db'
import { Type, type TSchema } from 'typebox'
import * as fs from 'fs'
import * as os from 'os'
import * as path from 'path'
import { DatabaseSessionStorage } from './database-session-storage'
import { createAnalyzeImageTool } from './tools/analyze-image'
import { metadataService } from '@shumai/core/src/metadata/metadata'
import { createCreateFileTool } from './tools/create-file'
import { createListAutofillFieldsTool } from './tools/list-autofill-fields'
import { createCreateFolderTool } from './tools/create-folder'
import { createCreateVersionTool } from './tools/create-version'
import { createDownloadAssetTool } from './tools/download-asset'
import { createListAssetsTool } from './tools/list-assets'
import { createReadPdfPagesTool } from './tools/read-pdf-pages'
import { createReadAssetTool } from './tools/read-asset'
import { createReadSkillTool } from './tools/read-skill'
import { createReadThreadTool } from './tools/read-thread'
import { createSandboxedBashTool } from './tools/sandboxed-bash'
import { createScreenshotTool } from './tools/screenshot'
import { serializeContextToXml } from './context/serialize-context'
import type { ShumaiMessageContext } from '@shumai/dtos'
export interface DbModelInfo {
modelId: string
@@ -67,10 +70,10 @@ export function getModelFromDb(providers: DbProviderInfo[], providerName: string
if (dbProvider.config.baseUrl) {
m.baseUrl = dbProvider.config.baseUrl
}
Object.assign(m, dbModel.config)
if (dbProvider.config.api) {
m.api = dbProvider.config.api
}
Object.assign(m, dbModel.config)
if (dbModel.name) {
m.name = dbModel.name
}
@@ -102,6 +105,7 @@ export interface CreateAgentSessionParams {
allowedDomains: string[]
sessionId?: string
userId?: string
projectId?: string
userCommentId?: string | null
customTools?: AgentTool[]
thinkingLevel?: string
@@ -111,18 +115,6 @@ export interface CreateAgentSessionParams {
baseDelayMs?: number
}
async function resolveTeamMemberRole(
teamId: string,
userId?: string,
): Promise<TeamMemberRole | undefined> {
if (!userId) return undefined
const member = await prisma.teamMember.findUnique({
where: { teamIdUserId: { teamId, userId } },
select: { role: true },
})
return member?.role
}
/**
* Security instructions appended to the system prompt when the requesting user is not a team
* owner (i.e. editor/reviewer). These complement the hard enforcement in the bash tool itself
@@ -150,6 +142,7 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
allowedDomains,
sessionId,
userId,
projectId,
userCommentId: passedUserCommentId,
customTools = [],
thinkingLevel,
@@ -183,11 +176,17 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
blockedHost: '',
}
const role = userId ? await resolveEffectiveRole(teamId, projectId, userId) : undefined
const isOwner = !userId || role === 'owner'
const restricted = !isOwner
const sandboxAskCallback = async ({ host }: { host: string; port?: number }) => {
try {
const sandbox = await prisma.sandbox.findUnique({
where: { teamId },
})
if (sandbox && !sandbox.networkSandboxEnabled) {
return true
}
const pendingDomains = sandbox?.pendingDomains || []
if (!pendingDomains.includes(host)) {
await prisma.sandbox.upsert({
@@ -212,6 +211,54 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
return false
}
const wrapToolWithQuota = (tool: AgentTool): AgentTool => {
const originalExecute = tool.execute
return {
...tool,
execute: async (toolCallId, args, signal, onUpdate) => {
try {
await quotaService.checkQuota(
{
teamId,
userId,
role,
resource: 'agent_tool_call_count',
resourceData: { name: tool.name, toolName: tool.name },
},
1,
)
} catch (err) {
if (err instanceof QuotaExceededError) {
return {
content: [{ type: 'text', text: `Quota exceeded: ${err.message}` }],
details: { error: err.message },
}
}
throw err
}
const result = await originalExecute(toolCallId, args, signal, onUpdate)
try {
await quotaService.consumeQuota(
{
teamId,
userId,
role,
resource: 'agent_tool_call_count',
resourceData: { name: tool.name, toolName: tool.name },
},
1,
)
} catch (err) {
logger.error({ err }, 'Failed to record quota usage for agent tool call')
}
return result
},
}
}
// SandboxManager.initialize is a global operation that applies to the entire process.
// We reset it first to ensure any previous process-global SOCKS/HTTP proxy servers
// and callbacks are cleaned up, allowing the new callback and allowedDomains to take effect.
@@ -237,13 +284,6 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
Object.assign(skillEnvs, envs)
}
// Resolve the requesting user's team role to enforce owner-only bash privileges.
// A user who is not a member of the team is treated as restricted (fail-closed),
// while flows without a user context (e.g. autofill) remain unrestricted.
const role = await resolveTeamMemberRole(teamId, userId)
const isOwner = !userId || role === 'owner'
const restricted = !isOwner
// Restore env variables from previously loaded skills in this session
let skillLoadedInSession = false
if (sessionId) {
@@ -252,6 +292,7 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
for (const entry of entries) {
if (
entry.type === 'message' &&
entry.message &&
entry.message.role === 'toolResult' &&
entry.message.toolName === 'read_skill' &&
!entry.message.isError
@@ -269,17 +310,9 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
}
}
const metadata = await storage.getMetadata()
const userCommentId =
passedUserCommentId !== undefined ? passedUserCommentId : metadata.userCommentId
const mediaTools: AgentTool[] = []
if (userId) {
mediaTools.push(
createAnalyzeImageTool(userId, userCommentId),
createScreenshotTool(userId, userCommentId),
createReadPdfPagesTool(userId, userCommentId),
)
mediaTools.push(createReadAssetTool(userId))
}
const agent = await prisma.agent.findUnique({
@@ -289,6 +322,9 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
const deniedTools = agentConfig?.deniedTools || []
const sandboxedBash = createSandboxedBashTool(process.cwd(), skillEnvs, {
teamId,
userId,
role,
getBlockedHost: () => sandboxState.blockedHost,
clearBlockedHost: () => {
sandboxState.blockedHost = ''
@@ -309,7 +345,7 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
bashInjected = true
// Pass the full post-update tool list as activeToolNames: setTools only updates the
// registry otherwise, and the model would never see the bash tool on the next turn.
const next = [...harness.getTools(), sandboxedBash]
const next = [...harness.getTools(), wrapToolWithQuota(sandboxedBash)]
await harness.setTools(
next,
next.map((t) => t.name),
@@ -320,22 +356,23 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
onEnvsAdded,
restricted ? onSkillLoaded : undefined,
enabledSkillIds,
projectId,
)
const systemTools: AgentTool[] = []
if (userId) {
const metadataSchema = await buildProjectMetadataSchema(projectId)
systemTools.push(
createListAssetsTool(userId),
createCreateFolderTool(userId),
createCreateFileTool(userId),
createCreateVersionTool(userId),
createCreateFileTool(userId, metadataSchema),
createCreateVersionTool(userId, metadataSchema),
createDownloadAssetTool(userId),
createListAutofillFieldsTool(userId),
)
}
const readThread = createReadThreadTool()
const allTools = [
const rawTools = [
...mediaTools,
readSkill,
readThread,
@@ -343,6 +380,7 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
...systemTools,
...customTools,
]
const allTools = rawTools.map((tool) => wrapToolWithQuota(tool))
const enabledTools = disableTools
? []
: allTools.filter((tool) => !deniedTools.includes(tool.name))
@@ -400,6 +438,29 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
tools: enabledTools,
})
harness.on('context', async ({ messages }) => {
return {
messages: messages.map((msg) => {
if (
msg.role === 'custom' &&
msg.customType === 'shumai_message' &&
msg.details &&
typeof msg.details === 'object'
) {
const context = msg.details as ShumaiMessageContext
const xml = serializeContextToXml(context)
const text = typeof msg.content === 'string' ? msg.content : ''
const fullContent = xml ? `${text}\n\n${xml}`.trim() : text
return {
...msg,
content: fullContent,
}
}
return msg
}),
}
})
return { session, harness }
}
@@ -410,58 +471,116 @@ export interface AutofillField {
}
export function fieldsToTypeBoxSchema(fields: AutofillField[]) {
const newOptionSchema = Type.Object(
{
newOption: Type.Object(
{
value: Type.String({
minLength: 1,
description:
'The new option display name/value to create and select if none of the existing options match.',
}),
},
{ additionalProperties: false },
),
},
{ additionalProperties: false },
)
const properties: Record<string, TSchema> = {}
for (const f of fields) {
let schema: TSchema
const fieldName = f.config.name
const fieldDesc = f.description || fieldName
const baseDescription = `The field '${fieldName}' represents ${fieldDesc}.`
switch (f.config.type) {
case 'text':
case 'longText':
schema = Type.String()
schema = Type.String({ description: baseDescription })
break
case 'number':
case 'rating':
schema = Type.Number()
schema = Type.Number({ description: baseDescription })
break
case 'toggle':
schema = Type.Boolean()
schema = Type.Boolean({ description: baseDescription })
break
case 'select': {
const options = f.config.select?.options || []
schema = Type.String({
enum: options.map((o) => o.id),
})
const description =
options.length > 0
? `${baseDescription}\nSelect one existing option ID or provide {"newOption": {"value": "..."}} to create a new option.\n\nAvailable options:\n${options.map((o) => `- ${o.displayName} => ${o.id}`).join('\n')}`
: `${baseDescription}\nProvide {"newOption": {"value": "..."}} with the desired option name.`
if (options.length > 0) {
const optionLines = options.map((o) => `- ${o.displayName} => ${o.id}`).join('\n')
schema.description = `The field '${fieldName}' represents ${fieldDesc}.\nSelect one option and return the option ID as the value.\n\nAvailable options:\n${optionLines}`
schema = Type.Union(
[
Type.String({
enum: options.map((o) => o.id),
}),
newOptionSchema,
],
{ description },
)
} else {
schema = Type.Object(newOptionSchema.properties, {
additionalProperties: false,
description,
})
}
break
}
case 'selectMulti': {
const options = f.config.selectMulti?.options || []
schema = Type.Array(
Type.String({
enum: options.map((o) => o.id),
}),
)
if (options.length > 0) {
const optionLines = options.map((o) => `- ${o.displayName} => ${o.id}`).join('\n')
schema.description = `The field '${fieldName}' represents ${fieldDesc}.\nSelect applicable options and return the option IDs as the value.\n\nAvailable options:\n${optionLines}`
}
const description =
options.length > 0
? `${baseDescription}\nSelect applicable option IDs or provide {"newOption": {"value": "..."}} objects for new options (e.g. ["opt1", {"newOption": {"value": "new_name"}}]).\n\nAvailable options:\n${options.map((o) => `- ${o.displayName} => ${o.id}`).join('\n')}`
: `${baseDescription}\nProvide an array of {"newOption": {"value": "..."}} objects for new options.`
const itemSchema =
options.length > 0
? Type.Union([
Type.String({
enum: options.map((o) => o.id),
}),
newOptionSchema,
])
: newOptionSchema
schema = Type.Array(itemSchema, { description })
break
}
default:
schema = Type.String()
schema = Type.String({ description: baseDescription })
}
if (!schema.description) {
schema.description = `The field '${fieldName}' represents ${fieldDesc}.`
}
properties[f.id] = schema
// Every field is required but nullable: the agent must return each field key,
// using `null` for values it does not know. Nulls are skipped when persisting.
// This matches the OpenAI structured-outputs strict-mode convention (all
// properties required, optionality expressed via `null`).
properties[f.id] = Type.Union([schema, Type.Null()])
}
return Type.Object(properties)
return Type.Object(properties, { additionalProperties: false })
}
/**
* Builds the strict, all-optional metadata schema for the project's CREATION_CONTEXT
* fields (the same field set the agent may attach when creating files/versions).
* Returns undefined when no project is known or the project defines no fields, in
* which case the `metadata` parameter is omitted from the tools entirely.
*/
async function buildProjectMetadataSchema(projectId?: string): Promise<TSchema | undefined> {
if (!projectId) return undefined
const fields = await metadataService.listProjectFields('', projectId)
const creationFields: AutofillField[] = []
for (const f of fields) {
const config = f.field.config
if (!config || config.autofillSource !== 'CREATION_CONTEXT') continue
creationFields.push({
id: f.field.key,
config,
description: f.field.description,
})
}
if (creationFields.length === 0) return undefined
return fieldsToTypeBoxSchema(creationFields)
}
function escapeXml(str: string): string {
@@ -510,7 +629,6 @@ import * as aiActivities from './activities/ai'
import { agentAutofillMedia } from './workflows/agent-autofill'
import { agentChat } from './workflows/agent-chat'
import { agentEmbeddingMedia } from './workflows/agent-embedding'
import { agentToolCall } from './workflows/agent-tool-call'
import { queryEmbeddingForSearch } from './workflows/query-embedding-for-search'
export function initAgentWorkflows() {
@@ -518,7 +636,6 @@ export function initAgentWorkflows() {
registerWorkflow(WorkflowTaskType.query_embedding_for_search, queryEmbeddingForSearch)
registerWorkflow(WorkflowTaskType.ai_metadata_autofill, agentAutofillMedia)
registerWorkflow(WorkflowTaskType.chat, agentChat)
registerWorkflow(WorkflowTaskType.agent_tool_call, agentToolCall)
registerActivities(agentActivities)
registerActivities(aiActivities)
@@ -537,8 +654,8 @@ export {
type GenerateTextEmbeddingParams,
} from './activities/ai'
export * from './database-session-storage'
export * from './context/serialize-context'
export * from './workflows/agent-autofill'
export * from './workflows/agent-chat'
export * from './workflows/agent-embedding'
export * from './workflows/agent-tool-call'
export * from './workflows/query-embedding-for-search'
@@ -0,0 +1,117 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { validateToolArguments } from '@earendil-works/pi-ai'
import { createCreateFileTool } from './tools/create-file'
import { fieldsToTypeBoxSchema } from './index'
import { s3Service } from '@shumai/core/src/s3/s3'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
uploadFileToKey: vi.fn(),
putObject: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
Permission: { Edit: 'edit', Read: 'read' },
ResourceType: { Asset: 'asset' },
}))
vi.mock('@shumai/core/src/asset/asset', () => ({
assetService: {
createFile: vi
.fn()
.mockResolvedValue({ id: 'file-1', name: 'test', type: 'file', sizeByte: 100 }),
},
}))
/**
* Reproduce: pi-ai <= 0.82.1 runs its lenient coercion on typebox v1 schemas and
* converts `null` in a `[string, null]` union into `""` (see
* coerceWithUnionSchema in @earendil-works/pi-ai/dist/utils/validation.js).
*
* When the agent calls create_file with `path: null, data: {...}`, the harness
* validates first: the `null` path becomes `""`, so the tool's
* assertExactlyOneSource believes BOTH path and data were provided and throws
* "Provide exactly one of ..." even though the call is valid.
*
* Fixed upstream in pi-ai 0.84.0 (#7328): values that already match a union arm
* are preserved before coercion. This test should pass once @earendil-works/pi-ai
* is upgraded to >= 0.84.0.
*/
describe('reproduce null coercion in harness tool validation', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('keeps path: null when data is provided (no false "exactly one" error)', async () => {
const tool = createCreateFileTool('user-1')
const validated = validateToolArguments(
{
name: 'create_file',
description: '',
parameters: tool.parameters,
} as Parameters<typeof validateToolArguments>[0],
{
type: 'toolCall',
id: 'call-1',
name: 'create_file',
arguments: {
parent: '01KYKNP52YERGJ6Q4GTPF5FRQF',
path: null,
data: { name: 'random_text.txt', content: 'hello' },
},
} as Parameters<typeof validateToolArguments>[1],
) as { path?: unknown; data?: unknown }
// The `null` path must survive validation untouched; coercion to "" makes
// the tool think both path and data were provided.
expect(validated.path).toBeNull()
expect(validated.data).toEqual({ name: 'random_text.txt', content: 'hello' })
// execute must accept the valid call (data-only) without throwing
const result = await tool.execute('call-1', validated as never)
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
id: 'file-1',
name: 'test',
type: 'file',
size: 100,
})
expect(s3Service.putObject).toHaveBeenCalledTimes(1)
expect(authzService.hasPermission).toHaveBeenCalledTimes(1)
expect(assetService.createFile).toHaveBeenCalledTimes(1)
})
it('keeps null metadata values (the "unknown" sentinel) through validation', () => {
const tool = createCreateFileTool(
'user-1',
fieldsToTypeBoxSchema([{ id: 'prompt', config: { name: 'Prompt', type: 'text' } }]),
)
const validated = validateToolArguments(
{
name: 'create_file',
description: '',
parameters: tool.parameters,
} as Parameters<typeof validateToolArguments>[0],
{
type: 'toolCall',
id: 'call-1',
name: 'create_file',
arguments: {
parent: 'folder-1',
path: null,
data: { name: 'a.md', content: 'x' },
metadata: { prompt: null },
},
} as Parameters<typeof validateToolArguments>[1],
) as { metadata?: Record<string, unknown> }
expect(validated.metadata).toEqual({ prompt: null })
})
})
+104 -18
View File
@@ -1,6 +1,22 @@
import { describe, it, expect } from 'vitest'
import { type TSchema } from 'typebox'
import { Value } from 'typebox/value'
import { fieldsToTypeBoxSchema, type AutofillField } from './index'
interface AnyOfFieldSchema {
type?: string
enum?: string[]
const?: string
description?: string
items?: { enum?: string[]; type?: string }
anyOf?: AnyOfFieldSchema[]
}
function propsOf(schema: TSchema): Record<string, { anyOf?: AnyOfFieldSchema[] }> {
return (schema as unknown as { properties: Record<string, { anyOf?: AnyOfFieldSchema[] }> })
.properties
}
describe('fieldsToTypeBoxSchema', () => {
it('converts text fields', () => {
const fields: AutofillField[] = [
@@ -8,10 +24,12 @@ describe('fieldsToTypeBoxSchema', () => {
{ id: 'f2', config: { name: 'Desc', type: 'longText' } },
]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('string')
expect(schema.properties.f1.description).toBe("The field 'Title' represents The title.")
expect(schema.properties.f2.type).toBe('string')
expect(schema.properties.f2.description).toBe("The field 'Desc' represents Desc.")
const props = propsOf(schema)
expect(props.f1.anyOf![0].type).toBe('string')
expect(props.f1.anyOf![1].type).toBe('null')
expect(props.f1.anyOf![0].description).toBe("The field 'Title' represents The title.")
expect(props.f2.anyOf![0].type).toBe('string')
expect(props.f2.anyOf![0].description).toBe("The field 'Desc' represents Desc.")
})
it('converts numeric fields', () => {
@@ -20,17 +38,18 @@ describe('fieldsToTypeBoxSchema', () => {
{ id: 'f2', config: { name: 'Rating', type: 'rating' } },
]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('number')
expect(schema.properties.f2.type).toBe('number')
const props = propsOf(schema)
expect(props.f1.anyOf![0].type).toBe('number')
expect(props.f2.anyOf![0].type).toBe('number')
})
it('converts toggle fields', () => {
const fields: AutofillField[] = [{ id: 'f1', config: { name: 'Active', type: 'toggle' } }]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('boolean')
expect(propsOf(schema).f1.anyOf![0].type).toBe('boolean')
})
it('converts select fields with enums and option descriptions', () => {
it('converts select fields with enums, newOption support and option descriptions', () => {
const fields: AutofillField[] = [
{
id: 'f1',
@@ -48,14 +67,41 @@ describe('fieldsToTypeBoxSchema', () => {
},
]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('string')
expect(schema.properties.f1.enum).toEqual(['opt1', 'opt2'])
expect(schema.properties.f1.description).toBe(
"The field 'Species' represents Species of animal.\nSelect one option and return the option ID as the value.\n\nAvailable options:\n- Option 1 => opt1\n- Option 2 => opt2",
const selectSchema = propsOf(schema).f1.anyOf![0]
expect(selectSchema.anyOf![0].type).toBe('string')
expect(selectSchema.anyOf![0].enum).toEqual(['opt1', 'opt2'])
expect(selectSchema.anyOf![1].type).toBe('object')
expect(selectSchema.description).toBe(
'The field \'Species\' represents Species of animal.\nSelect one existing option ID or provide {"newOption": {"value": "..."}} to create a new option.\n\nAvailable options:\n- Option 1 => opt1\n- Option 2 => opt2',
)
// Declared option IDs, newOption objects, and null are valid
expect(Value.Check(schema, { f1: 'opt1' })).toBe(true)
expect(Value.Check(schema, { f1: 'opt2' })).toBe(true)
expect(Value.Check(schema, { f1: { newOption: { value: 'kling' } } })).toBe(true)
expect(Value.Check(schema, { f1: null })).toBe(true)
// Arbitrary strings or invalid objects are rejected
expect(Value.Check(schema, { f1: 'not-an-option' })).toBe(false)
expect(Value.Check(schema, { f1: { newOption: { foo: 'bar' } } })).toBe(false)
})
it('converts selectMulti fields with array enums and option descriptions', () => {
it('converts select fields with empty options to newOption schema only', () => {
const fields: AutofillField[] = [
{
id: 'f1',
config: {
name: 'Provider',
type: 'select',
select: { options: [] },
},
},
]
const schema = fieldsToTypeBoxSchema(fields)
expect(Value.Check(schema, { f1: { newOption: { value: 'kling' } } })).toBe(true)
expect(Value.Check(schema, { f1: null })).toBe(true)
expect(Value.Check(schema, { f1: 'any-string' })).toBe(false)
})
it('converts selectMulti fields with array enums, newOption support and option descriptions', () => {
const fields: AutofillField[] = [
{
id: 'f1',
@@ -72,11 +118,35 @@ describe('fieldsToTypeBoxSchema', () => {
},
]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('array')
expect(schema.properties.f1.items.enum).toEqual(['opt1', 'opt2'])
expect(schema.properties.f1.description).toBe(
"The field 'Tags' represents Tags.\nSelect applicable options and return the option IDs as the value.\n\nAvailable options:\n- Tag 1 => opt1\n- Tag 2 => opt2",
const selectMultiSchema = propsOf(schema).f1.anyOf![0]
expect(selectMultiSchema.type).toBe('array')
expect(selectMultiSchema.description).toBe(
'The field \'Tags\' represents Tags.\nSelect applicable option IDs or provide {"newOption": {"value": "..."}} objects for new options (e.g. ["opt1", {"newOption": {"value": "new_name"}}]).\n\nAvailable options:\n- Tag 1 => opt1\n- Tag 2 => opt2',
)
expect(Value.Check(schema, { f1: ['opt1', 'opt2'] })).toBe(true)
expect(Value.Check(schema, { f1: [{ newOption: { value: 'Tag 3' } }] })).toBe(true)
expect(Value.Check(schema, { f1: ['opt1', { newOption: { value: 'Tag 3' } }] })).toBe(true)
expect(Value.Check(schema, { f1: null })).toBe(true)
expect(Value.Check(schema, { f1: ['opt1', 'bad'] })).toBe(false)
expect(Value.Check(schema, { f1: ['opt1', { newOption: { bad: 'value' } }] })).toBe(false)
})
it('converts selectMulti fields with empty options to array of newOption schema', () => {
const fields: AutofillField[] = [
{
id: 'f1',
config: {
name: 'Tags',
type: 'selectMulti',
selectMulti: { options: [] },
},
},
]
const schema = fieldsToTypeBoxSchema(fields)
expect(Value.Check(schema, { f1: [{ newOption: { value: 'Tag 1' } }] })).toBe(true)
expect(Value.Check(schema, { f1: [] })).toBe(true)
expect(Value.Check(schema, { f1: null })).toBe(true)
expect(Value.Check(schema, { f1: ['tag1'] })).toBe(false)
})
it('defaults to string for unknown types', () => {
@@ -85,6 +155,22 @@ describe('fieldsToTypeBoxSchema', () => {
{ id: 'f1', config: { name: 'Unknown', type: 'invalid' } },
]
const schema = fieldsToTypeBoxSchema(fields)
expect(schema.properties.f1.type).toBe('string')
expect(propsOf(schema).f1.anyOf![0].type).toBe('string')
})
it('marks every property required (nullable) and rejects unknown keys', () => {
const fields: AutofillField[] = [
{ id: 'f1', config: { name: 'Title', type: 'text' } },
{ id: 'f2', config: { name: 'Rating', type: 'rating' } },
]
const schema = fieldsToTypeBoxSchema(fields)
const props = propsOf(schema)
// typebox v1 objects are strict by default: unknown keys are rejected
expect(props.f1.anyOf![0].type).toBe('string')
expect(props.f1.anyOf![1].type).toBe('null')
expect(Value.Check(schema, { f1: 'x', f2: null })).toBe(true)
expect(Value.Check(schema, { f1: null, f2: 3 })).toBe(true)
expect(Value.Check(schema, { f1: 'x' })).toBe(false)
expect(Value.Check(schema, { f1: 'x', f2: null, extra: 1 })).toBe(false)
})
})
@@ -1,175 +0,0 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createAnalyzeImageTool } from './analyze-image'
import {
prisma,
WorkflowTaskStatus,
type Asset,
type AssetComment,
type User,
type WorkflowTask,
} from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/db', async (importOriginal) => {
const actual = await importOriginal<typeof import('@shumai/db')>()
return {
...actual,
prisma: {
user: {
findUnique: vi.fn(),
},
asset: {
findUnique: vi.fn(),
},
assetComment: {
findUnique: vi.fn(),
},
workflowTask: {
create: vi.fn(),
},
},
}
})
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
getObject: vi.fn(),
},
}))
vi.mock('@shumai/workflow-core', () => ({
workflowService: {
executeWait: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn(),
},
Permission: { Read: 'Read' },
ResourceType: { Asset: 'asset' },
}))
describe('analyzeImageTool', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('should check user permissions and fetch the image from S3 when comment has no annotations', async () => {
vi.mocked(prisma.user.findUnique).mockResolvedValue({ id: 'user-1' } as User)
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
storageKey: { key: 'raw-image.png' },
media: null,
} as unknown as Asset)
vi.mocked(prisma.assetComment.findUnique).mockResolvedValue({
id: 'comment-1',
assetId: 'asset-1',
annotation: null,
} as unknown as AssetComment)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-image-bytes'),
contentType: 'image/png',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createAnalyzeImageTool('user-1', 'comment-1')
const result = await tool.execute('call-1', { assetId: 'asset-1' })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-1',
})
expect(result.content[0].type).toBe('image')
expect((result.content[0] as unknown as { data: string }).data).toBe(
Buffer.from('fake-image-bytes').toString('base64'),
)
expect(result.details.sourceKeys).toEqual(['raw-image.png'])
expect(prisma.workflowTask.create).not.toHaveBeenCalled()
})
it('should trigger overlay transcode workflow when comment has annotations', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
storageKey: { key: 'raw-image.png' },
media: {
imageTranscodes: [{ key: 'transcoded-image.webp' }],
},
} as unknown as Asset)
vi.mocked(prisma.assetComment.findUnique).mockResolvedValue({
id: 'comment-1',
assetId: 'asset-1',
annotation: [
{
type: 'box',
color: '#ff0000',
points: [
[0, 0],
[1, 1],
],
},
],
} as unknown as AssetComment)
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-1',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-1',
status: WorkflowTaskStatus.completed,
output: { key: 'annotated-image.webp' },
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-annotated-image-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createAnalyzeImageTool('user-1', 'comment-1')
const result = await tool.execute('call-1', { assetId: 'asset-1' })
expect(prisma.workflowTask.create).toHaveBeenCalledWith({
data: {
assetId: 'asset-1',
projectId: 'project-1',
type: 'transcode_image_annotation',
status: 'pending',
payload: {
projectId: 'project-1',
imageAnnotation: {
annotations: [
{
type: 'box',
color: '#ff0000',
points: [
[0, 0],
[1, 1],
],
},
],
},
},
},
})
expect(result.content[0].type).toBe('image')
expect((result.content[0] as unknown as { data: string }).data).toBe(
Buffer.from('fake-annotated-image-bytes').toString('base64'),
)
expect(result.details.sourceKeys).toEqual(['annotated-image.webp'])
})
})
-139
View File
@@ -1,139 +0,0 @@
import { Type } from '@sinclair/typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { type ImageContent } from '@earendil-works/pi-ai'
import { prisma, WorkflowTaskType, WorkflowTaskStatus, type User } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
const analyzeImageSchema = Type.Object({
assetId: Type.String({
description: 'The asset ID of the image to analyze. This parameter is required.',
}),
})
export function createAnalyzeImageTool(
userId: string,
userCommentId?: string | null,
): AgentTool<typeof analyzeImageSchema> {
return {
name: 'analyze_image',
label: 'Analyze Image',
description:
'Retrieves and views the image content. Call this tool if you need to analyze the image.',
parameters: analyzeImageSchema,
execute: async (_toolCallId, params) => {
const assetId = params.assetId
if (!userId) {
throw new Error('User ID is required for authorization.')
}
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: assetId,
})
let asset = await prisma.asset.findUnique({
where: { id: assetId },
include: { storageKey: true },
})
if (!asset) {
throw new Error(`Asset with ID ${assetId} not found.`)
}
if (asset.type === 'version_stack') {
const latestVersion = await prisma.asset.findFirst({
where: { parentId: asset.id, isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
})
if (latestVersion) {
asset = latestVersion
}
}
// Get default media key
let mediaKey = asset.storageKey?.key
if (asset.media) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const mediaInfo = asset.media as any
if (mediaInfo.imageTranscodes && mediaInfo.imageTranscodes.length > 0) {
mediaKey = mediaInfo.imageTranscodes[0].key || mediaKey
}
}
if (!mediaKey) {
throw new Error('No media content found for this asset.')
}
// Check if trigger comment has draw annotations for this asset
let annotations: unknown = null
if (userCommentId) {
const comment = await prisma.assetComment.findUnique({
where: { id: userCommentId },
})
if (comment && comment.assetId === assetId && comment.annotation) {
const list = comment.annotation
if (Array.isArray(list) && list.length > 0) {
annotations = list
}
}
}
let keyToUse = mediaKey
if (annotations) {
// Trigger transcode workflow to draw annotation
const task = await prisma.workflowTask.create({
data: {
assetId,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_image_annotation,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
imageAnnotation: {
annotations: annotations as PrismaJson.AnnotationList,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const output = completedTask.output as any
if (output?.key) {
keyToUse = output.key
} else {
throw new Error('Image annotation transcode workflow failed to output S3 key.')
}
}
// Retrieve the image content from S3
const bucket = process.env.S3_BUCKET || 'shumai'
const { buffer, contentType } = await s3Service.getObject(bucket, keyToUse)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
const content: ImageContent[] = [
{
type: 'image',
data: buffer.toString('base64'),
mimeType,
},
]
return {
content,
details: {
sourceKeys: [keyToUse],
},
}
},
}
}
@@ -0,0 +1,270 @@
import { describe, it, expect, beforeEach } from 'vitest'
import { prisma } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import { resolveAnnotationsById } from './annotation-resolver'
import { DatabaseSessionStorage } from '../database-session-storage'
import type { SessionTreeEntry } from '@earendil-works/pi-agent-core'
describe('resolveAnnotationsById', () => {
setupTestDbHooks()
beforeEach(() => {
// DB hooks automatically manage transactions
})
it('returns null annotations and null timestamp when annotationId is undefined or empty', async () => {
const result1 = await resolveAnnotationsById('asset-1', undefined)
expect(result1).toEqual({ annotations: null, timestamp: null })
const result2 = await resolveAnnotationsById('asset-1', '')
expect(result2).toEqual({ annotations: null, timestamp: null })
})
it('resolves annotations and timestamp from AssetComment (Comments Mode)', async () => {
const user = await prisma.user.create({
data: { name: 'Comment Author', email: 'author@test.com', password: 'pw' },
})
const team = await prisma.team.create({
data: { name: 'Test Team' },
})
const project = await prisma.project.create({
data: { name: 'Test Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'video.mp4',
type: 'file',
projectId: project.id,
status: 'uploaded',
},
})
const comment = await prisma.assetComment.create({
data: {
assetId: asset.id,
creatorId: user.id,
message: 'Look at the highlighted area',
second: 18.5,
annotation: [
{
type: 'line',
color: '#ff0000',
points: [
[10, 10],
[50, 50],
],
},
],
},
})
const result = await resolveAnnotationsById(asset.id, comment.id)
expect(result.timestamp).toBe(18.5)
expect(result.annotations).toEqual([
{
type: 'line',
color: '#ff0000',
points: [
[10, 10],
[50, 50],
],
},
])
})
it('throws an error when AssetComment belongs to a different asset', async () => {
const user = await prisma.user.create({
data: { name: 'Author', email: 'author2@test.com', password: 'pw' },
})
const team = await prisma.team.create({
data: { name: 'Team' },
})
const project = await prisma.project.create({
data: { name: 'Project', teamId: team.id },
})
const assetA = await prisma.asset.create({
data: { name: 'videoA.mp4', type: 'file', projectId: project.id, status: 'uploaded' },
})
const assetB = await prisma.asset.create({
data: { name: 'videoB.mp4', type: 'file', projectId: project.id, status: 'uploaded' },
})
const commentB = await prisma.assetComment.create({
data: {
assetId: assetB.id,
creatorId: user.id,
message: 'Comment on B',
annotation: [
{
type: 'box',
color: '#ff0000',
points: [
[0, 0],
[10, 10],
],
},
],
},
})
await expect(resolveAnnotationsById(assetA.id, commentB.id)).rejects.toThrow(
`Annotation "${commentB.id}" belongs to asset "${assetB.id}", not target asset "${assetA.id}".`,
)
})
it('resolves annotations and timestamp from AgentSessionEntry (1-on-1 Chat Mode)', async () => {
const user = await prisma.user.create({
data: { name: 'Chat User', email: 'chatuser@test.com', password: 'pw' },
})
const team = await prisma.team.create({
data: { name: 'Chat Team' },
})
const project = await prisma.project.create({
data: { name: 'Chat Project', teamId: team.id },
})
const asset = await prisma.asset.create({
data: {
name: 'chat-video.mp4',
type: 'file',
projectId: project.id,
status: 'uploaded',
},
})
const agentUser = await prisma.user.create({
data: { name: 'Chat Bot User', email: 'bot@example.com', type: 'agent' },
})
const agent = await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'chat',
config: { provider: 'test-provider', model: 'test-model' },
},
})
// Create session storage
const storage = await DatabaseSessionStorage.create({
agentId: agent.id,
userId: user.id,
assetId: asset.id,
})
// Set active message context with markup annotations and position
storage.currentMessageContext = {
currentAsset: {
id: asset.id,
name: asset.name,
type: 'file',
path: asset.name,
},
position: { type: 'time', seconds: 12.34 },
annotation: true,
annotations: [
{
type: 'rectangle',
x: 100,
y: 200,
width: 50,
height: 50,
},
],
}
// User sends a message -> DatabaseSessionStorage.appendEntry is called
const userEntry: SessionTreeEntry = {
type: 'message',
id: '01JTESTENTRY1234567890',
parentId: null,
timestamp: new Date().toISOString(),
message: {
role: 'user',
content: 'Check this rectangle markup on video',
timestamp: Date.now(),
},
}
await storage.appendEntry(userEntry)
// Verify DatabaseSessionStorage persisted entry with details.id = entry.id
const dbEntry = await prisma.agentSessionEntry.findUnique({
where: { id: '01JTESTENTRY1234567890' },
})
expect(dbEntry).toBeDefined()
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const details = (dbEntry?.data as any)?.details
expect(details.id).toBe('01JTESTENTRY1234567890')
expect(details.position).toEqual({ type: 'time', seconds: 12.34 })
expect(details.annotations).toHaveLength(1)
// Now resolveAnnotationsById directly with the entry id and matching asset.id
const result = await resolveAnnotationsById(asset.id, '01JTESTENTRY1234567890')
expect(result.timestamp).toBe(12.34)
expect(result.annotations).toEqual([
{
type: 'rectangle',
x: 100,
y: 200,
width: 50,
height: 50,
},
])
})
it('throws an error when AgentSessionEntry belongs to a different asset', async () => {
const user = await prisma.user.create({
data: { name: 'User', email: 'user3@test.com', password: 'pw' },
})
const team = await prisma.team.create({
data: { name: 'Team' },
})
const project = await prisma.project.create({
data: { name: 'Project', teamId: team.id },
})
const assetA = await prisma.asset.create({
data: { name: 'chatA.mp4', type: 'file', projectId: project.id, status: 'uploaded' },
})
const assetB = await prisma.asset.create({
data: { name: 'chatB.mp4', type: 'file', projectId: project.id, status: 'uploaded' },
})
const agentUser = await prisma.user.create({
data: { name: 'Bot User', email: 'bot2@example.com', type: 'agent' },
})
const agent = await prisma.agent.create({
data: {
id: agentUser.id,
teamId: team.id,
type: 'chat',
config: { provider: 'test-provider', model: 'test-model' },
},
})
const storage = await DatabaseSessionStorage.create({
agentId: agent.id,
userId: user.id,
assetId: assetB.id,
})
storage.currentMessageContext = {
currentAsset: { id: assetB.id, name: assetB.name, type: 'file', path: assetB.name },
annotation: true,
annotations: [{ type: 'box' }],
}
const userEntry: SessionTreeEntry = {
type: 'message',
id: '01JENTRYONASSETB123456',
parentId: null,
timestamp: new Date().toISOString(),
message: { role: 'user', content: 'Markup on B', timestamp: Date.now() },
}
await storage.appendEntry(userEntry)
await expect(resolveAnnotationsById(assetA.id, '01JENTRYONASSETB123456')).rejects.toThrow(
`Annotation "01JENTRYONASSETB123456" belongs to asset "${assetB.id}", not target asset "${assetA.id}".`,
)
})
it('throws an error if annotationId does not match any record', async () => {
await expect(resolveAnnotationsById('asset-1', 'non_existent_id')).rejects.toThrow(
'Annotation with ID "non_existent_id" not found.',
)
})
})
@@ -0,0 +1,52 @@
import { prisma } from '@shumai/db'
import type { ShumaiMessageContext } from '@shumai/dtos'
export async function resolveAnnotationsById(
assetId: string,
annotationId?: string | null,
): Promise<{ annotations: PrismaJson.AnnotationList | null; timestamp: number | null }> {
if (!annotationId) {
return { annotations: null, timestamp: null }
}
// 1. Check AssetComment table (Comments Mode)
const comment = await prisma.assetComment.findUnique({
where: { id: annotationId },
select: { assetId: true, annotation: true, second: true },
})
if (comment) {
if (comment.assetId !== assetId) {
throw new Error(
`Annotation "${annotationId}" belongs to asset "${comment.assetId}", not target asset "${assetId}".`,
)
}
const list = comment.annotation
const annotations =
Array.isArray(list) && list.length > 0 ? (list as PrismaJson.AnnotationList) : null
return { annotations, timestamp: comment.second }
}
// 2. Check AgentSessionEntry table (1-on-1 Chat Mode)
const entry = await prisma.agentSessionEntry.findUnique({
where: { id: annotationId },
select: { assetId: true, data: true },
})
if (entry) {
const details = (entry.data as Record<string, unknown> | null)?.details as
| ShumaiMessageContext
| undefined
const entryAssetId = entry.assetId || details?.currentAsset?.id
if (entryAssetId && entryAssetId !== assetId) {
throw new Error(
`Annotation "${annotationId}" belongs to asset "${entryAssetId}", not target asset "${assetId}".`,
)
}
const list = details?.annotations
const annotations =
Array.isArray(list) && list.length > 0 ? (list as unknown as PrismaJson.AnnotationList) : null
const timestamp = details?.position?.type === 'time' ? details.position.seconds : null
return { annotations, timestamp }
}
throw new Error(`Annotation with ID "${annotationId}" not found.`)
}
+187 -50
View File
@@ -1,7 +1,10 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
import { createCreateFileTool } from './create-file'
import { fieldsToTypeBoxSchema } from '../index'
import { s3Service } from '@shumai/core/src/s3/s3'
import { executeAgentToolWorkflow } from './utils'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService } from '@shumai/core/src/authz/authz'
import { Value } from 'typebox/value'
import * as fs from 'fs'
import * as os from 'os'
import * as path from 'path'
@@ -13,8 +16,20 @@ vi.mock('@shumai/core/src/s3/s3', () => ({
},
}))
vi.mock('./utils', () => ({
executeAgentToolWorkflow: vi.fn().mockResolvedValue({ id: 'file-1', name: 'test', type: 'file' }),
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
Permission: { Edit: 'edit', Read: 'read' },
ResourceType: { Asset: 'asset' },
}))
vi.mock('@shumai/core/src/asset/asset', () => ({
assetService: {
createFile: vi
.fn()
.mockResolvedValue({ id: 'file-1', name: 'test', type: 'file', sizeByte: 100 }),
},
}))
describe('createCreateFileTool', () => {
@@ -44,7 +59,7 @@ describe('createCreateFileTool', () => {
const filePath = createTempFile('# Hello from disk', 'create-file-test.md')
const tool = createCreateFileTool('user-1')
const result = await tool.execute('call-1', { parent: 'folder-1', path: filePath })
const result = await tool.execute('call-1', { parent: 'folder-1', path: filePath, data: null })
expect(s3Service.uploadFileToKey).toHaveBeenCalledTimes(1)
expect(s3Service.putObject).not.toHaveBeenCalled()
@@ -54,62 +69,127 @@ describe('createCreateFileTool', () => {
expect(s3Key).toMatch(/^files\/.+\.md$/)
expect(uploadArgs[2]).toBe('text/markdown')
expect(executeAgentToolWorkflow).toHaveBeenCalledWith({
toolName: 'create_file',
args: {
parent: 'folder-1',
s3Key,
name: 'create-file-test.md',
size: Buffer.byteLength('# Hello from disk', 'utf-8'),
contentType: 'text/markdown',
},
userId: 'user-1',
assetId: 'folder-1',
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'edit',
type: 'asset',
id: 'folder-1',
})
expect(result.details).toEqual({ id: 'file-1', name: 'test', type: 'file' })
expect(assetService.createFile).toHaveBeenCalledWith({
parentId: 'folder-1',
name: 'create-file-test.md',
key: s3Key,
sizeByte: Buffer.byteLength('# Hello from disk', 'utf-8'),
contentType: 'text/markdown',
creatorId: 'user-1',
metadata: undefined,
})
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
id: 'file-1',
name: 'test',
type: 'file',
size: 100,
})
})
it('should forward the optional context to the agent tool workflow when provided', async () => {
const filePath = createTempFile('# Hello from disk', 'create-file-context.md')
it('should forward the metadata to assetService.createFile when provided', async () => {
const filePath = createTempFile('# Hello from disk', 'create-file-metadata.md')
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
])
const tool = createCreateFileTool('user-1')
const tool = createCreateFileTool('user-1', metadataSchema)
await tool.execute('call-1', {
parent: 'folder-1',
path: filePath,
context: 'Generated using gemini',
data: null,
metadata: { prompt: 'Generated using gemini' },
})
expect(executeAgentToolWorkflow).toHaveBeenCalledWith(
expect(assetService.createFile).toHaveBeenCalledWith(
expect.objectContaining({
args: expect.objectContaining({ context: 'Generated using gemini' }),
metadata: { prompt: 'Generated using gemini' },
}),
)
})
it('should not include context in args when not provided', async () => {
const filePath = createTempFile('# Hello from disk', 'create-file-no-context.md')
it('should forward select and selectMulti newOption metadata', async () => {
const filePath = createTempFile('# Hello from disk', 'create-file-new-option.md')
const metadataSchema = fieldsToTypeBoxSchema([
{
id: 'provider',
config: {
name: 'Provider',
type: 'select',
select: { options: [{ id: 'openai', displayName: 'OpenAI', color: '#f43f5e' }] },
},
},
{
id: 'tags',
config: {
name: 'Tags',
type: 'selectMulti',
selectMulti: { options: [{ id: 'tag1', displayName: 'Tag 1', color: '#3b82f6' }] },
},
},
])
const tool = createCreateFileTool('user-1')
await tool.execute('call-1', { parent: 'folder-1', path: filePath })
const tool = createCreateFileTool('user-1', metadataSchema)
await tool.execute('call-1', {
parent: 'folder-1',
path: filePath,
data: null,
metadata: {
provider: { newOption: { value: 'Kling' } },
tags: ['tag1', { newOption: { value: 'Tag 2' } }],
},
})
const callArgs = vi.mocked(executeAgentToolWorkflow).mock.calls[0][0].args
expect(callArgs).not.toHaveProperty('context')
expect(assetService.createFile).toHaveBeenCalledWith(
expect.objectContaining({
metadata: {
provider: { newOption: { value: 'Kling' } },
tags: ['tag1', { newOption: { value: 'Tag 2' } }],
},
}),
)
})
it('should not include metadata when metadata is null', async () => {
const filePath = createTempFile('# Hello from disk', 'create-file-no-metadata.md')
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
])
const tool = createCreateFileTool('user-1', metadataSchema)
await tool.execute('call-1', {
parent: 'folder-1',
path: filePath,
data: null,
metadata: null,
})
const callArgs = vi.mocked(assetService.createFile).mock.calls[0][0]
expect(callArgs.metadata).toBeUndefined()
})
it('should throw when the local file does not exist', async () => {
const tool = createCreateFileTool('user-1')
await expect(
tool.execute('call-1', { parent: 'folder-1', path: '/nonexistent/file.md' }),
tool.execute('call-1', { parent: 'folder-1', path: '/nonexistent/file.md', data: null }),
).rejects.toThrow('Local file not found at path: /nonexistent/file.md')
expect(s3Service.uploadFileToKey).not.toHaveBeenCalled()
expect(executeAgentToolWorkflow).not.toHaveBeenCalled()
expect(assetService.createFile).not.toHaveBeenCalled()
})
it('should create a file directly from name and content', async () => {
const tool = createCreateFileTool('user-1')
const result = await tool.execute('call-1', {
parent: 'folder-1',
path: null,
data: { name: 'notes.md', content: '# Notes\n\nHello' },
})
@@ -125,26 +205,30 @@ describe('createCreateFileTool', () => {
expect(size).toBe(Buffer.byteLength('# Notes\n\nHello', 'utf-8'))
expect(contentType).toBe('text/markdown')
expect(executeAgentToolWorkflow).toHaveBeenCalledWith({
toolName: 'create_file',
args: {
parent: 'folder-1',
s3Key,
name: 'notes.md',
size: Buffer.byteLength('# Notes\n\nHello', 'utf-8'),
contentType: 'text/markdown',
},
userId: 'user-1',
assetId: 'folder-1',
expect(assetService.createFile).toHaveBeenCalledWith({
parentId: 'folder-1',
name: 'notes.md',
key: s3Key,
sizeByte: Buffer.byteLength('# Notes\n\nHello', 'utf-8'),
contentType: 'text/markdown',
creatorId: 'user-1',
metadata: undefined,
})
expect(result.details).toEqual({ id: 'file-1', name: 'test', type: 'file' })
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
id: 'file-1',
name: 'test',
type: 'file',
size: 100,
})
})
it('should default unknown extensions to text/plain when creating from content', async () => {
const tool = createCreateFileTool('user-1')
await tool.execute('call-1', {
parent: 'folder-1',
path: null,
data: { name: 'script.ts', content: 'const x = 1' },
})
@@ -162,6 +246,7 @@ describe('createCreateFileTool', () => {
const tool = createCreateFileTool('user-1')
await tool.execute('call-1', {
parent: 'folder-1',
path: null,
data: { name: 'my/notes.md', content: 'hello' },
})
@@ -174,18 +259,16 @@ describe('createCreateFileTool', () => {
]
expect(s3Key).toContain('my_notes.md')
expect(executeAgentToolWorkflow).toHaveBeenCalledWith(
expect.objectContaining({
args: expect.objectContaining({ name: 'my_notes.md' }),
}),
expect(assetService.createFile).toHaveBeenCalledWith(
expect.objectContaining({ name: 'my_notes.md' }),
)
})
it('should throw when neither "path" nor "data" is provided', async () => {
const tool = createCreateFileTool('user-1')
await expect(tool.execute('call-1', { parent: 'folder-1' })).rejects.toThrow(
'Provide exactly one of "path" (a local file) or "data" (name and content).',
)
await expect(
tool.execute('call-1', { parent: 'folder-1', path: null, data: null }),
).rejects.toThrow('Provide exactly one of "path" (a local file) or "data" (name and content).')
})
it('should throw when both "path" and "data" are provided', async () => {
@@ -199,6 +282,60 @@ describe('createCreateFileTool', () => {
).rejects.toThrow('Provide exactly one of "path" (a local file) or "data" (name and content).')
expect(s3Service.uploadFileToKey).not.toHaveBeenCalled()
expect(s3Service.putObject).not.toHaveBeenCalled()
expect(executeAgentToolWorkflow).not.toHaveBeenCalled()
expect(assetService.createFile).not.toHaveBeenCalled()
})
it('should expose a strict, all-required (nullable) metadata schema when fields are provided', () => {
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
{
id: 'source',
config: {
name: 'Source',
type: 'select',
select: { options: [{ id: 'gemini', displayName: 'Gemini', color: '#ffffff' }] },
},
},
{ id: 'rating', config: { name: 'Rating', type: 'rating' } },
])
const tool = createCreateFileTool('user-1', metadataSchema)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
const meta = (tool.parameters as any).properties.metadata
// `metadata` is a nullable union: [strict object, null]
const metaObject = meta.anyOf[0]
expect(meta).toBeDefined()
expect(metaObject.required).toEqual(['prompt', 'source', 'rating'])
expect(metaObject.additionalProperties).toBe(false)
expect(metaObject.properties.prompt.anyOf[0].type).toBe('string')
expect(metaObject.properties.prompt.anyOf[1].type).toBe('null')
expect(metaObject.properties.source.anyOf[0].anyOf[0].enum).toEqual(['gemini'])
expect(metaObject.properties.rating.anyOf[0].type).toBe('number')
// Every field must be present (use null when unknown); unknown keys and bad enums are rejected
expect(Value.Check(meta, { prompt: 'hello', source: 'gemini', rating: 5 })).toBe(true)
expect(
Value.Check(meta, {
prompt: 'hello',
source: { newOption: { value: 'kling' } },
rating: 5,
}),
).toBe(true)
expect(Value.Check(meta, { prompt: null, source: null, rating: null })).toBe(true)
expect(Value.Check(meta, { prompt: 'hello', source: null, rating: null })).toBe(true)
expect(Value.Check(meta, { prompt: 'hello' })).toBe(false)
expect(
Value.Check(meta, { prompt: 'hello', source: 'gemini', rating: 5, unknownKey: 'x' }),
).toBe(false)
expect(Value.Check(meta, { prompt: 'hello', source: 'not-an-option', rating: null })).toBe(
false,
)
})
it('should omit the metadata parameter when no schema is provided', () => {
const tool = createCreateFileTool('user-1')
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
expect((tool.parameters as any).properties.metadata).toBeUndefined()
})
})
+80 -57
View File
@@ -1,70 +1,81 @@
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { s3Service } from '@shumai/core/src/s3/s3'
import { getFileMimeType, readFileMimeType } from '@shumai/core/src/utils/file-mime'
import { sanitizeFilename } from '@shumai/core/src/utils/filename'
import { Type } from '@sinclair/typebox'
import { type User } from '@shumai/db'
import { Type, type TSchema } from 'typebox'
import * as fs from 'fs'
import * as path from 'path'
import { ulid } from 'ulid'
import { executeAgentToolWorkflow } from './utils'
const createFileSchema = Type.Object({
parent: Type.String({
description: 'The parent folder ID under which to create the file. This parameter is required.',
}),
path: Type.Optional(
Type.String({
description:
'The absolute or relative local path to an existing file on disk. Provide either this or "data", but not both.',
}),
),
data: Type.Optional(
Type.Object({
name: Type.String({
minLength: 1,
description:
'The desired name for the new file, including its extension (e.g. "notes.md").',
}),
content: Type.String({
description: 'The full content of the file as a plain string.',
}),
}),
),
context: Type.Optional(
Type.String({
description:
'Optional context passed to the AI metadata autofill workflow for fields that cannot be inferred from file content alone ' +
'(e.g. the model or prompt used to generate the asset). ' +
'Use list_autofill_fields first to check which fields can be autofilled. ' +
'Keep the context short and only include information relevant to those fields.',
}),
),
})
interface CreateFileToolParams {
parent: string
path: string | null
data: { name: string; content: string } | null
metadata?: Record<string, unknown> | null
}
function assertExactlyOneSource(params: {
path?: string
data?: { name: string; content: string }
}) {
const hasPath = params.path !== undefined
const hasData = params.data !== undefined
function assertExactlyOneSource(params: CreateFileToolParams) {
const hasPath = params.path !== null
const hasData = params.data !== null
if (hasPath === hasData) {
throw new Error('Provide exactly one of "path" (a local file) or "data" (name and content).')
}
return hasPath ? params.path! : params.data!
}
export function createCreateFileTool(userId: string): AgentTool<typeof createFileSchema> {
return {
export function createCreateFileTool(userId: string, metadataSchema?: TSchema): AgentTool {
const createFileSchema = Type.Object(
{
parent: Type.String({
description:
'The parent folder ID under which to create the file. This parameter is required.',
}),
path: Type.Union([
Type.String({
description:
'The absolute or relative local path to an existing file on disk. Provide a value here OR in "data", but not both. Use null if you are providing "data".',
}),
Type.Null(),
]),
data: Type.Union([
Type.Object(
{
name: Type.String({
minLength: 1,
description:
'The desired name for the new file, including its extension (e.g. "notes.md").',
}),
content: Type.String({
description: 'The full content of the file as a plain string.',
}),
},
{ additionalProperties: false },
),
Type.Null(),
]),
...(metadataSchema ? { metadata: Type.Union([metadataSchema, Type.Null()]) } : {}),
},
{ additionalProperties: false },
)
const tool: AgentTool = {
name: 'create_file',
label: 'Create File',
description:
'Create a new file under a folder. You MUST choose exactly ONE creation method: ' +
'(1) provide "path" to upload an existing local file, OR ' +
'(2) provide "data" with name and content to create a new text file. ' +
'Never provide both "path" and "data" together. Never omit both.',
'Set the unused parameter to null. ' +
'The optional "metadata" parameter accepts only the CREATION_CONTEXT fields declared in its schema; never invent field keys, and use null for fields you do not know.',
parameters: createFileSchema,
execute: async (_toolCallId, params) => {
const source = assertExactlyOneSource(params)
// The harness validates tool args against `createFileSchema` before calling
// execute, so `params` is guaranteed to match this shape.
const p = params as CreateFileToolParams
const source = assertExactlyOneSource(p)
let name: string
let size: number
@@ -96,23 +107,35 @@ export function createCreateFileTool(userId: string): AgentTool<typeof createFil
await s3Service.putObject(process.env.S3_BUCKET || 'shumai', s3Key, content, size, mimeType)
}
const result = await executeAgentToolWorkflow({
toolName: 'create_file',
args: {
parent: params.parent,
s3Key,
name,
size,
contentType: mimeType,
...(params.context ? { context: params.context } : {}),
},
userId,
assetId: params.parent,
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Edit,
type: ResourceType.Asset,
id: p.parent,
})
const asset = await assetService.createFile({
parentId: p.parent,
name,
key: s3Key,
sizeByte: size,
contentType: mimeType,
creatorId: userId,
metadata: p.metadata ?? undefined,
})
const result = {
id: asset.id,
name: asset.name,
type: asset.type,
size: Number(asset.sizeByte),
}
return {
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
details: result,
details: {},
}
},
}
return tool
}
@@ -0,0 +1,56 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createCreateFolderTool } from './create-folder'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
Permission: { Edit: 'edit', Read: 'read' },
ResourceType: { Asset: 'asset' },
}))
vi.mock('@shumai/core/src/asset/asset', () => ({
assetService: {
createAsset: vi.fn().mockResolvedValue({
id: 'folder-1',
name: 'New Folder',
type: 'folder',
sizeByte: 0,
}),
},
}))
describe('createCreateFolderTool', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('checks edit permission and calls assetService.createAsset with folder type', async () => {
const tool = createCreateFolderTool('user-1')
const result = await tool.execute('call-1', { parent: 'root-folder', name: 'New Folder' })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'edit',
type: 'asset',
id: 'root-folder',
})
expect(assetService.createAsset).toHaveBeenCalledWith({
name: 'New Folder',
parentId: 'root-folder',
type: 'folder',
creatorId: 'user-1',
})
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
id: 'folder-1',
name: 'New Folder',
type: 'folder',
size: 0,
})
})
})
+25 -8
View File
@@ -1,6 +1,8 @@
import { Type } from '@sinclair/typebox'
import { Type } from 'typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { executeAgentToolWorkflow } from './utils'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { type User } from '@shumai/db'
const createFolderSchema = Type.Object({
parent: Type.String({
@@ -19,15 +21,30 @@ export function createCreateFolderTool(userId: string): AgentTool<typeof createF
description: 'Create a new folder in a specified parent folder.',
parameters: createFolderSchema,
execute: async (_toolCallId, params) => {
const result = await executeAgentToolWorkflow({
toolName: 'create_folder',
args: params,
userId,
assetId: params.parent,
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Edit,
type: ResourceType.Asset,
id: params.parent,
})
const newFolder = await assetService.createAsset({
name: params.name,
parentId: params.parent,
type: 'folder',
creatorId: userId,
})
const result = {
id: newFolder.id,
name: newFolder.name,
type: newFolder.type,
size: Number(newFolder.sizeByte),
}
return {
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
details: result,
details: {},
}
},
}
+131 -29
View File
@@ -1,7 +1,10 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
import { createCreateVersionTool } from './create-version'
import { fieldsToTypeBoxSchema } from '../index'
import { s3Service } from '@shumai/core/src/s3/s3'
import { executeAgentToolWorkflow } from './utils'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService } from '@shumai/core/src/authz/authz'
import { Value } from 'typebox/value'
import * as fs from 'fs'
import * as os from 'os'
import * as path from 'path'
@@ -12,8 +15,20 @@ vi.mock('@shumai/core/src/s3/s3', () => ({
},
}))
vi.mock('./utils', () => ({
executeAgentToolWorkflow: vi.fn().mockResolvedValue({ id: 'file-1', name: 'v2', type: 'file' }),
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
Permission: { Edit: 'edit', Read: 'read' },
ResourceType: { Asset: 'asset' },
}))
vi.mock('@shumai/core/src/asset/asset', () => ({
assetService: {
createVersion: vi
.fn()
.mockResolvedValue({ id: 'file-1', name: 'v2', type: 'file', sizeByte: 100 }),
},
}))
describe('createCreateVersionTool', () => {
@@ -39,7 +54,7 @@ describe('createCreateVersionTool', () => {
return filePath
}
it('should upload a local file and create a version via the agent tool workflow', async () => {
it('should upload a local file and create a version via assetService.createVersion', async () => {
const filePath = createTempFile('# Hello from disk', 'create-version-test.md')
const tool = createCreateVersionTool('user-1')
@@ -51,47 +66,104 @@ describe('createCreateVersionTool', () => {
expect(s3Key).toMatch(/^files\/.+\.md$/)
expect(uploadArgs[2]).toBe('text/markdown')
expect(executeAgentToolWorkflow).toHaveBeenCalledWith({
toolName: 'create_version',
args: {
parent: 'file-1',
s3Key,
name: 'create-version-test.md',
size: Buffer.byteLength('# Hello from disk', 'utf-8'),
contentType: 'text/markdown',
},
userId: 'user-1',
assetId: 'file-1',
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'edit',
type: 'asset',
id: 'file-1',
})
expect(result.details).toEqual({ id: 'file-1', name: 'v2', type: 'file' })
expect(assetService.createVersion).toHaveBeenCalledWith({
parentId: 'file-1',
name: 'create-version-test.md',
key: s3Key,
sizeByte: Buffer.byteLength('# Hello from disk', 'utf-8'),
contentType: 'text/markdown',
creatorId: 'user-1',
metadata: undefined,
})
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
id: 'file-1',
name: 'v2',
type: 'file',
size: 100,
})
})
it('should forward the optional context to the agent tool workflow when provided', async () => {
const filePath = createTempFile('# Hello from disk', 'create-version-context.md')
it('should forward the metadata to assetService.createVersion when provided', async () => {
const filePath = createTempFile('# Hello from disk', 'create-version-metadata.md')
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
])
const tool = createCreateVersionTool('user-1')
const tool = createCreateVersionTool('user-1', metadataSchema)
await tool.execute('call-1', {
parent: 'file-1',
path: filePath,
context: 'Generated using gemini',
metadata: { prompt: 'Generated using gemini' },
})
expect(executeAgentToolWorkflow).toHaveBeenCalledWith(
expect(assetService.createVersion).toHaveBeenCalledWith(
expect.objectContaining({
args: expect.objectContaining({ context: 'Generated using gemini' }),
metadata: { prompt: 'Generated using gemini' },
}),
)
})
it('should not include context in args when not provided', async () => {
const filePath = createTempFile('# Hello from disk', 'create-version-no-context.md')
it('should forward select and selectMulti newOption metadata', async () => {
const filePath = createTempFile('# Hello from disk', 'create-version-new-option.md')
const metadataSchema = fieldsToTypeBoxSchema([
{
id: 'provider',
config: {
name: 'Provider',
type: 'select',
select: { options: [{ id: 'openai', displayName: 'OpenAI', color: '#f43f5e' }] },
},
},
{
id: 'tags',
config: {
name: 'Tags',
type: 'selectMulti',
selectMulti: { options: [{ id: 'tag1', displayName: 'Tag 1', color: '#3b82f6' }] },
},
},
])
const tool = createCreateVersionTool('user-1')
await tool.execute('call-1', { parent: 'file-1', path: filePath })
const tool = createCreateVersionTool('user-1', metadataSchema)
await tool.execute('call-1', {
parent: 'file-1',
path: filePath,
metadata: {
provider: { newOption: { value: 'Kling' } },
tags: ['tag1', { newOption: { value: 'Tag 2' } }],
},
})
const callArgs = vi.mocked(executeAgentToolWorkflow).mock.calls[0][0].args
expect(callArgs).not.toHaveProperty('context')
expect(assetService.createVersion).toHaveBeenCalledWith(
expect.objectContaining({
metadata: {
provider: { newOption: { value: 'Kling' } },
tags: ['tag1', { newOption: { value: 'Tag 2' } }],
},
}),
)
})
it('should not include metadata when metadata is null', async () => {
const filePath = createTempFile('# Hello from disk', 'create-version-no-metadata.md')
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
])
const tool = createCreateVersionTool('user-1', metadataSchema)
await tool.execute('call-1', { parent: 'file-1', path: filePath, metadata: null })
const callArgs = vi.mocked(assetService.createVersion).mock.calls[0][0]
expect(callArgs.metadata).toBeUndefined()
})
it('should throw when the local file does not exist', async () => {
@@ -100,6 +172,36 @@ describe('createCreateVersionTool', () => {
tool.execute('call-1', { parent: 'file-1', path: '/nonexistent/file.md' }),
).rejects.toThrow('Local file not found at path: /nonexistent/file.md')
expect(s3Service.uploadFileToKey).not.toHaveBeenCalled()
expect(executeAgentToolWorkflow).not.toHaveBeenCalled()
expect(assetService.createVersion).not.toHaveBeenCalled()
})
it('should expose a strict, all-required (nullable) metadata schema when fields are provided', () => {
const metadataSchema = fieldsToTypeBoxSchema([
{ id: 'prompt', config: { name: 'Prompt', type: 'text' } },
{ id: 'toggle', config: { name: 'Active', type: 'toggle' } },
])
const tool = createCreateVersionTool('user-1', metadataSchema)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
const meta = (tool.parameters as any).properties.metadata
// `metadata` is a nullable union: [strict object, null]
const metaObject = meta.anyOf[0]
expect(meta).toBeDefined()
expect(metaObject.required).toEqual(['prompt', 'toggle'])
expect(metaObject.additionalProperties).toBe(false)
expect(metaObject.properties.prompt.anyOf[0].type).toBe('string')
expect(metaObject.properties.prompt.anyOf[1].type).toBe('null')
expect(metaObject.properties.toggle.anyOf[0].type).toBe('boolean')
expect(Value.Check(meta, { prompt: 'p', toggle: true })).toBe(true)
expect(Value.Check(meta, { prompt: null, toggle: null })).toBe(true)
expect(Value.Check(meta, { prompt: 'p', toggle: null, unknownKey: 'x' })).toBe(false)
})
it('should omit the metadata parameter when no schema is provided', () => {
const tool = createCreateVersionTool('user-1')
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- inspecting the runtime schema shape of the dynamically-built tool
expect((tool.parameters as any).properties.metadata).toBeUndefined()
})
})
+64 -43
View File
@@ -1,69 +1,90 @@
import { Type } from '@sinclair/typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { executeAgentToolWorkflow } from './utils'
import * as fs from 'fs'
import * as path from 'path'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { s3Service } from '@shumai/core/src/s3/s3'
import { readFileMimeType } from '@shumai/core/src/utils/file-mime'
import { ulid } from 'ulid'
import { sanitizeFilename } from '@shumai/core/src/utils/filename'
import { type User } from '@shumai/db'
import { Type, type TSchema } from 'typebox'
import * as fs from 'fs'
import * as path from 'path'
import { ulid } from 'ulid'
const createVersionSchema = Type.Object({
parent: Type.String({
description:
'The parent file ID for which to create a new version. This parameter is required.',
}),
path: Type.String({
description:
'The absolute or relative local path to the file on disk. This parameter is required.',
}),
context: Type.Optional(
Type.String({
description:
'Optional context passed to the AI metadata autofill workflow for fields that cannot be inferred from file content alone ' +
'(e.g. the model or prompt used to generate the asset). ' +
'Use list_autofill_fields first to check which fields can be autofilled. ' +
'Keep the context short and only include information relevant to those fields.',
}),
),
})
interface CreateVersionToolParams {
parent: string
path: string
metadata?: Record<string, unknown> | null
}
export function createCreateVersionTool(userId: string): AgentTool<typeof createVersionSchema> {
return {
export function createCreateVersionTool(userId: string, metadataSchema?: TSchema): AgentTool {
const createVersionSchema = Type.Object(
{
parent: Type.String({
description:
'The parent file ID for which to create a new version. This parameter is required.',
}),
path: Type.String({
description:
'The absolute or relative local path to the file on disk. This parameter is required.',
}),
...(metadataSchema ? { metadata: Type.Union([metadataSchema, Type.Null()]) } : {}),
},
{ additionalProperties: false },
)
const tool: AgentTool = {
name: 'create_version',
label: 'Create Version',
description: 'Create a new version of an existing file from a local file path.',
description:
'Create a new version of an existing file from a local file path. ' +
'The optional "metadata" parameter accepts only the CREATION_CONTEXT fields declared in its schema; never invent field keys, and use null for fields you do not know.',
parameters: createVersionSchema,
execute: async (_toolCallId, params) => {
const absolutePath = path.resolve(process.cwd(), params.path)
// The harness validates tool args against `createVersionSchema` before calling
// execute, so `params` is guaranteed to match this shape.
const p = params as CreateVersionToolParams
const absolutePath = path.resolve(process.cwd(), p.path)
if (!fs.existsSync(absolutePath)) {
throw new Error(`Local file not found at path: ${params.path}`)
throw new Error(`Local file not found at path: ${p.path}`)
}
const fileSize = fs.statSync(absolutePath).size
const mimeType = readFileMimeType(absolutePath)
const name = path.basename(absolutePath)
// Generate compliant S3 key matching normal file upload format
const s3Key = `files/${ulid()}/${sanitizeFilename(path.basename(absolutePath))}`
const s3Key = `files/${ulid()}/${sanitizeFilename(name)}`
await s3Service.uploadFileToKey(absolutePath, s3Key, mimeType)
const result = await executeAgentToolWorkflow({
toolName: 'create_version',
args: {
parent: params.parent,
s3Key,
name: path.basename(absolutePath),
size: fileSize,
contentType: mimeType,
...(params.context ? { context: params.context } : {}),
},
userId,
assetId: params.parent,
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Edit,
type: ResourceType.Asset,
id: p.parent,
})
const asset = await assetService.createVersion({
parentId: p.parent,
name,
key: s3Key,
sizeByte: fileSize,
contentType: mimeType,
creatorId: userId,
metadata: p.metadata ?? undefined,
})
const result = {
id: asset.id,
name: asset.name,
type: asset.type,
size: Number(asset.sizeByte),
}
return {
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
details: result,
details: {},
}
},
}
return tool
}
+272 -143
View File
@@ -1,6 +1,6 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
import { createDownloadAssetTool } from './download-asset'
import { prisma, type Asset } from '@shumai/db'
import { createDownloadAssetTool, resolveAssetIdFromKey } from './download-asset'
import { prisma, type Asset, type StorageKey } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { authzService } from '@shumai/core/src/authz/authz'
import * as fs from 'fs'
@@ -15,6 +15,10 @@ vi.mock('@shumai/db', async (importOriginal) => {
findUnique: vi.fn(),
findFirst: vi.fn(),
},
storageKey: {
findFirst: vi.fn(),
findUnique: vi.fn(),
},
},
}
})
@@ -52,163 +56,288 @@ describe('downloadAssetTool', () => {
it('should throw authorization error if user ID is empty', async () => {
const tool = createDownloadAssetTool('')
await expect(tool.execute('call-1', { assetId: 'asset-1' })).rejects.toThrow(
await expect(tool.execute('call-1', { assetId: 'asset-1', key: null })).rejects.toThrow(
'User ID is required for authorization.',
)
})
it('should throw error if asset not found', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
it('should throw error if both assetId and key are null', async () => {
const tool = createDownloadAssetTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1' })).rejects.toThrow(
'Asset with ID asset-1 not found.',
await expect(tool.execute('call-1', { assetId: null, key: null })).rejects.toThrow(
'Provide exactly one of "assetId" (to download a workspace asset) or "key" (to download a specific S3 storage key). Set the unused parameter to null.',
)
})
it('should download proxy image to .pi directory when imageTranscodes is present', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-img-1',
name: 'sample_photo.jpg',
storageKey: { key: 'raw/sample_photo.jpg' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/sample_photo.webp' }],
},
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-proxy-image-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
it('should throw error if both assetId and key are provided', async () => {
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'asset-img-1' })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-img-1',
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/sample_photo.webp')
const expectedPath = path.join(piDir, 'asset-img-1_sample_photo.jpg')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(fs.readFileSync(expectedPath, 'utf-8')).toBe('fake-proxy-image-bytes')
expect(result.details.filePath).toBe(path.join('.pi', 'asset-img-1_sample_photo.jpg'))
expect(result.details.contentType).toBe('image/webp')
expect((result.content[0] as { type: 'text'; text: string }).text).toContain(
'Remember to delete this temporary file when finished.',
await expect(
tool.execute('call-1', { assetId: 'asset-1', key: 'files/path/file.png' }),
).rejects.toThrow(
'Provide exactly one of "assetId" (to download a workspace asset) or "key" (to download a specific S3 storage key). Set the unused parameter to null.',
)
})
it('should download proxy video to .pi directory when videoTranscodes is present', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-vid-1',
name: 'video_test.mov',
storageKey: { key: 'raw/video_test.mov' },
media: {
proxyType: 'video',
videoTranscodes: [{ key: 'proxy/video_test.mp4' }],
},
} as unknown as Asset)
describe('Download by assetId', () => {
it('should throw error if asset not found', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-video-bytes'),
contentType: 'video/mp4',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'asset-vid-1' })
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/video_test.mp4')
const expectedPath = path.join(piDir, 'asset-vid-1_video_test.mov')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(result.details.size).toBe(Buffer.from('fake-video-bytes').length)
})
it('should download original file to .pi directory even when pdfTranscode is present', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-md-1',
name: 'doc.md',
storageKey: { key: 'raw/doc.md' },
media: {
proxyType: 'pdf',
pdfTranscode: { key: 'proxy/doc.pdf' },
original: { key: 'raw/doc.md' },
},
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('# fake markdown bytes'),
contentType: 'text/markdown',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'asset-md-1' })
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'raw/doc.md')
const expectedPath = path.join(piDir, 'asset-md-1_doc.md')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(result.details.contentType).toBe('text/markdown')
})
it('should resolve version_stack asset to its latest version file', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'stack-1',
name: '',
type: 'version_stack',
storageKey: null,
media: null,
} as unknown as Asset)
vi.mocked(prisma.asset.findFirst).mockResolvedValue({
id: 'file-ver-2',
name: 'banana_pig.png',
type: 'file',
storageKey: { key: 'raw/banana_pig.png' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/banana_pig.webp' }],
},
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-version-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'stack-1' })
expect(prisma.asset.findFirst).toHaveBeenCalledWith({
where: { parentId: 'stack-1', isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
const tool = createDownloadAssetTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', key: null })).rejects.toThrow(
'Asset with ID asset-1 not found.',
)
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/banana_pig.webp')
it('should download proxy image to .pi directory when downloading by assetId', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-img-1',
name: 'sample_photo.jpg',
storageKey: { key: 'raw/sample_photo.jpg' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/sample_photo.webp' }],
},
} as unknown as Asset)
const expectedPath = path.join(piDir, 'file-ver-2_banana_pig.png')
createdFiles.push(expectedPath)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-proxy-image-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
expect(fs.existsSync(expectedPath)).toBe(true)
expect(result.details.name).toBe('banana_pig.png')
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'asset-img-1', key: null })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-img-1',
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/sample_photo.webp')
const expectedPath = path.join(piDir, 'asset-img-1_sample_photo.jpg')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(fs.readFileSync(expectedPath, 'utf-8')).toBe('fake-proxy-image-bytes')
expect(result.details.filePath).toBe(path.join('.pi', 'asset-img-1_sample_photo.jpg'))
expect(result.details.contentType).toBe('image/webp')
expect((result.content[0] as { type: 'text'; text: string }).text).toContain(
'Remember to delete this temporary file when finished.',
)
})
it('should resolve version_stack asset to its latest version file', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'stack-1',
name: '',
type: 'version_stack',
storageKey: null,
media: null,
} as unknown as Asset)
vi.mocked(prisma.asset.findFirst).mockResolvedValue({
id: 'file-ver-2',
name: 'banana_pig.png',
type: 'file',
storageKey: { key: 'raw/banana_pig.png' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/banana_pig.webp' }],
},
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-version-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', { assetId: 'stack-1', key: null })
expect(prisma.asset.findFirst).toHaveBeenCalledWith({
where: { parentId: 'stack-1', isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/banana_pig.webp')
const expectedPath = path.join(piDir, 'file-ver-2_banana_pig.png')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(result.details.name).toBe('banana_pig.png')
})
})
describe('Download by S3 key and Authorization', () => {
it('should reject keys that do not start with files/', async () => {
const tool = createDownloadAssetTool('user-1')
await expect(
tool.execute('call-1', { assetId: null, key: 'avatars/user-1.png' }),
).rejects.toThrow('Invalid storage key format: key must start with "files/"')
})
it('should reject keys with path traversal (..)', async () => {
const tool = createDownloadAssetTool('user-1')
await expect(
tool.execute('call-1', { assetId: null, key: 'files/../../etc/passwd' }),
).rejects.toThrow('cannot contain ".."')
})
it('should reject keys with insufficient path segments', async () => {
const tool = createDownloadAssetTool('user-1')
await expect(tool.execute('call-1', { assetId: null, key: 'files/invalid' })).rejects.toThrow(
'Invalid storage key format',
)
})
it('should reject keys that do not belong to any valid asset in database', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findFirst).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findUnique).mockResolvedValue(null)
const tool = createDownloadAssetTool('user-1')
await expect(
tool.execute('call-1', { assetId: null, key: 'files/nonexistent-id/shot.webp' }),
).rejects.toThrow('does not belong to any valid asset or the asset has been deleted')
})
it('should reject download when user lacks read permission on the owning asset (cross-project protection)', async () => {
// Key belongs to asset-in-other-project
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-in-other-project',
} as unknown as Asset)
// Authz rejects because user is in a different project/team
vi.mocked(authzService.hasPermission).mockRejectedValue(
new Error('User does not have Read permission on asset asset-in-other-project'),
)
const tool = createDownloadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: null,
key: 'files/asset-in-other-project/screenshots/secret_frame.webp',
}),
).rejects.toThrow('User does not have Read permission on asset asset-in-other-project')
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-in-other-project',
})
expect(s3Service.getObject).not.toHaveBeenCalled()
})
it('should authorize and download derived artifact key (files/<assetId>/...)', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'ast-123',
} as unknown as Asset)
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-screenshot-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: null,
key: 'files/ast-123/screenshots/shot_5.0s.webp',
})
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'ast-123',
})
expect(s3Service.getObject).toHaveBeenCalledWith(
'shumai',
'files/ast-123/screenshots/shot_5.0s.webp',
)
const expectedPath = path.join(piDir, 'shot_5.0s.webp')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(fs.readFileSync(expectedPath, 'utf-8')).toBe('fake-screenshot-bytes')
expect(result.details.filePath).toBe(path.join('.pi', 'shot_5.0s.webp'))
expect(result.details.key).toBe('files/ast-123/screenshots/shot_5.0s.webp')
expect(result.details.assetId).toBe('ast-123')
expect(result.details.size).toBe(Buffer.from('fake-screenshot-bytes').length)
})
it('should authorize and download original upload key (files/<uploadUlid>/...) via StorageKey', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findFirst).mockResolvedValue({
id: 'sk-1',
key: 'files/upload-ulid-456/photo.png',
assets: [{ id: 'asset-photo-1' }],
} as unknown as StorageKey & { assets: Asset[] })
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-original-photo-bytes'),
contentType: 'image/png',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createDownloadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: null,
key: 'files/upload-ulid-456/photo.png',
})
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-photo-1',
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'files/upload-ulid-456/photo.png')
const expectedPath = path.join(piDir, 'photo.png')
createdFiles.push(expectedPath)
expect(fs.existsSync(expectedPath)).toBe(true)
expect(result.details.assetId).toBe('asset-photo-1')
})
})
describe('resolveAssetIdFromKey helper unit tests', () => {
it('resolves directly by asset ID', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue({ id: 'ast-999' } as unknown as Asset)
const assetId = await resolveAssetIdFromKey('files/ast-999/pdf_pages/page_1.webp')
expect(assetId).toBe('ast-999')
})
it('resolves via StorageKey prefix', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findFirst).mockResolvedValue({
assets: [{ id: 'ast-888' }],
} as unknown as StorageKey & { assets: Asset[] })
const assetId = await resolveAssetIdFromKey('files/ulid-777/document.pdf')
expect(assetId).toBe('ast-888')
})
it('resolves via exact StorageKey match', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findFirst).mockResolvedValue(null)
vi.mocked(prisma.storageKey.findUnique).mockResolvedValue({
assets: [{ id: 'ast-777' }],
} as unknown as StorageKey & { assets: Asset[] })
const assetId = await resolveAssetIdFromKey('files/ulid-special/document.pdf')
expect(assetId).toBe('ast-777')
})
})
})
+209 -84
View File
@@ -1,4 +1,4 @@
import { Type } from '@sinclair/typebox'
import { Type } from 'typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { prisma, type User } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
@@ -7,82 +7,107 @@ import { sanitizeFilename } from '@shumai/core/src/utils/filename'
import * as fs from 'fs'
import * as path from 'path'
const downloadAssetSchema = Type.Object({
assetId: Type.String({
description:
'The asset ID of the workspace asset to download to local disk. This parameter is required.',
}),
})
export const downloadAssetSchema = Type.Object(
{
assetId: Type.Union([
Type.String({
description:
'The asset ID of the workspace asset to download to local disk, or null if providing "key".',
}),
Type.Null(),
]),
key: Type.Union([
Type.String({
description:
'The storage S3 key of a specific media artifact (e.g. a screenshot or transcode key from read_asset) to download to local disk, or null if providing "assetId".',
}),
Type.Null(),
]),
},
{ additionalProperties: false },
)
export async function resolveAssetIdFromKey(key: string): Promise<string> {
if (!key.startsWith('files/') || key.includes('..')) {
throw new Error(
'Invalid storage key format: key must start with "files/" and cannot contain "..".',
)
}
const segments = key.split('/')
if (segments.length < 3) {
throw new Error(
'Invalid storage key format: key must follow "files/<identifier>/<path>" structure.',
)
}
const identifier = segments[1]
// 1. Check if identifier is directly an Asset ID (Derived Artifacts: screenshots, pdf_pages, annotations, proxies)
const assetById = await prisma.asset.findUnique({
where: { id: identifier, isDeleted: false },
select: { id: true },
})
if (assetById) {
return assetById.id
}
// 2. Check if identifier is a StorageKey directory prefix (Original uploads / agent-created files)
const storageKeyByPrefix = await prisma.storageKey.findFirst({
where: { key: { startsWith: `files/${identifier}/` } },
include: {
assets: {
where: { isDeleted: false },
select: { id: true },
},
},
})
if (storageKeyByPrefix?.assets && storageKeyByPrefix.assets.length > 0) {
return storageKeyByPrefix.assets[0].id
}
// 3. Exact StorageKey fallback
const exactStorageKey = await prisma.storageKey.findUnique({
where: { key },
include: {
assets: {
where: { isDeleted: false },
select: { id: true },
},
},
})
if (exactStorageKey?.assets && exactStorageKey.assets.length > 0) {
return exactStorageKey.assets[0].id
}
throw new Error(
`Storage key "${key}" does not belong to any valid asset or the asset has been deleted.`,
)
}
export function createDownloadAssetTool(userId: string): AgentTool<typeof downloadAssetSchema> {
return {
name: 'download_asset',
label: 'Download Asset',
description:
'Downloads a workspace asset (proxy version if available) to the local .pi directory for processing with local scripts or bash commands. Remember to delete the temporary downloaded file when finished.',
'Downloads a workspace asset or specific storage S3 key to the local .pi directory for processing with local scripts or bash commands. ' +
'Provide exactly one of "assetId" (for full assets) or "key" (for intermediate artifacts like screenshots from read_asset). ' +
'Remember to delete the temporary downloaded file when finished.',
parameters: downloadAssetSchema,
execute: async (_toolCallId, params) => {
const assetId = params.assetId
const { assetId, key } = params
if (!userId) {
throw new Error('User ID is required for authorization.')
}
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: assetId,
})
const hasAssetId = assetId !== null
const hasKey = key !== null
let asset = await prisma.asset.findUnique({
where: { id: assetId },
include: { storageKey: true },
})
if (!asset) {
throw new Error(`Asset with ID ${assetId} not found.`)
}
if (asset.type === 'version_stack') {
const latestVersion = await prisma.asset.findFirst({
where: { parentId: asset.id, isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
})
if (latestVersion) {
asset = latestVersion
}
}
let mediaKey = asset.storageKey?.key
if (asset.media) {
const mediaInfo = asset.media as unknown as PrismaJson.MediaInfo
if (
mediaInfo.proxyType === 'image' ||
(mediaInfo.imageTranscodes && mediaInfo.imageTranscodes.length > 0)
) {
if (mediaInfo.imageTranscodes && mediaInfo.imageTranscodes.length > 0) {
mediaKey = mediaInfo.imageTranscodes[0].key || mediaKey
}
} else if (
mediaInfo.proxyType === 'video' ||
(mediaInfo.videoTranscodes && mediaInfo.videoTranscodes.length > 0) ||
mediaInfo.videoPreview?.key
) {
if (mediaInfo.videoTranscodes && mediaInfo.videoTranscodes.length > 0) {
mediaKey = mediaInfo.videoTranscodes[0].key || mediaKey
} else if (mediaInfo.videoPreview?.key) {
mediaKey = mediaInfo.videoPreview.key
}
} else if (mediaInfo.original?.key) {
mediaKey = mediaInfo.original.key || mediaKey
}
}
if (!mediaKey) {
throw new Error(`No media content found for asset ${assetId}.`)
if (hasAssetId === hasKey) {
throw new Error(
'Provide exactly one of "assetId" (to download a workspace asset) or "key" (to download a specific S3 storage key). Set the unused parameter to null.',
)
}
const piDir = path.join(process.cwd(), '.pi')
@@ -90,32 +115,132 @@ export function createDownloadAssetTool(userId: string): AgentTool<typeof downlo
fs.mkdirSync(piDir, { recursive: true })
}
const safeName = sanitizeFilename(asset.name || 'file')
const filename = `${asset.id}_${safeName}`
const targetFilePath = path.join(piDir, filename)
const relativePath = path.join('.pi', filename)
const bucket = process.env.S3_BUCKET || 'shumai'
const { buffer, contentType } = await s3Service.getObject(bucket, mediaKey)
fs.writeFileSync(targetFilePath, buffer)
// Branch A: Download by assetId
if (assetId) {
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: assetId,
})
return {
content: [
{
type: 'text',
text: `Downloaded asset "${asset.name}" (ID: ${assetId}) to "${relativePath}". Remember to delete this temporary file when finished.`,
let asset = await prisma.asset.findUnique({
where: { id: assetId },
include: { storageKey: true },
})
if (!asset) {
throw new Error(`Asset with ID ${assetId} not found.`)
}
if (asset.type === 'version_stack') {
const latestVersion = await prisma.asset.findFirst({
where: { parentId: asset.id, isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
})
if (latestVersion) {
asset = latestVersion
}
}
let mediaKey = asset.storageKey?.key
if (asset.media) {
const mediaInfo = asset.media as unknown as PrismaJson.MediaInfo
if (
mediaInfo.proxyType === 'image' ||
(mediaInfo.imageTranscodes && mediaInfo.imageTranscodes.length > 0)
) {
if (mediaInfo.imageTranscodes && mediaInfo.imageTranscodes.length > 0) {
mediaKey = mediaInfo.imageTranscodes[0].key || mediaKey
}
} else if (
mediaInfo.proxyType === 'video' ||
(mediaInfo.videoTranscodes && mediaInfo.videoTranscodes.length > 0) ||
mediaInfo.videoPreview?.key
) {
if (mediaInfo.videoTranscodes && mediaInfo.videoTranscodes.length > 0) {
mediaKey = mediaInfo.videoTranscodes[0].key || mediaKey
} else if (mediaInfo.videoPreview?.key) {
mediaKey = mediaInfo.videoPreview.key
}
} else if (mediaInfo.original?.key) {
mediaKey = mediaInfo.original.key || mediaKey
}
}
if (!mediaKey) {
throw new Error(`No media content found for asset ${assetId}.`)
}
const safeName = sanitizeFilename(asset.name || 'file')
const filename = `${asset.id}_${safeName}`
const targetFilePath = path.join(piDir, filename)
const relativePath = path.join('.pi', filename)
const { buffer, contentType } = await s3Service.getObject(bucket, mediaKey)
fs.writeFileSync(targetFilePath, buffer)
return {
content: [
{
type: 'text',
text: `Downloaded asset "${asset.name}" (ID: ${assetId}) to "${relativePath}". Remember to delete this temporary file when finished.`,
},
],
details: {
assetId,
name: asset.name,
filePath: relativePath,
absolutePath: targetFilePath,
contentType: contentType || 'application/octet-stream',
size: buffer.length,
},
],
details: {
assetId,
name: asset.name,
filePath: relativePath,
absolutePath: targetFilePath,
contentType: contentType || 'application/octet-stream',
size: buffer.length,
},
}
}
// Branch B: Download by S3 storage key
if (key) {
const owningAssetId = await resolveAssetIdFromKey(key)
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: owningAssetId,
})
const rawBasename = path.basename(key)
const safeBasename = sanitizeFilename(rawBasename) || 'downloaded_file'
const filename = safeBasename
const targetFilePath = path.join(piDir, filename)
const relativePath = path.join('.pi', filename)
const { buffer, contentType } = await s3Service.getObject(bucket, key)
fs.writeFileSync(targetFilePath, buffer)
return {
content: [
{
type: 'text',
text: `Downloaded storage key "${key}" to "${relativePath}". Remember to delete this temporary file when finished.`,
},
],
details: {
key,
assetId: owningAssetId,
filePath: relativePath,
absolutePath: targetFilePath,
contentType: contentType || 'application/octet-stream',
size: buffer.length,
},
}
}
throw new Error('Invalid download_asset parameters.')
},
}
}
@@ -0,0 +1,79 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createListAssetsTool } from './list-assets'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
Permission: { Edit: 'edit', Read: 'read' },
ResourceType: { Asset: 'asset' },
}))
vi.mock('@shumai/core/src/asset/asset', () => ({
assetService: {
listChildren: vi.fn().mockResolvedValue({
data: [
{ id: '01M2FILE2', name: 'file2.txt', type: 'file', sizeByte: 200 },
{ id: '01M1FILE1', name: 'file1.txt', type: 'file', sizeByte: 100 },
],
pageInfo: { hasNextPage: false, hasPreviousPage: false, cursor: 'abc' },
}),
},
}))
describe('createListAssetsTool', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('checks read permissions and invokes assetService.listChildren with default all and id desc ordering', async () => {
const tool = createListAssetsTool('user-1')
const result = await tool.execute('call-1', { parent: 'folder-1' })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'read',
type: 'asset',
id: 'folder-1',
})
expect(assetService.listChildren).toHaveBeenCalledWith({
assetId: 'folder-1',
assetType: 'all',
sort: 'createdAt',
order: 'desc',
first: 20,
after: undefined,
})
expect(result.details).toEqual({})
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({
assets: [
{ id: '01M2FILE2', name: 'file2.txt', type: 'file', size: 200 },
{ id: '01M1FILE1', name: 'file1.txt', type: 'file', size: 100 },
],
pageInfo: { hasNextPage: false, hasPreviousPage: false, cursor: 'abc' },
})
})
it('passes explicit pagination and type filter options to assetService.listChildren', async () => {
const tool = createListAssetsTool('user-1')
await tool.execute('call-1', {
parent: 'folder-1',
page: 2,
pageSize: 10,
type: 'folder',
})
expect(assetService.listChildren).toHaveBeenCalledWith({
assetId: 'folder-1',
assetType: 'folder',
sort: 'createdAt',
order: 'desc',
first: 10,
after: expect.any(String),
})
})
})
+39 -11
View File
@@ -1,6 +1,9 @@
import { Type } from '@sinclair/typebox'
import { Type } from 'typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { executeAgentToolWorkflow } from './utils'
import { assetService } from '@shumai/core/src/asset/asset'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { encodeCursor } from '@shumai/core/src/pagination'
import { type User } from '@shumai/db'
const listAssetsSchema = Type.Object({
parent: Type.String({
@@ -19,9 +22,8 @@ const listAssetsSchema = Type.Object({
}),
),
type: Type.Optional(
Type.String({
Type.Union([Type.Literal('file'), Type.Literal('folder'), Type.Literal('all')], {
description: "Filter assets by type: 'file', 'folder', or 'all'. Default is 'all'.",
enum: ['file', 'folder', 'all'],
default: 'all',
}),
),
@@ -34,15 +36,41 @@ export function createListAssetsTool(userId: string): AgentTool<typeof listAsset
description: 'List the assets (files and folders) inside a parent folder with pagination.',
parameters: listAssetsSchema,
execute: async (_toolCallId, params) => {
const result = await executeAgentToolWorkflow({
toolName: 'list_assets',
args: params,
userId,
assetId: params.parent,
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: params.parent,
})
const page = params.page || 1
const pageSize = params.pageSize || 20
const type = params.type || 'all'
const result = await assetService.listChildren({
assetId: params.parent,
assetType: type,
sort: 'createdAt',
order: 'desc',
first: pageSize,
after: page > 1 ? encodeCursor((page - 1) * pageSize) : undefined,
})
const assets = result.data.map((a) => ({
id: a.id,
name: a.name,
type: a.type,
size: Number(a.sizeByte),
}))
const output = {
assets,
pageInfo: result.pageInfo,
}
return {
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
details: result,
content: [{ type: 'text', text: JSON.stringify(output, null, 2) }],
details: {},
}
},
}
@@ -1,31 +0,0 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createListAutofillFieldsTool } from './list-autofill-fields'
import { executeAgentToolWorkflow } from './utils'
vi.mock('./utils', () => ({
executeAgentToolWorkflow: vi.fn().mockResolvedValue({
fields: [{ name: 'Source', type: 'select', options: [{ displayName: 'Gemini' }] }],
}),
}))
describe('createListAutofillFieldsTool', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('should list autofill fields for the parent folder via the agent tool workflow', async () => {
const tool = createListAutofillFieldsTool('user-1')
const result = await tool.execute('call-1', { parent: 'folder-1' })
expect(executeAgentToolWorkflow).toHaveBeenCalledWith({
toolName: 'list_autofill_fields',
args: { parent: 'folder-1' },
userId: 'user-1',
assetId: 'folder-1',
})
expect(result.details).toEqual({
fields: [{ name: 'Source', type: 'select', options: [{ displayName: 'Gemini' }] }],
})
})
})
@@ -1,35 +0,0 @@
import { Type } from '@sinclair/typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { executeAgentToolWorkflow } from './utils'
const listAutofillFieldsSchema = Type.Object({
parent: Type.String({
description:
'The parent folder ID whose project autofillable metadata fields to list. This parameter is required.',
}),
})
export function createListAutofillFieldsTool(
userId: string,
): AgentTool<typeof listAutofillFieldsSchema> {
return {
name: 'list_autofill_fields',
label: 'List Autofill Fields',
description:
'List the AI-autofillable metadata fields defined for the project of a parent folder, including their types and available options. ' +
'Use this before creating a file or version so you can pass relevant context (e.g. the model or prompt used to generate the asset) to create_file/create_version.',
parameters: listAutofillFieldsSchema,
execute: async (_toolCallId, params) => {
const result = await executeAgentToolWorkflow({
toolName: 'list_autofill_fields',
args: params,
userId,
assetId: params.parent,
})
return {
content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
details: result,
}
},
}
}
+544
View File
@@ -0,0 +1,544 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createReadAssetTool } from './read-asset'
import {
prisma,
WorkflowTaskType,
WorkflowTaskStatus,
type Asset,
type WorkflowTask,
} from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService } from '@shumai/core/src/authz/authz'
import * as annotationResolver from './annotation-resolver'
vi.mock('@shumai/db', async (importOriginal) => {
const actual = await importOriginal<typeof import('@shumai/db')>()
return {
...actual,
prisma: {
asset: {
findUnique: vi.fn(),
findFirst: vi.fn(),
},
workflowTask: {
create: vi.fn(),
},
},
}
})
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
getObject: vi.fn(),
presign: vi.fn().mockResolvedValue('https://mock-presigned-url.com'),
},
}))
vi.mock('@shumai/workflow-core', () => ({
workflowService: {
executeWait: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn(),
},
Permission: { Read: 'Read' },
ResourceType: { Asset: 'asset' },
}))
describe('readAssetTool', () => {
beforeEach(() => {
vi.clearAllMocks()
vi.spyOn(annotationResolver, 'resolveAnnotationsById').mockResolvedValue({
annotations: null,
timestamp: null,
})
})
it('should throw authorization error if user ID is empty', async () => {
const tool = createReadAssetTool('')
await expect(
tool.execute('call-1', {
assetId: 'asset-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
}),
).rejects.toThrow('User ID is required for authorization.')
})
it('should throw error if asset not found', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'asset-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
}),
).rejects.toThrow('Asset with ID asset-1 not found.')
})
describe('Image Assets', () => {
it('should read image and return ImageContent and S3 key', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'img-1',
name: 'photo.png',
mediaType: 'image/png',
storageKey: { key: 'raw/photo.png' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/photo.webp' }],
},
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('fake-image-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'img-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
})
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'img-1',
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'proxy/photo.webp')
const textItem = result.content[0] as { type: 'text'; text: string }
const imageItem = result.content[1] as { type: 'image'; data: string; mimeType: string }
expect(textItem.text).toContain('proxy/photo.webp')
expect(imageItem.type).toBe('image')
expect(imageItem.data).toBe(Buffer.from('fake-image-bytes').toString('base64'))
expect(result.details.key).toBe('proxy/photo.webp')
})
it('should trigger image annotation transcode workflow when annotationId is provided', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'img-1',
name: 'photo.png',
mediaType: 'image/png',
projectId: 'project-1',
storageKey: { key: 'raw/photo.png' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/photo.webp' }],
},
} as unknown as Asset)
vi.spyOn(annotationResolver, 'resolveAnnotationsById').mockResolvedValue({
annotations: [
{
type: 'arrow',
color: '#ff0000',
points: [
[0, 0],
[10, 10],
],
},
],
timestamp: null,
})
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-anno',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-anno',
output: { key: 'annotated/photo.webp' },
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('annotated-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'img-1',
annotationId: 'comment-1',
imageConfig: null,
videoConfig: null,
docConfig: null,
})
expect(prisma.workflowTask.create).toHaveBeenCalledWith({
data: {
assetId: 'img-1',
projectId: 'project-1',
type: WorkflowTaskType.transcode_image_annotation,
status: WorkflowTaskStatus.pending,
payload: {
projectId: 'project-1',
imageAnnotation: {
annotations: [
{
type: 'arrow',
color: '#ff0000',
points: [
[0, 0],
[10, 10],
],
},
],
},
},
},
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'annotated/photo.webp')
expect(result.details.key).toBe('annotated/photo.webp')
})
})
describe('Video Assets', () => {
it('should throw error if videoConfig is missing for video assets', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'vid-1',
name: 'video.mp4',
mediaType: 'video/mp4',
media: { proxyType: 'video' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'vid-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
}),
).rejects.toThrow('videoConfig with start, end, and count is required')
})
it('should throw error on invalid video time range', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'vid-1',
name: 'video.mp4',
mediaType: 'video/mp4',
media: { proxyType: 'video' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'vid-1',
annotationId: null,
imageConfig: null,
videoConfig: { start: 10, end: 5, count: 1 },
docConfig: null,
}),
).rejects.toThrow(
'Invalid video time range: start (10) must be less than or equal to end (5).',
)
})
it('should extract frames and return ImageContent array and keys', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'vid-1',
name: 'video.mp4',
mediaType: 'video/mp4',
projectId: 'project-1',
media: { proxyType: 'video' },
} as unknown as Asset)
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-shot',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-shot',
output: {
screenshots: [
{ key: 'screenshots/shot1.webp', timestamp: 2.0 },
{ key: 'screenshots/shot2.webp', timestamp: 4.0 },
],
},
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockImplementation(async (_bucket, key) => ({
buffer: Buffer.from(`bytes-for-${key}`),
contentType: 'image/webp',
}))
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'vid-1',
annotationId: null,
imageConfig: null,
videoConfig: { start: 2.0, end: 4.0, count: 2 },
docConfig: null,
})
expect(prisma.workflowTask.create).toHaveBeenCalledWith({
data: {
assetId: 'vid-1',
projectId: 'project-1',
type: WorkflowTaskType.transcode_screenshot,
status: WorkflowTaskStatus.pending,
payload: {
projectId: 'project-1',
screenshot: {
start: 2.0,
end: 4.0,
count: 2,
commentTimestamp: null,
annotations: null,
},
},
},
})
expect(result.content.length).toBe(3) // 1 text + 2 images
expect(result.details.keys).toEqual(['screenshots/shot1.webp', 'screenshots/shot2.webp'])
})
})
describe('Document Assets', () => {
it('should throw error if docConfig is missing for document assets', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'doc.pdf',
mediaType: 'application/pdf',
media: { proxyType: 'pdf' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'doc-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
}),
).rejects.toThrow('docConfig with mode ("pages" or "text") is required')
})
it('should render PDF pages in mode: "pages"', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'doc.pdf',
mediaType: 'application/pdf',
projectId: 'project-1',
media: { proxyType: 'pdf' },
} as unknown as Asset)
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-pdf',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-pdf',
output: {
pages: [
{ key: 'pdf_pages/page1.webp', page: 1 },
{ key: 'pdf_pages/page2.webp', page: 2 },
],
},
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockImplementation(async (_bucket, key) => ({
buffer: Buffer.from(`bytes-for-${key}`),
contentType: 'image/webp',
}))
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'doc-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'pages', startPage: 1, endPage: 2 },
})
expect(result.content.length).toBe(3) // 1 text + 2 images
expect(result.details.keys).toEqual(['pdf_pages/page1.webp', 'pdf_pages/page2.webp'])
})
it('should throw error in mode: "pages" when page range exceeds 20 pages', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'doc.pdf',
mediaType: 'application/pdf',
media: { proxyType: 'pdf' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'doc-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'pages', startPage: 1, endPage: 25 },
}),
).rejects.toThrow('exceeds the maximum limit of 20 pages')
})
it('should throw error in mode: "text" when called on binary PDF', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'manual.pdf',
mediaType: 'application/pdf',
media: { proxyType: 'pdf' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'doc-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'text', startPage: null, endPage: null },
}),
).rejects.toThrow('is a binary PDF/document and cannot be read as raw text')
})
it('should throw error in mode: "text" when annotationId is provided', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'notes.md',
mediaType: 'text/markdown',
media: { proxyType: 'pdf' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'doc-1',
annotationId: 'comment-123',
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'text', startPage: null, endPage: null },
}),
).rejects.toThrow('annotationId cannot be used with docConfig mode "text"')
})
it('should throw error in mode: "text" when startPage or endPage is non-null', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-1',
name: 'notes.md',
mediaType: 'text/markdown',
media: { proxyType: 'pdf' },
} as unknown as Asset)
const tool = createReadAssetTool('user-1')
await expect(
tool.execute('call-1', {
assetId: 'doc-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'text', startPage: 1, endPage: 2 },
}),
).rejects.toThrow('startPage and endPage must be null when docConfig mode is "text"')
})
it('should read raw text from S3 in mode: "text" for markdown files', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'doc-md',
name: 'README.md',
mediaType: 'text/markdown',
storageKey: { key: 'raw/README.md' },
media: { proxyType: 'pdf' },
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('# Hello Shumai\nThis is markdown text.'),
contentType: 'text/markdown',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'doc-md',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: { mode: 'text', startPage: null, endPage: null },
})
expect(s3Service.getObject).toHaveBeenCalledWith('shumai', 'raw/README.md')
expect(result.content[0].type).toBe('text')
expect((result.content[0] as { type: 'text'; text: string }).text).toContain(
'# Hello Shumai\nThis is markdown text.',
)
expect(result.details.key).toBe('raw/README.md')
})
})
describe('Version Stack Resolution', () => {
it('should resolve version_stack to latest version asset', async () => {
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'stack-1',
name: '',
type: 'version_stack',
} as unknown as Asset)
vi.mocked(prisma.asset.findFirst).mockResolvedValue({
id: 'file-v2',
name: 'diagram_v2.png',
type: 'file',
mediaType: 'image/png',
storageKey: { key: 'raw/diagram_v2.png' },
media: {
proxyType: 'image',
imageTranscodes: [{ key: 'proxy/diagram_v2.webp' }],
},
} as unknown as Asset)
vi.mocked(s3Service.getObject).mockResolvedValue({
buffer: Buffer.from('v2-image-bytes'),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string })
const tool = createReadAssetTool('user-1')
const result = await tool.execute('call-1', {
assetId: 'stack-1',
annotationId: null,
imageConfig: null,
videoConfig: null,
docConfig: null,
})
expect(prisma.asset.findFirst).toHaveBeenCalledWith({
where: { parentId: 'stack-1', isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
})
expect(result.details.name).toBe('diagram_v2.png')
expect(result.details.key).toBe('proxy/diagram_v2.webp')
})
})
})
+576
View File
@@ -0,0 +1,576 @@
import { Type } from 'typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { type ImageContent, type TextContent } from '@earendil-works/pi-ai'
import { prisma, WorkflowTaskType, WorkflowTaskStatus, type User } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { resolveAnnotationsById } from './annotation-resolver'
export const readAssetSchema = Type.Object(
{
assetId: Type.String({
description: 'The asset ID of the workspace asset to inspect. Required.',
}),
annotationId: Type.Union([
Type.String({
description:
'The ID of the comment or message entry from <annotation id="..." /> whose visual markup should be overlaid, or null.',
}),
Type.Null(),
]),
imageConfig: Type.Union([
Type.Object(
{},
{
additionalProperties: false,
description: 'Optional configuration for image assets, or null.',
},
),
Type.Null(),
]),
videoConfig: Type.Union([
Type.Object(
{
start: Type.Number({ description: 'Start time in seconds.' }),
end: Type.Number({ description: 'End time in seconds.' }),
count: Type.Number({ description: 'Number of frames to extract (e.g. 1 to 10).' }),
},
{
additionalProperties: false,
description:
'Configuration for video assets. Required when inspecting videos, or null for other asset types.',
},
),
Type.Null(),
]),
docConfig: Type.Union([
Type.Object(
{
mode: Type.Union([
Type.Literal('pages', {
description:
'Render specific page range as images for visual layout, charts, or visual annotations (required for binary PDFs).',
}),
Type.Literal('text', {
description:
'Read raw text content directly (ideal for .md, .txt, .csv, code files).',
}),
]),
startPage: Type.Union([
Type.Number({
description:
'Start page number (1-based index, required when mode="pages"), or null when mode="text".',
}),
Type.Null(),
]),
endPage: Type.Union([
Type.Number({
description:
'End page number (1-based index, max 20 pages per call, required when mode="pages"), or null when mode="text".',
}),
Type.Null(),
]),
},
{
additionalProperties: false,
description:
'Configuration for document/PDF/text assets, or null if inspecting another asset type.',
},
),
Type.Null(),
]),
},
{ additionalProperties: false },
)
export function isPlainTextAsset(mediaType: string, filename: string): boolean {
if (
mediaType.startsWith('text/') ||
mediaType === 'application/json' ||
mediaType === 'application/javascript' ||
mediaType === 'application/typescript' ||
mediaType === 'application/xml' ||
mediaType === 'application/x-yaml'
) {
return true
}
const textExtensions = [
'.txt',
'.md',
'.markdown',
'.json',
'.csv',
'.tsv',
'.js',
'.ts',
'.tsx',
'.jsx',
'.html',
'.htm',
'.css',
'.scss',
'.yaml',
'.yml',
'.xml',
'.py',
'.rb',
'.go',
'.rs',
'.c',
'.cpp',
'.h',
'.hpp',
'.java',
'.sh',
'.bash',
'.zsh',
'.sql',
]
return textExtensions.some((ext) => filename.endsWith(ext))
}
const MAX_TEXT_BYTES = 50 * 1024
const MAX_TEXT_LINES = 1000
export function createReadAssetTool(userId: string): AgentTool<typeof readAssetSchema> {
return {
name: 'read_asset',
label: 'Read Asset',
description:
'Inspects and reads the content of an asset in the workspace. ' +
'Supports visual analysis for images, video frame extraction, PDF page rendering, and direct text reading for text/markdown files. ' +
'Pass assetId, and provide the specific config (imageConfig, videoConfig, or docConfig) while setting unused configs to null. ' +
'Optionally provide annotationId to view visual markups drawn on the asset.',
parameters: readAssetSchema,
execute: async (_toolCallId, params) => {
const { assetId, annotationId, videoConfig, docConfig } = params
if (!userId) {
throw new Error('User ID is required for authorization.')
}
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: assetId,
})
let asset = await prisma.asset.findUnique({
where: { id: assetId },
include: { storageKey: true },
})
if (!asset) {
throw new Error(`Asset with ID ${assetId} not found.`)
}
if (asset.type === 'version_stack') {
const latestVersion = await prisma.asset.findFirst({
where: { parentId: asset.id, isDeleted: false },
orderBy: { sortIndex: 'asc' },
include: { storageKey: true },
})
if (latestVersion) {
asset = latestVersion
}
}
const mediaInfo = asset.media as unknown as PrismaJson.MediaInfo | null
const proxyType = mediaInfo?.proxyType
const mediaType = asset.mediaType?.toLowerCase() || ''
const filename = asset.name?.toLowerCase() || ''
const bucket = process.env.S3_BUCKET || 'shumai'
// ----------------------------------------------------------------------
// Branch 1: Image Assets
// ----------------------------------------------------------------------
if (proxyType === 'image' || mediaType.startsWith('image/') || filename.endsWith('.psd')) {
let mediaKey = asset.storageKey?.key
if (mediaInfo?.imageTranscodes && mediaInfo.imageTranscodes.length > 0) {
mediaKey = mediaInfo.imageTranscodes[0].key || mediaKey
}
if (!mediaKey) {
throw new Error('No media content found for this asset.')
}
const { annotations } = await resolveAnnotationsById(asset.id, annotationId)
let keyToUse = mediaKey
if (annotations) {
const task = await prisma.workflowTask.create({
data: {
assetId: asset.id,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_image_annotation,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
imageAnnotation: {
annotations,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- transcode image annotation task output
const output = completedTask.output as any
if (output?.key) {
keyToUse = output.key
} else {
throw new Error('Image annotation transcode workflow failed to output S3 key.')
}
}
const { buffer, contentType } = await s3Service.getObject(bucket, keyToUse)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
const downloadUrl = await s3Service.presign(
bucket,
keyToUse,
'GET',
true,
asset.name || undefined,
)
const content: Array<TextContent | ImageContent> = [
{
type: 'text',
text: `Image asset "${asset.name}" (ID: ${asset.id}):\n- S3 Key: "${keyToUse}"`,
},
{
type: 'image',
data: buffer.toString('base64'),
mimeType,
},
]
return {
content,
details: {
assetId: asset.id,
name: asset.name,
mediaType: asset.mediaType,
key: keyToUse,
downloadUrl,
sourceKeys: [keyToUse],
},
}
}
// ----------------------------------------------------------------------
// Branch 2: Video Assets
// ----------------------------------------------------------------------
if (proxyType === 'video' || mediaType.startsWith('video/')) {
if (!videoConfig) {
throw new Error(
'videoConfig with start, end, and count is required when inspecting video assets.',
)
}
const { start, end, count } = videoConfig
if (start < 0 || end < 0) {
throw new Error('Invalid video time range: start and end must be non-negative numbers.')
}
if (start > end) {
throw new Error(
`Invalid video time range: start (${start}) must be less than or equal to end (${end}).`,
)
}
if (count < 1) {
throw new Error(`Invalid frame count: count (${count}) must be at least 1.`)
}
const { annotations, timestamp: commentTimestamp } = await resolveAnnotationsById(
asset.id,
annotationId,
)
const task = await prisma.workflowTask.create({
data: {
assetId: asset.id,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_screenshot,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
screenshot: {
start,
end,
count,
commentTimestamp,
annotations,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- transcode screenshot output
const output = completedTask.output as any
const screenshots = output?.screenshots as
| Array<{ key: string; timestamp: number }>
| undefined
if (!screenshots || screenshots.length === 0) {
throw new Error('No screenshots were generated.')
}
const content: Array<TextContent | ImageContent> = []
const sourceKeys: string[] = []
const results: Array<{ key: string; timestamp: number; downloadUrl: string }> = []
const textLines: string[] = [
`Extracted ${screenshots.length} frame(s) from "${asset.name}" (${start}s - ${end}s):`,
]
for (const shot of screenshots) {
const { buffer, contentType } = await s3Service.getObject(bucket, shot.key)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
const downloadUrl = await s3Service.presign(
bucket,
shot.key,
'GET',
true,
asset.name || undefined,
)
textLines.push(`- Frame at ${shot.timestamp}s: S3 Key: "${shot.key}"`)
content.push({
type: 'image',
data: buffer.toString('base64'),
mimeType,
})
sourceKeys.push(shot.key)
results.push({
key: shot.key,
timestamp: shot.timestamp,
downloadUrl,
})
}
content.unshift({
type: 'text',
text: textLines.join('\n'),
})
return {
content,
details: {
assetId: asset.id,
name: asset.name,
mediaType: asset.mediaType,
keys: sourceKeys,
results,
sourceKeys,
},
}
}
// ----------------------------------------------------------------------
// Branch 3: Documents & PDF Assets
// ----------------------------------------------------------------------
if (
proxyType === 'pdf' ||
mediaType === 'application/pdf' ||
isPlainTextAsset(mediaType, filename)
) {
if (!docConfig) {
throw new Error(
'docConfig with mode ("pages" or "text") is required when inspecting document/PDF assets.',
)
}
if (docConfig.mode === 'text') {
if (annotationId !== null) {
throw new Error(
'annotationId cannot be used with docConfig mode "text" because visual markups can only be rendered on visual pages. Use docConfig: { mode: "pages", startPage: ..., endPage: ... } instead.',
)
}
if (docConfig.startPage !== null || docConfig.endPage !== null) {
throw new Error(
'startPage and endPage must be null when docConfig mode is "text". To inspect specific page numbers, use mode: "pages".',
)
}
if (!isPlainTextAsset(mediaType, filename)) {
throw new Error(
`Asset "${asset.name}" is a binary PDF/document and cannot be read as raw text. Please call read_asset with docConfig: { mode: "pages", startPage: 1, endPage: ... } to view its visual pages.`,
)
}
const mediaKey = asset.storageKey?.key
if (!mediaKey) {
throw new Error(`No media content found for asset ${asset.id}.`)
}
const { buffer } = await s3Service.getObject(bucket, mediaKey)
const rawText = buffer.toString('utf-8')
let text = rawText
let isTruncated = false
const lines = text.split('\n')
if (lines.length > MAX_TEXT_LINES) {
text = lines.slice(0, MAX_TEXT_LINES).join('\n')
isTruncated = true
}
if (Buffer.byteLength(text, 'utf-8') > MAX_TEXT_BYTES) {
text = Buffer.from(text, 'utf-8').subarray(0, MAX_TEXT_BYTES).toString('utf-8')
isTruncated = true
}
if (isTruncated) {
text += `\n\n[Content truncated to ${MAX_TEXT_LINES} lines / ${MAX_TEXT_BYTES / 1024}KB limit. Use download_asset to process the full file.]`
}
const downloadUrl = await s3Service.presign(
bucket,
mediaKey,
'GET',
true,
asset.name || undefined,
)
return {
content: [
{
type: 'text',
text: `Document "${asset.name}" (ID: ${asset.id}, S3 Key: "${mediaKey}"):\n\n${text}`,
},
],
details: {
assetId: asset.id,
name: asset.name,
mediaType: asset.mediaType,
key: mediaKey,
downloadUrl,
sourceKeys: [mediaKey],
size: buffer.length,
isTruncated,
},
}
}
// docConfig.mode === 'pages'
if (docConfig.startPage === null || docConfig.endPage === null) {
throw new Error('startPage and endPage are required when docConfig mode is "pages".')
}
const start = docConfig.startPage
const end = docConfig.endPage
if (start < 1) {
throw new Error(`Invalid page range: start page (${start}) must be at least 1.`)
}
if (start > end) {
throw new Error(
`Invalid page range: start page (${start}) must be less than or equal to end page (${end}).`,
)
}
const MAX_PAGE_RANGE = 20
const requestedPages = end - start + 1
if (requestedPages > MAX_PAGE_RANGE) {
throw new Error(
`Page range (${requestedPages} pages requested) exceeds the maximum limit of ${MAX_PAGE_RANGE} pages per request.`,
)
}
const { annotations, timestamp: commentTimestamp } = await resolveAnnotationsById(
asset.id,
annotationId,
)
const task = await prisma.workflowTask.create({
data: {
assetId: asset.id,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_pdf_pages,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
pdfPages: {
start,
end,
commentTimestamp,
annotations,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- transcode pdf pages output
const output = completedTask.output as any
const pages = output?.pages as Array<{ key: string; page: number }> | undefined
if (!pages || pages.length === 0) {
throw new Error('No PDF page images were generated.')
}
const content: Array<TextContent | ImageContent> = []
const sourceKeys: string[] = []
const results: Array<{ key: string; page: number; downloadUrl: string }> = []
const textLines: string[] = [
`Rendered ${pages.length} page(s) of "${asset.name}" (Pages ${start} to ${end}):`,
]
for (const pageItem of pages) {
const { buffer, contentType } = await s3Service.getObject(bucket, pageItem.key)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
const downloadUrl = await s3Service.presign(
bucket,
pageItem.key,
'GET',
true,
asset.name || undefined,
)
textLines.push(`- Page ${pageItem.page}: S3 Key: "${pageItem.key}"`)
content.push({
type: 'image',
data: buffer.toString('base64'),
mimeType,
})
sourceKeys.push(pageItem.key)
results.push({
key: pageItem.key,
page: pageItem.page,
downloadUrl,
})
}
content.unshift({
type: 'text',
text: textLines.join('\n'),
})
return {
content,
details: {
assetId: asset.id,
name: asset.name,
mediaType: asset.mediaType,
keys: sourceKeys,
results,
sourceKeys,
},
}
}
throw new Error(
`Unsupported asset type for read_asset. Asset ${asset.id} has media type "${mediaType || 'unknown'}". Use download_asset to download this file to disk.`,
)
},
}
}
@@ -1,209 +0,0 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createReadPdfPagesTool } from './read-pdf-pages'
import {
prisma,
WorkflowTaskStatus,
type Asset,
type AssetComment,
type User,
type WorkflowTask,
} from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/db', async (importOriginal) => {
const actual = await importOriginal<typeof import('@shumai/db')>()
return {
...actual,
prisma: {
user: {
findUnique: vi.fn(),
},
asset: {
findUnique: vi.fn(),
},
assetComment: {
findUnique: vi.fn(),
},
workflowTask: {
create: vi.fn(),
},
},
}
})
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
getObject: vi.fn(),
},
}))
vi.mock('@shumai/workflow-core', () => ({
workflowService: {
executeWait: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn(),
},
Permission: { Read: 'Read' },
ResourceType: { Asset: 'asset' },
}))
describe('readPdfPagesTool', () => {
beforeEach(() => {
vi.clearAllMocks()
vi.mocked(authzService.hasPermission).mockResolvedValue()
})
it('should trigger pdfPages transcode workflow and return image outputs', async () => {
vi.mocked(prisma.user.findUnique).mockResolvedValue({ id: 'user-1' } as User)
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
media: { proxyType: 'pdf' },
} as unknown as Asset)
vi.mocked(prisma.assetComment.findUnique).mockResolvedValue({
id: 'comment-1',
assetId: 'asset-1',
second: 2,
annotation: [
{
type: 'box',
color: '#ff0000',
points: [
[0, 0],
[1, 1],
],
},
],
} as unknown as AssetComment)
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-1',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-1',
status: WorkflowTaskStatus.completed,
output: {
pages: [
{ key: 'pdf_pages/page_1.webp', page: 1 },
{ key: 'pdf_pages/page_2.webp', page: 2 },
],
},
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockImplementation(async (_bucket: string, key: string) => {
return {
buffer: Buffer.from(`fake-bytes-for-${key}`),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string }
})
const tool = createReadPdfPagesTool('user-1', 'comment-1')
const result = await tool.execute('call-1', { assetId: 'asset-1', start: 1, end: 2 })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-1',
})
expect(prisma.workflowTask.create).toHaveBeenCalledWith({
data: {
assetId: 'asset-1',
projectId: 'project-1',
type: 'transcode_pdf_pages',
status: 'pending',
payload: {
projectId: 'project-1',
pdfPages: {
start: 1,
end: 2,
commentTimestamp: 2,
annotations: [
{
type: 'box',
color: '#ff0000',
points: [
[0, 0],
[1, 1],
],
},
],
},
},
},
})
expect(result.content.length).toBe(2)
expect(result.content[0].type).toBe('image')
expect((result.content[0] as unknown as { data: string }).data).toBe(
Buffer.from('fake-bytes-for-pdf_pages/page_1.webp').toString('base64'),
)
expect(result.details.sourceKeys).toEqual(['pdf_pages/page_1.webp', 'pdf_pages/page_2.webp'])
})
it('should throw error if start page is less than 1', async () => {
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 0, end: 5 })).rejects.toThrow(
'Invalid page range: start page (0) must be at least 1.',
)
})
it('should throw error if start page is greater than end page', async () => {
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 5, end: 2 })).rejects.toThrow(
'Invalid page range: start page (5) must be less than or equal to end page (2).',
)
})
it('should throw error if page range exceeds maximum limit of 20', async () => {
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 1, end: 25 })).rejects.toThrow(
'Page range (25 pages requested) exceeds the maximum limit of 20 pages per request.',
)
})
it('should throw error if asset not found', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue(null)
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 1, end: 3 })).rejects.toThrow(
'Asset with ID asset-1 not found.',
)
})
it('should throw error if proxyType is not pdf', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
media: { proxyType: 'image' },
} as unknown as Asset)
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 1, end: 3 })).rejects.toThrow(
'Asset asset-1 is not a PDF or document.',
)
})
it('should throw error if proxyType is null/undefined (unprocessed)', async () => {
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
media: null,
} as unknown as Asset)
const tool = createReadPdfPagesTool('user-1')
await expect(tool.execute('call-1', { assetId: 'asset-1', start: 1, end: 3 })).rejects.toThrow(
'Asset asset-1 is not a PDF or document.',
)
})
})
-152
View File
@@ -1,152 +0,0 @@
import { Type } from '@sinclair/typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { type ImageContent } from '@earendil-works/pi-ai'
import { prisma, WorkflowTaskType, WorkflowTaskStatus, type User } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
const readPdfPagesSchema = Type.Object({
assetId: Type.String({
description: 'Asset ID of the PDF document. This parameter is required.',
}),
start: Type.Number({ description: 'Start page number (1-based index)' }),
end: Type.Number({ description: 'End page number (1-based index)' }),
})
export function createReadPdfPagesTool(
userId: string,
userCommentId?: string | null,
): AgentTool<typeof readPdfPagesSchema> {
return {
name: 'read_pdf_pages',
label: 'Read PDF Pages',
description:
'Converts specific page range of a PDF document into 1080p WebP images for visual analysis. Maximum 20 pages allowed per call.',
parameters: readPdfPagesSchema,
execute: async (_toolCallId, params) => {
if (params.start < 1) {
throw new Error(`Invalid page range: start page (${params.start}) must be at least 1.`)
}
if (params.start > params.end) {
throw new Error(
`Invalid page range: start page (${params.start}) must be less than or equal to end page (${params.end}).`,
)
}
const MAX_PAGE_RANGE = 20
const requestedPages = params.end - params.start + 1
if (requestedPages > MAX_PAGE_RANGE) {
throw new Error(
`Page range (${requestedPages} pages requested) exceeds the maximum limit of ${MAX_PAGE_RANGE} pages per request.`,
)
}
const targetAssetId = params.assetId
if (!userId) {
throw new Error('User ID is required for authorization.')
}
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: targetAssetId,
})
let asset = await prisma.asset.findUnique({
where: { id: targetAssetId },
})
if (!asset) {
throw new Error(`Asset with ID ${targetAssetId} not found.`)
}
if (asset.type === 'version_stack') {
const latestVersion = await prisma.asset.findFirst({
where: { parentId: asset.id, isDeleted: false },
orderBy: { sortIndex: 'asc' },
})
if (latestVersion) {
asset = latestVersion
}
}
const proxyType = (asset.media as PrismaJson.MediaInfo | null)?.proxyType
if (proxyType !== 'pdf') {
throw new Error(`Asset ${targetAssetId} is not a PDF or document.`)
}
// Fetch trigger comment details for comment page and annotations
let commentTimestamp: number | null = null
let annotations: unknown = null
if (userCommentId) {
const comment = await prisma.assetComment.findUnique({
where: { id: userCommentId },
})
if (comment && comment.assetId === targetAssetId) {
commentTimestamp = comment.second !== null ? comment.second : null
if (comment.annotation) {
const list = comment.annotation
if (Array.isArray(list) && list.length > 0) {
annotations = list
}
}
}
}
// Trigger transcode workflow to render PDF pages
const task = await prisma.workflowTask.create({
data: {
assetId: targetAssetId,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_pdf_pages,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
pdfPages: {
start: params.start,
end: params.end,
commentTimestamp,
annotations: annotations as PrismaJson.AnnotationList,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const output = completedTask.output as any
const pages = output?.pages as Array<{ key: string; page: number }> | undefined
if (!pages || pages.length === 0) {
throw new Error('No PDF page images were generated.')
}
const content: ImageContent[] = []
const sourceKeys: string[] = []
const bucket = process.env.S3_BUCKET || 'shumai'
for (const pageItem of pages) {
const { buffer, contentType } = await s3Service.getObject(bucket, pageItem.key)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
content.push({
type: 'image',
data: buffer.toString('base64'),
mimeType,
})
sourceKeys.push(pageItem.key)
}
return {
content,
details: {
sourceKeys,
},
}
},
}
}
@@ -307,6 +307,77 @@ describe('readSkillTool', () => {
reviewerTool.execute('1', { skillId: ownerSkill.id }, undefined, undefined),
).rejects.toThrow('Permission denied')
})
it('uses the effective project role for skill permission checks', async () => {
const team = await prisma.team.create({ data: { name: 'Project Skill Team' } })
const project = await prisma.project.create({ data: { name: 'Skill Proj', teamId: team.id } })
const editorSkill = await prisma.skill.create({
data: {
name: 'Editor Project Skill',
assetId: 'asset1',
hash: 'proj-editor-hash',
teamId: team.id,
permission: 'editor',
},
})
// Team editor demoted to project reviewer: denied inside the project
const demoted = await prisma.user.create({
data: { name: 'Demoted Skill', email: 'dem-skill@test.com' },
})
const demotedTm = await prisma.teamMember.create({
data: { teamId: team.id, userId: demoted.id, role: 'editor', scope: 'team' },
})
await prisma.projectMember.create({
data: { projectId: project.id, teamMemberId: demotedTm.id, role: 'reviewer' },
})
const demotedTool = createReadSkillTool(
demoted.id,
() => {},
undefined,
undefined,
project.id,
)
await expect(
demotedTool.execute('1', { skillId: editorSkill.id }, undefined, undefined),
).rejects.toThrow('Permission denied')
// Project-scoped member invited as editor: allowed
const projEditor = await prisma.user.create({
data: { name: 'PE Skill', email: 'pe-skill@test.com' },
})
const projEditorTm = await prisma.teamMember.create({
data: { teamId: team.id, userId: projEditor.id, role: 'reviewer', scope: 'project' },
})
await prisma.projectMember.create({
data: { projectId: project.id, teamMemberId: projEditorTm.id, role: 'editor' },
})
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs calls
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
if (path.toString().endsWith('.hash')) return 'proj-editor-hash'
if (path.toString().endsWith('SKILL.md')) return '# Editor Project Skill Content'
return ''
})
const projEditorTool = createReadSkillTool(
projEditor.id,
() => {},
undefined,
undefined,
project.id,
)
const result = await projEditorTool.execute(
'1',
{ skillId: editorSkill.id },
undefined,
undefined,
)
// eslint-disable-next-line @typescript-eslint/no-explicit-any
expect((result.content[0] as any).text).toBe('# Editor Project Skill Content')
})
})
describe('onSkillLoaded callback', () => {
+9 -10
View File
@@ -1,7 +1,8 @@
import { prisma } from '@shumai/db'
import { Type } from '@sinclair/typebox'
import { Type } from 'typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { agentService } from '@shumai/core/src/agent/agent'
import { resolveEffectiveRole } from '@shumai/core/src/authz/authz'
const readSkillSchema = Type.Object({
skillId: Type.String({ description: 'The ID of the skill to read' }),
@@ -18,6 +19,7 @@ export const createReadSkillTool = (
onEnvsAdded: (envs: Record<string, string>) => void,
onSkillLoaded?: () => Promise<void> | void,
enabledSkillIds?: string[],
projectId?: string,
): AgentTool<typeof readSkillSchema, { skillId: string }> => ({
name: 'read_skill',
label: 'Read Agent Skill',
@@ -39,17 +41,14 @@ export const createReadSkillTool = (
}
const requiredLevel = ROLE_HIERARCHY[skill.permission] || 1
const effectiveRole = userId
? await resolveEffectiveRole(skill.teamId, projectId, userId)
: null
if (userId) {
const member = await prisma.teamMember.findUnique({
where: {
teamIdUserId: {
teamId: skill.teamId,
userId,
},
},
})
const userLevel = member ? ROLE_HIERARCHY[member.role] || 0 : 0
// Resolve the user's effective role for the project context (project
// override wins; project-scoped members without project access are denied).
const userLevel = effectiveRole ? ROLE_HIERARCHY[effectiveRole] || 0 : 0
if (userLevel < requiredLevel) {
throw new Error(
`Permission denied: Insufficient role to load skill "${skill.name}". Minimum required role is "${skill.permission}".`,
+47 -4
View File
@@ -51,6 +51,8 @@ describe('createReadThreadTool', () => {
id: 'root-1',
message: 'Initial top-level question',
creator: { name: 'Alice' },
second: 12.5,
annotation: [{ type: 'box' }],
// eslint-disable-next-line @typescript-eslint/no-explicit-any
} as any)
vi.mocked(prisma.assetComment.findMany).mockResolvedValue([])
@@ -59,7 +61,9 @@ describe('createReadThreadTool', () => {
const result = await tool.execute('call-2', { threadId: 'root-1' })
const textContent = (result.content[0] as { text: string }).text
expect(textContent).toContain('Thread Root [Alice] (root-1): Initial top-level question')
expect(textContent).toContain(
'Thread Root [Alice] (id: root-1, time: 12.5s, has_markup: true): Initial top-level question',
)
expect(textContent).toContain('(No replies in this thread yet)')
})
@@ -75,6 +79,8 @@ describe('createReadThreadTool', () => {
id: 'reply-1',
message: 'First reply to thread',
creator: { name: 'Bob' },
second: 15.0,
annotation: [{ type: 'line' }],
},
{
id: 'reply-2',
@@ -88,8 +94,45 @@ describe('createReadThreadTool', () => {
const result = await tool.execute('call-3', { threadId: 'root-1' })
const textContent = (result.content[0] as { text: string }).text
expect(textContent).toContain('Thread Root [Alice] (root-1): Initial top-level question')
expect(textContent).toContain('- [Bob]: First reply to thread')
expect(textContent).toContain('- [Charlie]: Second reply to thread')
expect(textContent).toContain(
'Thread Root [Alice] (id: root-1, has_markup: false): Initial top-level question',
)
expect(textContent).toContain(
'- [Bob] (id: reply-1, time: 15s, has_markup: true): First reply to thread',
)
expect(textContent).toContain(
'- [Charlie] (id: reply-2, has_markup: false): Second reply to thread',
)
})
it('should label agent replies as [Ai Agent] when creator is null but sessionId exists', async () => {
vi.mocked(prisma.assetComment.findUnique).mockResolvedValue({
id: 'root-1',
message: 'Question to agent',
creator: { name: 'Alice' },
// eslint-disable-next-line @typescript-eslint/no-explicit-any
} as any)
vi.mocked(prisma.assetComment.findMany).mockResolvedValue([
{
id: 'reply-1',
message: 'Agent answer',
creator: null,
sessionId: 'session-123',
},
// eslint-disable-next-line @typescript-eslint/no-explicit-any
] as any)
const tool = createReadThreadTool()
const result = await tool.execute('call-agent', { threadId: 'root-1' })
const textContent = (result.content[0] as { text: string }).text
expect(textContent).toContain('- [Ai Agent] (id: reply-1, has_markup: false): Agent answer')
})
it('should describe threadId parameter as referencing <thread id="..." /> in <context>', () => {
const tool = createReadThreadTool()
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const description = (tool.parameters as any).properties.threadId.description
expect(description).toContain('<thread id="..." />')
})
})
+24 -6
View File
@@ -1,14 +1,28 @@
import { prisma } from '@shumai/db'
import { Type } from '@sinclair/typebox'
import { Type } from 'typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
const readThreadSchema = Type.Object({
threadId: Type.String({
description:
'The root comment ID of a thread (e.g. from [Thread ID: ...]). Do NOT pass an asset ID or project ID.',
'The root comment ID of a thread (from <thread id="..." /> in the <context> block). Do NOT pass an asset ID or project ID.',
}),
})
function formatCommentMeta(c: {
id: string
second?: number | null
annotation?: unknown
}): string {
const parts = [`id: ${c.id}`]
if (c.second !== null && c.second !== undefined) {
parts.push(`time: ${c.second}s`)
}
const hasMarkup = Boolean(c.annotation && Array.isArray(c.annotation) && c.annotation.length > 0)
parts.push(`has_markup: ${hasMarkup}`)
return `(${parts.join(', ')})`
}
export const createReadThreadTool = (): AgentTool<
typeof readThreadSchema,
{ threadId: string }
@@ -47,15 +61,19 @@ export const createReadThreadTool = (): AgentTool<
include: { creator: true },
})
const rootAuthor = rootComment.creator?.name || 'User'
let output = `Thread Root [${rootAuthor}] (${rootComment.id}): ${rootComment.message || ''}\n\nReplies (${replies.length}):`
const isRootAgent = rootComment.creator?.type === 'agent' || rootComment.sessionId !== null
const rootAuthor = rootComment.creator?.name || (isRootAgent ? 'Ai Agent' : 'User')
const rootMeta = formatCommentMeta(rootComment)
let output = `Thread Root [${rootAuthor}] ${rootMeta}: ${rootComment.message || ''}\n\nReplies (${replies.length}):`
if (replies.length === 0) {
output += '\n(No replies in this thread yet)'
} else {
for (const reply of replies) {
const author = reply.creator?.name || 'User'
output += `\n- [${author}]: ${reply.message || ''}`
const isAgent = reply.creator?.type === 'agent' || reply.sessionId !== null
const author = reply.creator?.name || (isAgent ? 'Ai Agent' : 'User')
const replyMeta = formatCommentMeta(reply)
output += `\n- [${author}] ${replyMeta}: ${reply.message || ''}`
}
}
+35 -1
View File
@@ -1,7 +1,7 @@
import { logger } from '@shumai/core/src/logger'
import { SandboxManager } from '@anthropic-ai/sandbox-runtime'
import { type AgentTool, type AgentToolResult } from '@earendil-works/pi-agent-core'
import { Type } from '@sinclair/typebox'
import { Type } from 'typebox'
import { spawn } from 'node:child_process'
import { OutputAccumulator } from '../utils/output-accumulator'
import {
@@ -10,6 +10,8 @@ import {
formatSize,
type TruncationResult,
} from '../utils/truncate'
import { quotaService } from '@shumai/core/src/quota/quota-service'
import type { TeamMemberRole } from '@shumai/db'
const BashSource = Type.String({
enum: ['user', 'skill'],
@@ -33,6 +35,9 @@ export interface BashDetails {
}
export interface SandboxedBashOptions {
teamId?: string
userId?: string
role?: TeamMemberRole | null
getBlockedHost?: () => string
clearBlockedHost?: () => void
/**
@@ -66,6 +71,18 @@ export const createSandboxedBashTool = (
'Only commands required by a loaded skill (source="skill") can be executed.',
)
}
if (options?.teamId) {
await quotaService.checkQuota(
{
teamId: options.teamId,
userId: options.userId,
role: options.role,
resource: 'agent_bash_call_count',
resourceData: { command },
},
1,
)
}
options?.clearBlockedHost?.()
const wrappedCommand = await SandboxManager.wrapWithSandbox(command)
@@ -219,6 +236,23 @@ export const createSandboxedBashTool = (
const snapshot = await finishOutput()
const { text: outputText, details: truncationDetails } = formatOutput(snapshot)
if (options?.teamId) {
try {
await quotaService.consumeQuota(
{
teamId: options.teamId,
userId: options.userId,
role: options.role,
resource: 'agent_bash_call_count',
resourceData: { command },
},
1,
)
} catch (err) {
logger.error({ err }, 'Failed to record quota usage for bash command')
}
}
if (signal?.aborted) {
logger.error({ command, outputText }, 'Sandboxed bash command aborted')
reject(new Error(appendStatus(outputText, 'Command aborted')))
-156
View File
@@ -1,156 +0,0 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createScreenshotTool } from './screenshot'
import {
prisma,
WorkflowTaskStatus,
type Asset,
type AssetComment,
type User,
type WorkflowTask,
} from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService } from '@shumai/core/src/authz/authz'
vi.mock('@shumai/db', async (importOriginal) => {
const actual = await importOriginal<typeof import('@shumai/db')>()
return {
...actual,
prisma: {
user: {
findUnique: vi.fn(),
},
asset: {
findUnique: vi.fn(),
},
assetComment: {
findUnique: vi.fn(),
},
workflowTask: {
create: vi.fn(),
},
},
}
})
vi.mock('@shumai/core/src/s3/s3', () => ({
s3Service: {
getObject: vi.fn(),
},
}))
vi.mock('@shumai/workflow-core', () => ({
workflowService: {
executeWait: vi.fn(),
},
}))
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn(),
},
Permission: { Read: 'Read' },
ResourceType: { Asset: 'asset' },
}))
describe('screenshotTool', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('should trigger screenshot transcode workflow and return image outputs', async () => {
vi.mocked(prisma.user.findUnique).mockResolvedValue({ id: 'user-1' } as User)
vi.mocked(authzService.hasPermission).mockResolvedValue()
vi.mocked(prisma.asset.findUnique).mockResolvedValue({
id: 'asset-1',
projectId: 'project-1',
} as unknown as Asset)
vi.mocked(prisma.assetComment.findUnique).mockResolvedValue({
id: 'comment-1',
assetId: 'asset-1',
second: 5.0,
annotation: [
{
type: 'line',
color: '#00ff00',
points: [
[0, 0],
[1, 1],
],
},
],
} as unknown as AssetComment)
vi.mocked(prisma.workflowTask.create).mockResolvedValue({
id: 'task-1',
} as unknown as WorkflowTask)
vi.mocked(workflowService.executeWait).mockResolvedValue({
id: 'task-1',
status: WorkflowTaskStatus.completed,
output: {
screenshots: [
{ key: 'screenshots/shot1.webp', timestamp: 0.0 },
{ key: 'screenshots/shot2.webp', timestamp: 5.0 },
],
},
} as unknown as WorkflowTask)
vi.mocked(s3Service.getObject).mockImplementation(async (_bucket: string, key: string) => {
return {
buffer: Buffer.from(`fake-bytes-for-${key}`),
contentType: 'image/webp',
} as unknown as { buffer: Buffer; contentType: string }
})
const tool = createScreenshotTool('user-1', 'comment-1')
const result = await tool.execute('call-1', { assetId: 'asset-1', start: 0, end: 10, count: 2 })
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user-1' },
permission: 'Read',
type: 'asset',
id: 'asset-1',
})
expect(prisma.workflowTask.create).toHaveBeenCalledWith({
data: {
assetId: 'asset-1',
projectId: 'project-1',
type: 'transcode_screenshot',
status: 'pending',
payload: {
projectId: 'project-1',
screenshot: {
start: 0,
end: 10,
count: 2,
commentTimestamp: 5.0,
annotations: [
{
type: 'line',
color: '#00ff00',
points: [
[0, 0],
[1, 1],
],
},
],
},
},
},
})
expect(result.content.length).toBe(2)
expect(result.content[0].type).toBe('image')
expect((result.content[0] as unknown as { data: string }).data).toBe(
Buffer.from('fake-bytes-for-screenshots/shot1.webp').toString('base64'),
)
expect((result.content[1] as unknown as { data: string }).data).toBe(
Buffer.from('fake-bytes-for-screenshots/shot2.webp').toString('base64'),
)
expect(result.details.sourceKeys).toEqual(['screenshots/shot1.webp', 'screenshots/shot2.webp'])
})
})
-123
View File
@@ -1,123 +0,0 @@
import { Type } from '@sinclair/typebox'
import { type AgentTool } from '@earendil-works/pi-agent-core'
import { type ImageContent } from '@earendil-works/pi-ai'
import { prisma, WorkflowTaskType, WorkflowTaskStatus, type User } from '@shumai/db'
import { s3Service } from '@shumai/core/src/s3/s3'
import { workflowService } from '@shumai/workflow-core'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
const screenshotSchema = Type.Object({
assetId: Type.String({
description: 'The asset ID of the video asset. This parameter is required.',
}),
start: Type.Number({ description: 'Start time in seconds' }),
end: Type.Number({ description: 'End time in seconds' }),
count: Type.Number({ description: 'Number of screenshots to take' }),
})
export function createScreenshotTool(
userId: string,
userCommentId?: string | null,
): AgentTool<typeof screenshotSchema> {
return {
name: 'screenshot',
label: 'Take Video Screenshots',
description:
'Takes screenshots for a video at specified interval. It returns the images to the agent context.',
parameters: screenshotSchema,
execute: async (_toolCallId, params) => {
const assetId = params.assetId
if (!userId) {
throw new Error('User ID is required for authorization.')
}
await authzService.hasPermission({
user: { id: userId } as User,
permission: Permission.Read,
type: ResourceType.Asset,
id: assetId,
})
const asset = await prisma.asset.findUnique({
where: { id: assetId },
})
if (!asset) {
throw new Error(`Asset with ID ${assetId} not found.`)
}
// Fetch trigger comment details for timestamp and annotations
let commentTimestamp: number | null = null
let annotations: unknown = null
if (userCommentId) {
const comment = await prisma.assetComment.findUnique({
where: { id: userCommentId },
})
if (comment && comment.assetId === assetId) {
commentTimestamp = comment.second !== null ? comment.second : null
if (comment.annotation) {
const list = comment.annotation
if (Array.isArray(list) && list.length > 0) {
annotations = list
}
}
}
}
// Trigger transcode workflow to take screenshots
const task = await prisma.workflowTask.create({
data: {
assetId,
projectId: asset.projectId || 'none',
type: WorkflowTaskType.transcode_screenshot,
status: WorkflowTaskStatus.pending,
payload: {
projectId: asset.projectId || 'none',
screenshot: {
start: params.start,
end: params.end,
count: params.count,
commentTimestamp,
annotations: annotations as PrismaJson.AnnotationList,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const output = completedTask.output as any
const screenshots = output?.screenshots as
| Array<{ key: string; timestamp: number }>
| undefined
if (!screenshots || screenshots.length === 0) {
throw new Error('No screenshots were generated.')
}
const content: ImageContent[] = []
const sourceKeys: string[] = []
const bucket = process.env.S3_BUCKET || 'shumai'
for (const shot of screenshots) {
const { buffer, contentType } = await s3Service.getObject(bucket, shot.key)
const mimeType =
contentType && contentType !== 'application/octet-stream' ? contentType : 'image/webp'
content.push({
type: 'image',
data: buffer.toString('base64'),
mimeType,
})
sourceKeys.push(shot.key)
}
return {
content,
details: {
sourceKeys,
},
}
},
}
}
-35
View File
@@ -1,35 +0,0 @@
import { prisma } from '@shumai/db'
import { workflowService } from '@shumai/workflow-core'
import { WorkflowTaskType, WorkflowTaskStatus } from '@shumai/db'
export async function executeAgentToolWorkflow(params: {
toolName: string
args: Record<string, unknown>
userId: string
assetId: string
}): Promise<unknown> {
const task = await prisma.workflowTask.create({
data: {
type: WorkflowTaskType.agent_tool_call,
status: WorkflowTaskStatus.pending,
assetId: params.assetId,
payload: {
projectId: 'none',
agentToolCall: {
toolName: params.toolName,
args: params.args,
userId: params.userId,
},
},
},
})
const completedTask = await workflowService.executeWait(task)
if (completedTask.status === WorkflowTaskStatus.failed) {
const output = completedTask.output as Record<string, unknown> | null
const errorMsg = (output?.error as string) || 'Workflow task failed'
throw new Error(errorMsg)
}
return completedTask.output
}
@@ -64,6 +64,7 @@ describe('Agent Autofill Workflow', () => {
mockActivities.createCommentActivity.mockResolvedValue({ id: 'comment-placeholder-id' })
mockActivities.getAssetActivity.mockResolvedValue({
id: 'a1',
projectId: 'p1',
storageKey: { key: 'asset-key' },
project: { id: 'p1', teamId: 't1' },
mediaType: 'image/png',
@@ -78,7 +79,6 @@ describe('Agent Autofill Workflow', () => {
{ key: 'title', config: { name: 'Title', type: 'text' }, description: 'The title' },
])
mockActivities.getAgentAutofillContextActivity.mockResolvedValue({ agent: { id: 'b1' } })
mockActivities.getAssetAutofillContextActivity.mockResolvedValue('Generated using gemini')
mockActivities.autofillAiActivity.mockResolvedValue({
text: '{"title":"Extracted Title"}',
sessionId: 'session-123',
@@ -118,6 +118,9 @@ describe('Agent Autofill Workflow', () => {
status: 'processing',
})
// Verify asset fetched
expect(mockActivities.getAssetActivity).toHaveBeenCalledWith('a1')
// Verify placeholder comment created
expect(mockActivities.createCommentActivity).toHaveBeenCalledWith({
assetId: 'a1',
@@ -140,11 +143,12 @@ describe('Agent Autofill Workflow', () => {
expect(mockActivities.getAgentAutofillContextActivity).toHaveBeenCalledWith({
teamId: 't1',
})
expect(mockActivities.getAssetAutofillContextActivity).toHaveBeenCalledWith('a1')
// Verify AI autofill called with mapped fields
expect(mockActivities.autofillAiActivity).toHaveBeenCalledWith({
teamId: 't1',
assetId: 'a1',
projectId: 'p1',
images: ['/tmp/test-dir/1.webp'],
fields: [
{
@@ -154,7 +158,6 @@ describe('Agent Autofill Workflow', () => {
},
],
context: { agent: { id: 'b1' } },
agentContext: 'Generated using gemini',
})
// Verify task usage update
@@ -195,42 +198,6 @@ describe('Agent Autofill Workflow', () => {
})
})
it('should run autofill without agentContext when the asset has no context', async () => {
mockActivities.getAssetAutofillContextActivity.mockResolvedValue(null)
const task = await prisma.workflowTask.create({
data: {
type: 'ai_metadata_autofill',
status: 'pending',
assetId: 'a1',
payload: {
projectId: 'p1',
agent: { sessionId: 's1', agentId: 'agent-1' },
},
},
})
await agentAutofillMedia(task)
expect(mockActivities.getAssetAutofillContextActivity).toHaveBeenCalledWith('a1')
expect(mockActivities.autofillAiActivity).toHaveBeenCalledWith(
expect.objectContaining({
teamId: 't1',
images: ['/tmp/test-dir/1.webp'],
fields: [
{
id: 'title',
config: { name: 'Title', type: 'text' },
description: 'The title',
},
],
context: { agent: { id: 'b1' } },
}),
)
const aiCallArgs = mockActivities.autofillAiActivity.mock.calls[0][0]
expect(aiCallArgs).not.toHaveProperty('agentContext')
})
it('should complete early if no image files could be extracted', async () => {
mockActivities.extractAiMetadataActivity.mockResolvedValue([])
+8 -13
View File
@@ -15,7 +15,6 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
extractAiMetadataActivity,
getProjectAutofillFieldsActivity,
getAgentAutofillContextActivity,
getAssetAutofillContextActivity,
autofillAiActivity,
updateTaskUsageActivity,
updateAssetMetadataActivity,
@@ -132,17 +131,12 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
teamId,
})
// 3c. Fetch Agent-Provided Autofill Context (from create_file/create_version)
const agentContext = await executeActivity(
agentWorkerQueue,
getAssetAutofillContextActivity,
asset.id,
)
// 4. Call AI Service
const aiResult = await executeActivity(agentWorkerQueue, autofillAiActivity, {
teamId,
images: generatedFiles,
assetId: asset.id,
projectId: asset.projectId ?? undefined,
fields: fields.map(
(f: { key: string; config: Record<string, unknown>; description?: string | null }) => ({
id: f.key,
@@ -151,7 +145,6 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
}),
),
context,
...(agentContext ? { agentContext } : {}),
})
if (aiResult.usage) {
@@ -166,10 +159,12 @@ export async function agentAutofillMedia(task: WorkflowTask): Promise<void> {
// 6. Update Asset Metadata
const result = JSON.parse(aiResult.text)
const metadataUpdates = Object.entries(result).map(([key, value]) => ({
key,
value,
}))
const metadataUpdates = Object.entries(result)
.filter(([, value]) => value !== null)
.map(([key, value]) => ({
key,
value,
}))
if (metadataUpdates.length > 0) {
await executeActivity(agentWorkerQueue, updateAssetMetadataActivity, {
@@ -1,171 +0,0 @@
import { describe, it, expect } from 'vitest'
import { AgentChatPromptBuilder } from './agent-chat-prompt-builder'
describe('AgentChatPromptBuilder', () => {
it('should build prompt with baseline configurations', () => {
const builder = new AgentChatPromptBuilder('a1')
const result = builder.build()
expect(result).toContain('The user is discussing an asset with ID: a1.')
})
it('should build prompt with path context', () => {
const builder = new AgentChatPromptBuilder('a1').withPathContext('folder/sub/file.png')
const result = builder.build()
expect(result).toContain('Asset Path Context:\nfolder/sub/file.png')
})
it('should build prompt with image asset details', () => {
const builder = new AgentChatPromptBuilder('a1').withAssetDetails(
'test.png',
'image/png',
undefined,
undefined,
'image',
)
const result = builder.build()
expect(result).toContain('File Name: test.png')
expect(result).toContain('File Type: image')
expect(result).toContain("call the 'analyze_image' tool")
expect(result).not.toContain("call the 'screenshot' tool")
})
it('should build prompt with video asset details and duration', () => {
const builder = new AgentChatPromptBuilder('a1').withAssetDetails(
'movie.mp4',
'video/mp4',
12.34,
undefined,
'video',
)
const result = builder.build()
expect(result).toContain('File Name: movie.mp4')
expect(result).toContain('File Type: video')
expect(result).toContain('Video Length: 12.34 seconds')
expect(result).toContain("call the 'screenshot' tool")
expect(result).not.toContain("call the 'analyze_image' tool")
})
it('should build prompt with PDF asset details and total pages', () => {
const builder = new AgentChatPromptBuilder('a1').withAssetDetails(
'doc.pdf',
'application/pdf',
undefined,
15,
'pdf',
)
const result = builder.build()
expect(result).toContain('File Name: doc.pdf')
expect(result).toContain('File Type: pdf')
expect(result).toContain('Total Pages: 15')
expect(result).toContain("call the 'read_pdf_pages' tool")
expect(result).toContain('Maximum 20 pages allowed per call.')
expect(result).not.toContain("call the 'screenshot' tool")
expect(result).not.toContain("call the 'analyze_image' tool")
})
it('should build prompt with comment page for PDF asset', () => {
const builder = new AgentChatPromptBuilder('a1')
.withAssetDetails('doc.pdf', 'application/pdf', undefined, 10, 'pdf')
.withCommentTimestamp(3)
const result = builder.build()
expect(result).toContain('Comment Page: 3')
})
it('should build prompt for PDF continuation mode with comment page', () => {
const builder = new AgentChatPromptBuilder('a1')
.withContinuation(true)
.withAssetDetails('doc.pdf', 'application/pdf', undefined, 10, 'pdf')
.withCommentTimestamp(4)
const result = builder.build()
expect(result).toContain('Comment Page: 4')
expect(result).not.toContain('Comment Timestamp:')
})
it('should build prompt with comment timestamp', () => {
const builder = new AgentChatPromptBuilder('a1').withCommentTimestamp(5.5)
const result = builder.build()
expect(result).toContain('Comment Timestamp: 5.50 seconds')
})
it('should build prompt with all fields combined', () => {
const builder = new AgentChatPromptBuilder('a1')
.withPathContext('src/main.ts')
.withAssetDetails('movie.mp4', 'video/mp4', 12.34, undefined, 'video')
.withCommentTimestamp(8.2)
const result = builder.build()
expect(result).toContain('The user is discussing an asset with ID: a1.')
expect(result).toContain('File Name: movie.mp4')
expect(result).toContain('File Type: video')
expect(result).toContain('Video Length: 12.34 seconds')
expect(result).toContain('Comment Timestamp: 8.20 seconds')
expect(result).toContain('Asset Path Context:\nsrc/main.ts')
expect(result).toContain("call the 'screenshot' tool")
})
it('should build prompt with attached files and referenced assets', () => {
const builder = new AgentChatPromptBuilder('a1')
.withAttachedFiles(['- file1.txt', '- file2.png'])
.withReferencedAssets(['- folder1', '- file3.mp4'])
const result = builder.build()
expect(result).toContain('[Context: Attached Files & Referenced Assets]')
expect(result).toContain('Attached Files:\n- file1.txt\n- file2.png')
expect(result).toContain('Referenced Workspace Assets:\n- folder1\n- file3.mp4')
})
it('should build prompt for continuation mode returning only turn-specific modifications', () => {
const builder = new AgentChatPromptBuilder('a1')
.withContinuation(true)
.withAssetDetails('movie.mp4', 'video/mp4', 12.34)
.withPathContext('src/main.ts')
.withCommentTimestamp(8.2)
.withAttachedFiles(['- file1.txt'])
.withReferencedAssets(['- folder1'])
const result = builder.build()
expect(result).not.toContain('The user is discussing an asset with ID:')
expect(result).not.toContain('File Name: movie.mp4')
expect(result).not.toContain('Asset Path Context:')
expect(result).toContain('Comment Timestamp: 8.20 seconds')
expect(result).toContain('[Context: New Attached Files & Referenced Assets]')
expect(result).toContain('Attached Files:\n- file1.txt')
expect(result).toContain('Referenced Workspace Assets:\n- folder1')
})
it('should include location change context block in continuation mode when asset has changed', () => {
const builder = new AgentChatPromptBuilder('a2')
.withContinuation(true)
.withAssetChanged(true)
.withAssetDetails('new-folder', 'folder', undefined, undefined, 'folder')
.withPathContext('projects/my-proj/new-folder')
const result = builder.build()
expect(result).toContain('[Context: User Navigated to a New Location]')
expect(result).toContain('New Location Asset ID: a2')
expect(result).toContain('File Name: new-folder')
expect(result).toContain('Asset Path Context:\nprojects/my-proj/new-folder')
})
it('should build prompt with user info when provided and not continuation', () => {
const builder = new AgentChatPromptBuilder('a1').withUserInfo('Alice', 'owner')
const result = builder.build()
expect(result).toContain('User Info:\nName: Alice\nRole: owner')
})
it('should build prompt with download_asset tool instruction', () => {
const builder = new AgentChatPromptBuilder('a1')
const result = builder.build()
expect(result).toContain("call the 'download_asset' tool with assetId.")
})
})
@@ -1,185 +0,0 @@
export function getSimpleMediaType(proxyType?: string | null): 'image' | 'video' | 'pdf' | 'other' {
if (proxyType === 'image') return 'image'
if (proxyType === 'video') return 'video'
if (proxyType === 'pdf') return 'pdf'
return 'other'
}
export class AgentChatPromptBuilder {
private assetId: string
private assetName?: string
private mediaType?: string
private proxyType?: string
private videoDuration?: number
private totalPages?: number
private commentTimestamp?: number
private pathContext?: string
private attachedFiles: string[] = []
private referencedAssets: string[] = []
private userInfo?: { name: string; role: string }
private isContinuation = false
private hasAssetChanged = false
constructor(assetId: string) {
this.assetId = assetId
}
withUserInfo(name?: string, role?: string): this {
if (name) {
this.userInfo = { name, role: role || 'user' }
}
return this
}
withContinuation(isContinuation: boolean): this {
this.isContinuation = isContinuation
return this
}
withAssetChanged(hasAssetChanged?: boolean): this {
this.hasAssetChanged = !!hasAssetChanged
return this
}
withAttachedFiles(files?: string[]): this {
if (files) {
this.attachedFiles = files
}
return this
}
withReferencedAssets(assets?: string[]): this {
if (assets) {
this.referencedAssets = assets
}
return this
}
withPathContext(pathContext?: string): this {
this.pathContext = pathContext
return this
}
withAssetDetails(
name: string,
mediaType: string | null,
duration?: number,
totalPages?: number,
proxyType?: string | null,
): this {
this.assetName = name
this.mediaType = mediaType || undefined
this.proxyType = proxyType || undefined
this.videoDuration = duration
this.totalPages = totalPages
return this
}
withCommentTimestamp(second?: number | null): this {
if (second !== undefined && second !== null) {
this.commentTimestamp = second
}
return this
}
build(): string {
const effectiveType = this.proxyType
if (this.isContinuation) {
let instruction = ''
if (this.hasAssetChanged) {
instruction += `[Context: User Navigated to a New Location]\nNew Location Asset ID: ${this.assetId}`
if (this.assetName) {
instruction += `\nFile Name: ${this.assetName}`
}
if (effectiveType) {
const type = getSimpleMediaType(effectiveType)
instruction += `\nFile Type: ${type}`
if (type === 'video' && this.videoDuration !== undefined) {
instruction += `\nVideo Length: ${this.videoDuration.toFixed(2)} seconds`
} else if (type === 'pdf' && this.totalPages !== undefined) {
instruction += `\nTotal Pages: ${this.totalPages}`
}
}
if (this.pathContext) {
instruction += `\n\nAsset Path Context:\n${this.pathContext}`
}
}
if (this.commentTimestamp !== undefined) {
const type = getSimpleMediaType(effectiveType)
if (instruction) instruction += '\n\n'
if (type === 'pdf') {
instruction += `Comment Page: ${Math.round(this.commentTimestamp)}`
} else {
instruction += `Comment Timestamp: ${this.commentTimestamp.toFixed(2)} seconds`
}
}
if (this.attachedFiles.length > 0 || this.referencedAssets.length > 0) {
if (instruction) instruction += '\n\n'
instruction += `[Context: New Attached Files & Referenced Assets]`
if (this.attachedFiles.length > 0) {
instruction += `\nAttached Files:\n${this.attachedFiles.join('\n')}`
}
if (this.referencedAssets.length > 0) {
instruction += `\nReferenced Workspace Assets:\n${this.referencedAssets.join('\n')}`
}
}
return instruction.trim()
}
let instruction = `The user is discussing an asset with ID: ${this.assetId}.`
if (this.assetName) {
instruction += `\nFile Name: ${this.assetName}`
}
if (effectiveType) {
const type = getSimpleMediaType(effectiveType)
instruction += `\nFile Type: ${type}`
if (type === 'video' && this.videoDuration !== undefined) {
instruction += `\nVideo Length: ${this.videoDuration.toFixed(2)} seconds`
} else if (type === 'pdf' && this.totalPages !== undefined) {
instruction += `\nTotal Pages: ${this.totalPages}`
}
}
if (this.commentTimestamp !== undefined) {
const type = getSimpleMediaType(effectiveType)
if (type === 'pdf') {
instruction += `\nComment Page: ${Math.round(this.commentTimestamp)}`
} else {
instruction += `\nComment Timestamp: ${this.commentTimestamp.toFixed(2)} seconds`
}
}
if (this.pathContext) {
instruction += `\n\nAsset Path Context:\n${this.pathContext}`
}
if (this.attachedFiles.length > 0 || this.referencedAssets.length > 0) {
instruction += `\n\n[Context: Attached Files & Referenced Assets]`
if (this.attachedFiles.length > 0) {
instruction += `\nAttached Files:\n${this.attachedFiles.join('\n')}`
}
if (this.referencedAssets.length > 0) {
instruction += `\nReferenced Workspace Assets:\n${this.referencedAssets.join('\n')}`
}
}
if (effectiveType) {
const type = getSimpleMediaType(effectiveType)
if (type === 'image') {
instruction += `\n\nIf you need to view the image data, call the 'analyze_image' tool with assetId "${this.assetId}".`
} else if (type === 'video') {
instruction += `\n\nIf you need to view visual frames or take screenshots of the video, call the 'screenshot' tool. You must specify the 'start' (seconds), 'end' (seconds), and 'count' (number of screenshots) parameters.`
} else if (type === 'pdf') {
instruction += `\n\nIf you need to view pages of the PDF document, call the 'read_pdf_pages' tool. You must specify the 'start' (page number) and 'end' (page number) parameters. Maximum 20 pages allowed per call.`
}
}
instruction += `\n\nIf you need a local file on disk to run bash commands or scripts (e.g. processing or combining assets), call the 'download_asset' tool with assetId.`
if (this.userInfo) {
instruction += `\n\nUser Info:\nName: ${this.userInfo.name}\nRole: ${this.userInfo.role}`
}
return instruction
}
}
+313 -58
View File
@@ -3,7 +3,6 @@ import { agentChat } from './agent-chat'
import { prisma } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import * as workflowUtils from '@shumai/workflow-core'
import { AgentChatPromptBuilder } from './agent-chat-prompt-builder'
vi.mock('@shumai/workflow-core', async () => {
const actual = await vi.importActual('@shumai/workflow-core')
@@ -43,6 +42,9 @@ describe('Agent Chat Workflow', () => {
updateCommentActivity: Object.assign(vi.fn(), { _activityName: 'updateCommentActivity' }),
updateTaskUsageActivity: Object.assign(vi.fn(), { _activityName: 'updateTaskUsageActivity' }),
deleteCommentActivity: Object.assign(vi.fn(), { _activityName: 'deleteCommentActivity' }),
getAssetPathHierarchyActivity: Object.assign(vi.fn(), {
_activityName: 'getAssetPathHierarchyActivity',
}),
getAssetPathContextActivity: Object.assign(vi.fn(), {
_activityName: 'getAssetPathContextActivity',
}),
@@ -70,13 +72,21 @@ describe('Agent Chat Workflow', () => {
})
mockActivities.getCommentActivity.mockResolvedValue({
id: 'c1',
creatorId: 'user-c1',
message: 'hello agent',
replyToId: null,
attachments: [],
})
mockActivities.initializeAgentSessionActivity.mockResolvedValue('session-123')
mockActivities.getAgentChatContextActivity.mockResolvedValue({ agent: { id: 'b1' } })
mockActivities.getAssetPathContextActivity.mockResolvedValue('Path: folder/subfolder/file.png')
mockActivities.getAssetPathHierarchyActivity.mockResolvedValue({
path: 'folder/subfolder/test-file.png',
ancestors: [
{ id: 'f1', name: 'folder' },
{ id: 'f2', name: 'subfolder' },
],
})
mockActivities.getAssetPathContextActivity.mockResolvedValue('folder/subfolder/test-file.png')
mockActivities.agentChatActivity.mockResolvedValue({
text: 'Hello user, how can I help you?',
sessionId: 'session-123',
@@ -118,6 +128,14 @@ describe('Agent Chat Workflow', () => {
// Verify comment context fetched
expect(mockActivities.getCommentActivity).toHaveBeenCalledWith('c1')
// Verify agent context fetched with the project context forwarded
expect(mockActivities.getAgentChatContextActivity).toHaveBeenCalledWith({
teamId: 't1',
agentId: 'b1',
userId: undefined,
projectId: 'p1',
})
// Verify placeholder comment created
expect(mockActivities.createCommentActivity).toHaveBeenCalledWith({
assetId: 'a1',
@@ -135,21 +153,31 @@ describe('Agent Chat Workflow', () => {
userId: undefined,
})
// Verify agent instruction composition
const expectedInstruction1 = new AgentChatPromptBuilder('a1')
.withPathContext('Path: folder/subfolder/file.png')
.withAssetDetails('test-file.png', 'image/png', undefined, undefined, 'image')
.withCommentTimestamp(null)
.build()
// Verify agent chat called with structured messageContext
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
teamId: 't1',
agentId: 'b1',
message: 'hello agent',
agentsInstruction: expectedInstruction1,
sessionId: 'session-123',
folderId: 'parent-folder-id',
assetId: 'a1',
messageContext: expect.objectContaining({
user: { id: 'user-c1', name: 'Test User', role: 'owner' },
currentAsset: expect.objectContaining({
id: 'a1',
name: 'test-file.png',
type: 'file',
mediaType: 'image',
mimeType: 'image/png',
parentId: 'parent-folder-id',
path: 'folder/subfolder/test-file.png',
ancestors: [
{ id: 'f1', name: 'folder' },
{ id: 'f2', name: 'subfolder' },
],
}),
}),
}),
)
@@ -201,17 +229,13 @@ describe('Agent Chat Workflow', () => {
await agentChat(task)
expect(mockActivities.initializeAgentSessionActivity).not.toHaveBeenCalled()
const expectedInstruction2 = new AgentChatPromptBuilder('a1')
.withContinuation(true)
.withPathContext('Path: folder/subfolder/file.png')
.withAssetDetails('test-file.png', 'image/png', undefined)
.withCommentTimestamp(null)
.build()
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
sessionId: 'existing-session-456',
agentsInstruction: expectedInstruction2,
messageContext: expect.objectContaining({
currentAsset: expect.objectContaining({ id: 'a1', name: 'test-file.png' }),
}),
}),
)
@@ -263,16 +287,9 @@ describe('Agent Chat Workflow', () => {
await agentChat(task)
const expectedInstruction4 = new AgentChatPromptBuilder('a1')
.withPathContext('Path: folder/subfolder/file.png')
.withAssetDetails('test-file.png', 'image/png', undefined, undefined, 'image')
.withCommentTimestamp(null)
.build()
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
imageUrls: ['attachments/image1.png', 'attachments/image2.jpg'],
agentsInstruction: expectedInstruction4,
}),
)
})
@@ -305,15 +322,13 @@ describe('Agent Chat Workflow', () => {
await agentChat(task)
const expectedInstruction5 = new AgentChatPromptBuilder('a1')
.withPathContext('Path: folder/subfolder/file.png')
.withAssetDetails('test-asset', 'video/mp4', 10)
.withCommentTimestamp(null)
.build()
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
agentsInstruction: expectedInstruction5,
messageContext: expect.objectContaining({
currentAsset: expect.objectContaining({
durationSeconds: 10,
}),
}),
}),
)
})
@@ -381,6 +396,7 @@ describe('Agent Chat Workflow', () => {
name: 'attachment.png',
type: 'file',
mediaType: 'image/png',
media: { proxyType: 'image' },
projectId: 'p1',
}
}
@@ -390,6 +406,7 @@ describe('Agent Chat Workflow', () => {
name: 'ref-folder',
type: 'folder',
mediaType: null,
media: null,
projectId: 'p1',
}
}
@@ -398,6 +415,7 @@ describe('Agent Chat Workflow', () => {
name: 'test-file.png',
type: 'file',
mediaType: 'image/png',
media: { proxyType: 'image' },
projectId: 'p1',
project: { teamId: 't1' },
}
@@ -433,20 +451,7 @@ describe('Agent Chat Workflow', () => {
expect(mockActivities.createCommentActivity).not.toHaveBeenCalled()
expect(mockActivities.updateCommentActivity).not.toHaveBeenCalled()
// Verify prompt builder is populated and workflow runs agentChatActivity with context instruction
const expectedInstruction = new AgentChatPromptBuilder('a1')
.withContinuation(true)
.withPathContext('folder/subfolder/file.png')
.withAssetDetails('test-file.png', 'image/png', undefined)
.withCommentTimestamp(undefined)
.withAttachedFiles([
'- Name: attachment.png (ID: file-attachment-1, Type: file, Media Type: image/png, Project ID: p1, Path: attachment.png)',
])
.withReferencedAssets([
'- Name: ref-folder (ID: referenced-asset-1, Type: folder, Media Type: unknown, Project ID: p1, Path: ref-folder)',
])
.build()
// Verify messageContext is populated with attachedFiles and referencedAssets
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
teamId: 't1',
@@ -455,15 +460,32 @@ describe('Agent Chat Workflow', () => {
imageUrls: [],
projectId: 'p1',
folderId: '',
agentsInstruction: expectedInstruction,
sessionId: 'session-direct-123',
userId: undefined,
userCommentId: undefined,
context: { agent: { id: 'b1' } },
attachedAssets: [
{ id: 'file-attachment-1', name: 'attachment.png', type: 'file' },
{ id: 'referenced-asset-1', name: 'ref-folder', type: 'folder' },
],
messageContext: expect.objectContaining({
attachedFiles: [
{
id: 'file-attachment-1',
name: 'attachment.png',
type: 'file',
mediaType: 'image',
mimeType: 'image/png',
path: 'attachment.png',
},
],
referencedAssets: [
{
id: 'referenced-asset-1',
name: 'ref-folder',
type: 'folder',
mediaType: undefined,
mimeType: undefined,
path: 'ref-folder',
},
],
}),
}),
)
})
@@ -502,19 +524,18 @@ describe('Agent Chat Workflow', () => {
await agentChat(task)
const expectedInstruction = new AgentChatPromptBuilder('v2-latest')
.withContinuation(true)
.withPathContext('Path: folder/subfolder/file.png')
.withAssetDetails('latest-video.mp4', 'video/mp4', 42, undefined, 'video')
.build()
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
teamId: 't1',
agentId: 'b1',
message: 'Explain this version stack video',
agentsInstruction: expectedInstruction,
sessionId: 'session-vs-123',
messageContext: expect.objectContaining({
currentAsset: expect.objectContaining({
id: 'v2-latest',
durationSeconds: 42,
}),
}),
}),
)
})
@@ -547,7 +568,241 @@ describe('Agent Chat Workflow', () => {
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
agentsInstruction: expect.stringContaining('User Info:\nName: Test User\nRole: owner'),
messageContext: expect.objectContaining({
user: { id: 'user-alice', name: 'Test User', role: 'owner' },
}),
}),
)
})
it('should populate comment attachments into structured messageContext.attachedFiles', async () => {
mockActivities.getCommentActivity.mockResolvedValue({
id: 'c1',
creatorId: 'user-c1',
message: 'check these attachments',
replyToId: null,
attachments: [
{
asset: {
id: 'att-doc-1',
name: 'brief.pdf',
type: 'file',
mediaType: 'application/pdf',
media: { proxyType: 'pdf' },
storageKey: { key: 'brief.pdf' },
},
},
{
asset: {
id: 'att-img-1',
name: 'mockup.png',
type: 'file',
mediaType: 'image/png',
media: { proxyType: 'image' },
storageKey: { key: 'mockup.png' },
},
},
],
})
mockActivities.getAssetPathContextActivity.mockImplementation(async (id: string) => {
if (id === 'att-doc-1') return 'Docs/brief.pdf'
if (id === 'att-img-1') return 'Designs/mockup.png'
return 'folder/subfolder/file.png'
})
const task = await prisma.workflowTask.create({
data: {
type: 'chat',
status: 'pending',
assetId: 'a1',
payload: {
projectId: 'p1',
agent: { userCommentId: 'c1', agentId: 'b1' },
},
},
})
await agentChat(task)
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
imageUrls: ['mockup.png'],
messageContext: expect.objectContaining({
attachedFiles: [
{
id: 'att-doc-1',
name: 'brief.pdf',
type: 'file',
mediaType: 'pdf',
mimeType: 'application/pdf',
path: 'Docs/brief.pdf',
},
{
id: 'att-img-1',
name: 'mockup.png',
type: 'file',
mediaType: 'image',
mimeType: 'image/png',
path: 'Designs/mockup.png',
},
],
}),
}),
)
})
it('should set totalFrames for video assets and totalPages for pdf assets in currentAsset context', async () => {
mockActivities.getAssetActivity.mockImplementation(async (id: string) => {
if (id === 'video-asset-1') {
return {
id: 'video-asset-1',
project: { teamId: 't1' },
type: 'file',
name: 'clip.mp4',
mediaType: 'video/mp4',
media: { proxyType: 'video', duration: 9, frames: 268 },
parentId: 'f1',
}
}
if (id === 'pdf-asset-1') {
return {
id: 'pdf-asset-1',
project: { teamId: 't1' },
type: 'file',
name: 'doc.pdf',
mediaType: 'application/pdf',
media: { proxyType: 'pdf', frames: 15, duration: 0 },
parentId: 'f1',
}
}
return null
})
// Test Video asset
const videoTask = await prisma.workflowTask.create({
data: {
type: 'chat',
status: 'pending',
assetId: 'video-asset-1',
payload: {
projectId: 'p1',
agent: { prompt: 'Analyze video', agentId: 'b1', sessionId: 'session-video-1' },
},
},
})
await agentChat(videoTask)
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
messageContext: expect.objectContaining({
currentAsset: expect.objectContaining({
id: 'video-asset-1',
mediaType: 'video',
durationSeconds: 9,
totalFrames: 268,
totalPages: undefined,
}),
}),
}),
)
// Test PDF asset
const pdfTask = await prisma.workflowTask.create({
data: {
type: 'chat',
status: 'pending',
assetId: 'pdf-asset-1',
payload: {
projectId: 'p1',
agent: { prompt: 'Analyze pdf', agentId: 'b1', sessionId: 'session-pdf-1' },
},
},
})
await agentChat(pdfTask)
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
messageContext: expect.objectContaining({
currentAsset: expect.objectContaining({
id: 'pdf-asset-1',
mediaType: 'pdf',
totalPages: 15,
totalFrames: undefined,
durationSeconds: undefined,
}),
}),
}),
)
})
it('should include annotations and second in direct chat messageContext', async () => {
const task = await prisma.workflowTask.create({
data: {
type: 'chat',
status: 'pending',
assetId: 'a1',
payload: {
projectId: 'p1',
agent: {
prompt: 'Look at the highlighted zone',
agentId: 'b1',
sessionId: 'session-markup-1',
second: 14.2,
annotations: [{ type: 'rectangle', x: 20, y: 30, width: 40, height: 50 }],
},
},
},
})
await agentChat(task)
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
messageContext: expect.objectContaining({
position: { type: 'time', seconds: 14.2 },
annotations: [{ type: 'rectangle', x: 20, y: 30, width: 40, height: 50 }],
}),
}),
)
})
it('should include userCommentId as id in comment messageContext', async () => {
mockActivities.getCommentActivity.mockResolvedValue({
id: 'comment-with-id-123',
creatorId: 'user-c1',
message: 'Check this out',
replyToId: null,
second: 5.0,
annotation: [{ type: 'arrow' }],
attachments: [],
})
const task = await prisma.workflowTask.create({
data: {
type: 'chat',
status: 'pending',
assetId: 'a1',
payload: {
projectId: 'p1',
agent: {
agentId: 'b1',
userCommentId: 'comment-with-id-123',
},
},
},
})
await agentChat(task)
expect(mockActivities.agentChatActivity).toHaveBeenCalledWith(
expect.objectContaining({
messageContext: expect.objectContaining({
id: 'comment-with-id-123',
position: { type: 'time', seconds: 5.0 },
annotation: true,
annotations: [{ type: 'arrow' }],
}),
}),
)
})
+161 -32
View File
@@ -1,7 +1,13 @@
import { ApplicationFailure } from '@temporalio/workflow'
import type { WorkflowTask } from '@shumai/db'
import { executeActivity, getActivities, TaskQueueAgent } from '@shumai/workflow-core'
import { AgentChatPromptBuilder } from './agent-chat-prompt-builder'
import type {
ShumaiMessageContext,
ShumaiAssetContext,
ShumaiAttachedFileContext,
ShumaiMediaPosition,
} from '@shumai/dtos'
import { getProxyType } from '@shumai/core/src/utils/mime'
export async function agentChat(task: WorkflowTask): Promise<void> {
const {
@@ -15,6 +21,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
updateTaskUsageActivity,
getAgentWorkerQueueActivity,
initializeAgentSessionActivity,
getAssetPathHierarchyActivity,
getAssetPathContextActivity,
generateSessionNameActivity,
getUserTeamInfoActivity,
@@ -47,19 +54,33 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
// 1. Get User Comment
const userCommentId = payload.agent?.userCommentId
let userComment: Awaited<ReturnType<typeof getCommentActivity>> | null = null
let prompt = ''
let attachmentImageUrls: string[] = []
let commentTimestamp: number | undefined
let commentAnnotation = false
let commentCreatorId: string | undefined
let commentAnnotations: unknown[] | undefined
if (userCommentId) {
const userComment = await executeActivity(agentWorkerQueue, getCommentActivity, userCommentId)
userComment = await executeActivity(agentWorkerQueue, getCommentActivity, userCommentId)
if (!userComment) {
throw ApplicationFailure.create({ message: 'User comment not found', nonRetryable: true })
}
prompt = userComment.message || ''
commentCreatorId = userComment.creatorId || undefined
if (userComment.second !== null && userComment.second !== undefined) {
commentTimestamp = userComment.second
}
if (
userComment.annotation &&
Array.isArray(userComment.annotation) &&
userComment.annotation.length > 0
) {
commentAnnotation = true
commentAnnotations = userComment.annotation
}
// 2. Create Placeholder Comment
const placeholder = await executeActivity(agentWorkerQueue, createCommentActivity, {
@@ -71,7 +92,7 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
})
placeholderCommentId = placeholder.id
// 5. Prepare Images (Attachments only)
// Prepare Images (Attachments only)
if (userComment.attachments) {
for (const att of userComment.attachments) {
const attProxyType = (att.asset?.media as PrismaJson.MediaInfo | null)?.proxyType
@@ -83,6 +104,17 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
} else {
prompt = payload.agent?.prompt || ''
attachmentImageUrls = payload.agent?.imageUrls || []
if (payload.agent?.second !== null && payload.agent?.second !== undefined) {
commentTimestamp = payload.agent.second
}
if (
payload.agent?.annotations &&
Array.isArray(payload.agent.annotations) &&
payload.agent.annotations.length > 0
) {
commentAnnotation = true
commentAnnotations = payload.agent.annotations
}
}
// 3. Get Asset
@@ -114,45 +146,111 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
const context = await executeActivity(agentWorkerQueue, getAgentChatContextActivity, {
teamId,
agentId: agentId,
userId: payload.agent?.userId,
projectId: payload.projectId,
})
const pathContext = await executeActivity(
const pathHierarchy = await executeActivity(
agentWorkerQueue,
getAssetPathContextActivity,
getAssetPathHierarchyActivity,
task.assetId,
)
const assetPath = pathHierarchy?.path || ''
const ancestors = pathHierarchy?.ancestors || []
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const mediaInfo = asset.media as any
const duration = mediaInfo?.duration
const totalPages = mediaInfo?.frames || mediaInfo?.metadata?.totalFrames
const proxyType =
(asset.media as PrismaJson.MediaInfo | null)?.proxyType ||
getProxyType(asset.mediaType, asset.name) ||
undefined
const duration =
(proxyType === 'video' || proxyType === 'audio') && typeof mediaInfo?.duration === 'number'
? mediaInfo.duration
: undefined
const totalPages =
proxyType === 'pdf' ? mediaInfo?.frames || mediaInfo?.metadata?.totalFrames : undefined
const totalFrames =
proxyType === 'video' ? mediaInfo?.frames || mediaInfo?.metadata?.totalFrames : undefined
// Resolve Attached Files Details
const attachedFileDetailsList: string[] = []
const attachedFiles: ShumaiAttachedFileContext[] = []
const attachedAssets: Array<{ id: string; name: string; type: string }> = []
if (payload.agent?.attachedFiles) {
if (userCommentId && userComment?.attachments) {
for (const att of userComment.attachments) {
if (att.asset) {
const filePath = await executeActivity(
agentWorkerQueue,
getAssetPathContextActivity,
att.asset.id,
)
const attProxyType =
(att.asset.media as PrismaJson.MediaInfo | null)?.proxyType ||
getProxyType(att.asset.mediaType, att.asset.name) ||
undefined
attachedFiles.push({
id: att.asset.id,
name: att.asset.name,
type: att.asset.type,
mediaType: attProxyType as ShumaiAttachedFileContext['mediaType'],
mimeType: att.asset.mediaType || undefined,
path: filePath || undefined,
url: (att.asset as { url?: string }).url || undefined,
})
attachedAssets.push({ id: att.asset.id, name: att.asset.name, type: att.asset.type })
}
}
} else if (payload.agent?.attachedFiles) {
for (const fileId of payload.agent.attachedFiles) {
const file = await executeActivity(agentWorkerQueue, getAssetActivity, fileId)
if (file) {
const path = await executeActivity(agentWorkerQueue, getAssetPathContextActivity, fileId)
attachedFileDetailsList.push(
`- Name: ${file.name} (ID: ${file.id}, Type: ${file.type}, Media Type: ${file.mediaType || 'unknown'}, Project ID: ${file.projectId || 'unknown'}, Path: ${path})`,
const filePath = await executeActivity(
agentWorkerQueue,
getAssetPathContextActivity,
fileId,
)
const fileProxyType =
(file.media as PrismaJson.MediaInfo | null)?.proxyType ||
getProxyType(file.mediaType, file.name) ||
undefined
attachedFiles.push({
id: file.id,
name: file.name,
type: file.type,
mediaType: fileProxyType as ShumaiAttachedFileContext['mediaType'],
mimeType: file.mediaType || undefined,
path: filePath || undefined,
url: (file as { url?: string }).url || undefined,
})
attachedAssets.push({ id: file.id, name: file.name, type: file.type })
}
}
}
// Resolve Referenced Workspace Assets Details
const referencedAssetDetailsList: string[] = []
const referencedAssets: ShumaiAttachedFileContext[] = []
if (payload.agent?.assetIds) {
for (const assetId of payload.agent.assetIds) {
const referencedAsset = await executeActivity(agentWorkerQueue, getAssetActivity, assetId)
if (referencedAsset) {
const path = await executeActivity(agentWorkerQueue, getAssetPathContextActivity, assetId)
referencedAssetDetailsList.push(
`- Name: ${referencedAsset.name} (ID: ${referencedAsset.id}, Type: ${referencedAsset.type}, Media Type: ${referencedAsset.mediaType || 'unknown'}, Project ID: ${referencedAsset.projectId || 'unknown'}, Path: ${path})`,
const filePath = await executeActivity(
agentWorkerQueue,
getAssetPathContextActivity,
assetId,
)
const refProxyType =
(referencedAsset.media as PrismaJson.MediaInfo | null)?.proxyType ||
getProxyType(referencedAsset.mediaType, referencedAsset.name) ||
undefined
referencedAssets.push({
id: referencedAsset.id,
name: referencedAsset.name,
type: referencedAsset.type,
mediaType: refProxyType as ShumaiAttachedFileContext['mediaType'],
mimeType: referencedAsset.mediaType || undefined,
path: filePath || undefined,
url: (referencedAsset as { url?: string }).url || undefined,
})
attachedAssets.push({
id: referencedAsset.id,
name: referencedAsset.name,
@@ -162,29 +260,59 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
}
}
// Resolve User Info
const targetUserId = userCommentId ? commentCreatorId : payload.agent?.userId
let userInfo: { name: string; role: string } | null = null
if (!userCommentId && isNewChat && payload.agent?.userId) {
if (targetUserId) {
userInfo = await executeActivity(agentWorkerQueue, getUserTeamInfoActivity, {
userId: payload.agent.userId,
userId: targetUserId,
teamId,
})
}
const proxyType = (asset.media as PrismaJson.MediaInfo | null)?.proxyType
const promptBuilder = new AgentChatPromptBuilder(asset.id)
.withContinuation(!isNewChat)
.withAssetChanged(payload.agent?.hasAssetChanged)
.withPathContext(pathContext)
.withAssetDetails(asset.name, asset.mediaType, duration, totalPages, proxyType)
.withCommentTimestamp(commentTimestamp)
.withAttachedFiles(attachedFileDetailsList)
.withReferencedAssets(referencedAssetDetailsList)
if (userInfo) {
promptBuilder.withUserInfo(userInfo.name, userInfo.role)
// Build position
let mediaPosition: ShumaiMediaPosition | undefined
if (commentTimestamp !== undefined && commentTimestamp !== null) {
if (proxyType === 'pdf') {
mediaPosition = { type: 'page', page: Math.round(commentTimestamp) }
} else {
mediaPosition = { type: 'time', seconds: commentTimestamp }
}
}
const instruction = promptBuilder.build()
// Build Current Asset Context
const currentAssetContext: ShumaiAssetContext = {
id: asset.id,
name: asset.name,
type: asset.type,
mediaType: proxyType as ShumaiAssetContext['mediaType'],
mimeType: asset.mediaType || undefined,
parentId: asset.parentId || undefined,
path: assetPath || undefined,
durationSeconds: duration !== undefined ? duration : undefined,
totalFrames: totalFrames !== undefined ? totalFrames : undefined,
totalPages: totalPages !== undefined ? totalPages : undefined,
navigated: payload.agent?.hasAssetChanged === true ? true : undefined,
ancestors: ancestors.length > 0 ? ancestors : undefined,
}
// Build structured ShumaiMessageContext
const messageContext: ShumaiMessageContext = {
...(userCommentId ? { id: userCommentId } : {}),
...(userInfo
? { user: { id: targetUserId!, name: userInfo.name, role: userInfo.role } }
: {}),
currentAsset: currentAssetContext,
...(mediaPosition ? { position: mediaPosition } : {}),
...(commentAnnotation
? {
annotation: true,
annotations: commentAnnotations as ShumaiMessageContext['annotations'],
}
: {}),
...(attachedFiles.length > 0 ? { attachedFiles } : {}),
...(referencedAssets.length > 0 ? { referencedAssets } : {}),
}
// 6. Call AI Chat
let folderId = ''
@@ -202,11 +330,12 @@ export async function agentChat(task: WorkflowTask): Promise<void> {
imageUrls: attachmentImageUrls,
projectId: payload.projectId,
folderId,
agentsInstruction: instruction,
assetId: asset.id,
sessionId,
userId: payload.agent?.userId,
userCommentId: userCommentId || undefined,
context,
messageContext,
attachedAssets,
})
@@ -1,140 +0,0 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { agentToolCall } from './agent-tool-call'
import { prisma } from '@shumai/db'
import { setupTestDbHooks } from '@shumai/db/test'
import * as workflowUtils from '@shumai/workflow-core'
vi.mock('@shumai/workflow-core', async () => {
const actual = await vi.importActual('@shumai/workflow-core')
return {
...actual,
getActivities: vi.fn(),
executeActivity: vi.fn(),
}
})
describe('Agent Tool Call Workflow', () => {
setupTestDbHooks()
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mockActivities holds mock functions cast to expected types
let mockActivities: any
beforeEach(() => {
vi.clearAllMocks()
mockActivities = {
updateTaskStatusActivity: Object.assign(vi.fn(), {
_activityName: 'updateTaskStatusActivity',
}),
executeAgentToolActivity: Object.assign(vi.fn(), {
_activityName: 'executeAgentToolActivity',
}),
getAgentWorkerQueueActivity: Object.assign(vi.fn(), {
_activityName: 'getAgentWorkerQueueActivity',
}),
}
mockActivities.getAgentWorkerQueueActivity.mockResolvedValue('agent_queue')
mockActivities.updateTaskStatusActivity.mockResolvedValue({})
mockActivities.executeAgentToolActivity.mockResolvedValue({ success: true, count: 42 })
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mockActivities contains vi.fn mock functions which are cast to expected activity proxy types
vi.mocked(workflowUtils.getActivities).mockReturnValue(mockActivities as any)
vi.mocked(workflowUtils.executeActivity).mockImplementation(async (_queue, act, ...args) => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- act is one of the mocked activities
return (act as any)(...args)
})
})
it('should run agent tool call workflow successfully', async () => {
const task = await prisma.workflowTask.create({
data: {
type: 'agent_tool_call',
status: 'pending',
assetId: 'a1',
payload: {
projectId: 'p1',
agentToolCall: {
toolName: 'list_assets',
args: { parent: 'folder-123' },
userId: 'user-456',
},
},
},
})
await agentToolCall(task)
// Verify queue discovery
expect(mockActivities.getAgentWorkerQueueActivity).toHaveBeenCalled()
// Verify task processing status
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: task.id,
status: 'processing',
})
// Verify tool execution
expect(mockActivities.executeAgentToolActivity).toHaveBeenCalledWith({
taskId: task.id,
toolName: 'list_assets',
args: { parent: 'folder-123' },
userId: 'user-456',
})
// Verify task completed status with output
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: task.id,
status: 'completed',
output: { success: true, count: 42 },
})
})
it('should throw if payload or agentToolCall is missing', async () => {
const task = await prisma.workflowTask.create({
data: {
type: 'agent_tool_call',
status: 'pending',
assetId: 'a1',
},
})
await expect(agentToolCall(task)).rejects.toThrow('Task payload or agentToolCall is missing')
// Verify status updated to failed
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: task.id,
status: 'failed',
output: { error: 'Task payload or agentToolCall is missing' },
})
})
it('should handle tool execution failures by updating task to failed and throwing', async () => {
mockActivities.executeAgentToolActivity.mockRejectedValue(new Error('Authz failure'))
const task = await prisma.workflowTask.create({
data: {
type: 'agent_tool_call',
status: 'pending',
assetId: 'a1',
payload: {
projectId: 'p1',
agentToolCall: {
toolName: 'create_folder',
args: { parent: 'folder-123', name: 'new-folder' },
userId: 'user-456',
},
},
},
})
await expect(agentToolCall(task)).rejects.toThrow('Authz failure')
// Verify task status failed
expect(mockActivities.updateTaskStatusActivity).toHaveBeenCalledWith({
taskId: task.id,
status: 'failed',
output: { error: 'Authz failure' },
})
})
})
@@ -1,60 +0,0 @@
import { ApplicationFailure } from '@temporalio/workflow'
import type { WorkflowTask } from '@shumai/db'
import { getActivities, executeActivity, TaskQueueAgent } from '@shumai/workflow-core'
export async function agentToolCall(task: WorkflowTask): Promise<void> {
const { updateTaskStatusActivity, executeAgentToolActivity, getAgentWorkerQueueActivity } =
getActivities()
let agentWorkerQueue = ''
try {
// 0. Discover queue
agentWorkerQueue = await executeActivity(TaskQueueAgent, getAgentWorkerQueueActivity)
// Update status to processing
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
taskId: task.id,
status: 'processing',
})
const payload = task.payload
if (!payload || !payload.agentToolCall) {
throw ApplicationFailure.create({
message: 'Task payload or agentToolCall is missing',
nonRetryable: true,
})
}
const { toolName, args, userId } = payload.agentToolCall
// Execute the agent activity on specific queue
const result = await executeActivity(agentWorkerQueue, executeAgentToolActivity, {
taskId: task.id,
toolName,
args,
userId,
})
// Update status to completed with output
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
taskId: task.id,
status: 'completed',
output: result,
})
} catch (err) {
console.error(`AgentToolCall failed for task ${task.id}:`, err)
// Update status to failed
if (agentWorkerQueue) {
await executeActivity(agentWorkerQueue, updateTaskStatusActivity, {
taskId: task.id,
status: 'failed',
output: { error: err instanceof Error ? err.message : String(err) },
})
}
throw err
}
}
export const agentToolCallWorkflow = agentToolCall
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/api",
"version": "0.3.6",
"version": "0.4.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
+152
View File
@@ -1,7 +1,9 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { HTTPException } from 'hono/http-exception'
import { agentService } from '@shumai/core/src/agent/agent'
import { authzService } from '@shumai/core/src/authz/authz'
import { auditLogService } from '@shumai/core/src/auditLog/auditLog'
import type { AgentInfo } from '@shumai/dtos'
import app from './agent'
vi.mock('@shumai/core/src/agent/agent')
@@ -12,6 +14,24 @@ vi.mock('@shumai/core/src/auditLog/auditLog', () => ({
},
}))
/**
* Shape of the agent rows returned by the (mocked) service methods. Mirrors
* the real AgentInfoSource consumed by AgentService.toAgentInfo.
*/
interface MockAgentRow {
id: string
user: { name: string; image?: string | null }
type: string
enabled?: boolean
permission?: string
providerId?: string | null
modelId?: string | null
soul?: string | null
config?: { thinkingLevel?: string; systemPrompt?: string; deniedTools?: string[] }
skills?: Array<{ id: string; skillId: string; skill?: { id: string; name: string } }>
mcpServers?: Array<{ mcpServerId: string }>
}
describe('Agent API', () => {
beforeEach(() => {
vi.clearAllMocks()
@@ -20,6 +40,32 @@ describe('Agent API', () => {
id: 'agent1',
teamId: 'team1',
} as unknown as Awaited<ReturnType<typeof agentService.getAgent>>)
// The route delegates AgentInfo mapping to the service; replicate the real
// mapping so response assertions keep working.
vi.mocked(agentService.toAgentInfo).mockImplementation(async (agent) => {
const config = agent.config as MockAgentRow['config']
return {
id: agent.id,
name: agent.user.name,
type: agent.type,
enabled: agent.enabled ?? true,
permission: agent.permission ?? 'reviewer',
avatar: agent.user.image || undefined,
providerId: agent.providerId || undefined,
modelId: agent.modelId || undefined,
thinkingLevel: config?.thinkingLevel || 'off',
systemPrompt: config?.systemPrompt,
soul: agent.soul || undefined,
skills: (agent.skills || []).map((s) => ({
id: s.id,
skillId: s.skillId,
skill: s.skill,
})),
mcpServerIds: (agent.mcpServers || []).map((m) => m.mcpServerId),
deniedTools: config?.deniedTools || [],
} as unknown as AgentInfo
})
})
describe('GET /teams/:teamId/agents', () => {
@@ -183,6 +229,63 @@ describe('Agent API', () => {
})
})
describe('PATCH /agents/:agentId/permission', () => {
it('updates agent permission when user is admin', async () => {
const mockAgent = {
id: 'agent1',
teamId: 'team1',
user: { name: 'Bot 1', image: 'avatar1' },
type: 'chat',
enabled: true,
permission: 'owner',
config: {
provider: 'prov1',
model: 'model1',
},
skills: [],
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
vi.mocked(agentService.updateAgentPermission).mockResolvedValue(mockAgent as any)
const res = await app.request('/agents/agent1/permission', {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ permission: 'owner' }),
})
expect(res.status).toBe(200)
const data = await res.json()
expect(data.permission).toBe('owner')
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: undefined,
permission: 'Admin',
type: 'agent',
id: 'agent1',
})
expect(agentService.updateAgentPermission).toHaveBeenCalledWith('agent1', 'owner')
expect(auditLogService.logAction).toHaveBeenCalledWith({
action: 'agent_update',
teamId: 'team1',
userId: undefined,
itemId: 'agent1',
})
})
it('denies access when user is not admin', async () => {
vi.mocked(authzService.hasPermission).mockRejectedValue(
new HTTPException(403, { message: 'Forbidden' }),
)
const res = await app.request('/agents/agent1/permission', {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ permission: 'owner' }),
})
expect(res.status).toBe(403)
})
})
describe('GET /agent-sessions/:sessionId/entries', () => {
it('returns entries of agent session if user is admin', async () => {
const mockEntries = [
@@ -258,4 +361,53 @@ describe('Agent API', () => {
})
})
})
describe('GET /projects/:projectId/chat-agents', () => {
it('returns project-scoped chat agents for the user', async () => {
const mockAgents = [
{
id: 'agent1',
name: 'Chat Bot',
type: 'chat',
enabled: true,
permission: 'reviewer',
skills: [],
mcpServerIds: [],
deniedTools: [],
},
]
vi.mocked(agentService.listProjectChatAgents).mockResolvedValue(
mockAgents as unknown as Awaited<ReturnType<typeof agentService.listProjectChatAgents>>,
)
const res = await app.request('/projects/project1/chat-agents', {
headers: { 'Content-Type': 'application/json' },
})
expect(res.status).toBe(200)
const data = await res.json()
expect(data).toHaveLength(1)
expect(data[0].name).toBe('Chat Bot')
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: undefined,
permission: 'Read',
type: 'project',
id: 'project1',
})
expect(agentService.listProjectChatAgents).toHaveBeenCalledWith('project1', undefined)
})
it('denies access when the user has no project access', async () => {
vi.mocked(authzService.hasPermission).mockRejectedValue(
new HTTPException(403, { message: 'Forbidden' }),
)
const res = await app.request('/projects/project1/chat-agents', {
headers: { 'Content-Type': 'application/json' },
})
expect(res.status).toBe(403)
expect(agentService.listProjectChatAgents).not.toHaveBeenCalled()
})
})
})
+48 -65
View File
@@ -2,13 +2,12 @@ import { Hono } from 'hono'
import { zValidator } from '@hono/zod-validator'
import { agentService } from '@shumai/core/src/agent/agent'
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
import { getAvatarUrl } from '@shumai/core/src/user/avatar'
import {
createAgentRequestSchema,
updateAgentRequestSchema,
updateAgentPermissionRequestSchema,
paginationParamsSchema,
AgentInfo,
AgentType,
AuditAction,
} from '@shumai/dtos'
@@ -29,35 +28,28 @@ const route = new Hono<{ Variables: { user: User } }>()
id: teamId,
})
const agents = await agentService.listAgents({ teamId })
const agents = await agentService.listAgents({ teamId, userId: userReq?.id })
const res: AgentInfo[] = await Promise.all(
agents.map(async (agent) => {
const config = agent.config as unknown as PrismaJson.AgentConfig
return {
id: agent.id,
name: agent.user.name,
type: agent.type as AgentType,
enabled: agent.enabled,
avatar: (await getAvatarUrl(agent.user.image)) || undefined,
providerId: agent.providerId || undefined,
modelId: agent.modelId || undefined,
thinkingLevel: config.thinkingLevel || 'off',
systemPrompt: config.systemPrompt,
soul: agent.soul || undefined,
skills: agent.skills.map((s) => ({
id: s.id,
skillId: s.skillId,
skill: s.skill,
})),
mcpServerIds: (agent.mcpServers || []).map((m) => m.mcpServerId),
deniedTools: config.deniedTools || [],
}
}),
agents.map((agent) => agentService.toAgentInfo(agent)),
)
return c.json(res, 200)
})
.get('/projects/:projectId/chat-agents', async (c) => {
const projectId = c.req.param('projectId')
const userReq = c.get('user')
await authzService.hasPermission({
user: userReq,
permission: Permission.Read,
type: ResourceType.Project,
id: projectId,
})
const agents = await agentService.listProjectChatAgents(projectId, userReq?.id)
return c.json(agents, 200)
})
.post('/teams/:teamId/agents', zValidator('json', createAgentRequestSchema), async (c) => {
const teamId = c.req.param('teamId')
const userReq = c.get('user')
@@ -77,26 +69,7 @@ const route = new Hono<{ Variables: { user: User } }>()
if (!agent) throw new Error('failed to create agent')
const config = agent.config as unknown as PrismaJson.AgentConfig
const info: AgentInfo = {
id: agent.id,
name: agent.user.name,
type: agent.type as AgentType,
enabled: agent.enabled,
avatar: (await getAvatarUrl(agent.user.image)) || undefined,
providerId: agent.providerId || undefined,
modelId: agent.modelId || undefined,
thinkingLevel: config.thinkingLevel || 'off',
systemPrompt: config.systemPrompt,
soul: agent.soul || undefined,
skills: agent.skills.map((s) => ({
id: s.id,
skillId: s.skillId,
skill: s.skill,
})),
mcpServerIds: (agent.mcpServers || []).map((m) => m.mcpServerId),
deniedTools: config.deniedTools || [],
}
const info = await agentService.toAgentInfo(agent)
await auditLogService.logAction({
action: AuditAction.agent_create,
@@ -131,29 +104,39 @@ const route = new Hono<{ Variables: { user: User } }>()
itemId: agent.id,
})
const config = agent.config as unknown as PrismaJson.AgentConfig
const info: AgentInfo = {
id: agent.id,
name: agent.user.name,
type: agent.type as AgentType,
enabled: agent.enabled,
avatar: (await getAvatarUrl(agent.user.image)) || undefined,
providerId: agent.providerId || undefined,
modelId: agent.modelId || undefined,
thinkingLevel: config.thinkingLevel || 'off',
systemPrompt: config.systemPrompt,
soul: agent.soul || undefined,
skills: agent.skills.map((s) => ({
id: s.id,
skillId: s.skillId,
skill: s.skill,
})),
mcpServerIds: (agent.mcpServers || []).map((m) => m.mcpServerId),
deniedTools: config.deniedTools || [],
}
const info = await agentService.toAgentInfo(agent)
return c.json(info, 200)
})
.patch(
'/agents/:agentId/permission',
zValidator('json', updateAgentPermissionRequestSchema),
async (c) => {
const agentId = c.req.param('agentId')
const userReq = c.get('user')
const req = c.req.valid('json')
await authzService.hasPermission({
user: userReq,
permission: Permission.Admin,
type: ResourceType.Agent,
id: agentId,
})
const agent = await agentService.updateAgentPermission(agentId, req.permission)
await auditLogService.logAction({
action: AuditAction.agent_update,
teamId: agent.teamId,
userId: userReq?.id,
itemId: agent.id,
})
const info = await agentService.toAgentInfo(agent)
return c.json(info, 200)
},
)
.delete('/agents/:agentId', async (c) => {
const agentId = c.req.param('agentId')
const userReq = c.get('user')
+42
View File
@@ -148,6 +148,48 @@ describe('Chat API', () => {
)
})
it('accepts chat request with annotations, second, and attachmentIds', async () => {
vi.mocked(chatService.startOrContinueChat).mockResolvedValue({
sessionId: 'session1',
taskId: 'task1',
})
vi.mocked(chatService.getNewSessionMessages).mockResolvedValue({
messages: [],
lastEntryId: null,
})
vi.mocked(chatService.getChatWorkflowStatus).mockResolvedValue({
status: 'completed',
output: {},
})
const res = await app.request('/teams/team1/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
agentId: 'agent1',
textPrompt: 'Review annotation',
attachedFiles: ['att1'],
annotations: [{ type: 'rectangle', x: 10, y: 20 }],
second: 5.5,
}),
})
expect(res.status).toBe(200)
expect(chatService.startOrContinueChat).toHaveBeenCalledWith(
expect.objectContaining({ id: 'user1' }),
'team1',
expect.objectContaining({
agentId: 'agent1',
textPrompt: 'Review annotation',
attachedFiles: ['att1'],
annotations: [{ type: 'rectangle', x: 10, y: 20 }],
second: 5.5,
}),
)
})
it('sends ping keep-alive SSE events when there are no new messages', async () => {
vi.mocked(chatService.startOrContinueChat).mockResolvedValue({
sessionId: 'session1',
+95
View File
@@ -188,6 +188,46 @@ describe('folder api', () => {
type: ResourceType.Asset,
id: 'test-id',
})
expect(assetService.listChildren).toHaveBeenCalledWith({
assetId: 'test-id',
assetType: 'folder',
})
})
it('GET /folders/:folderId/children forwards pagination parameters', async () => {
vi.mocked(assetService.listChildren).mockResolvedValue({
data: [
{
id: 'child-id-2',
name: 'child-folder-2',
sizeByte: 0,
fileCount: 0,
type: 'folder',
status: 'active',
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
},
],
pageInfo: { total: 30, cursor: 'cursorNextToken' },
})
const app = new Hono().use('*', authMiddleware).route('/', folderRoute)
const res = await app.request(
'/folders/test-id/children?assetType=folder&first=20&after=cursorToken123&prefix=sub',
)
expect(res.status).toBe(200)
const json = await res.json()
expect(json.data).toHaveLength(1)
expect(json.pageInfo.cursor).toBe('cursorNextToken')
expect(assetService.listChildren).toHaveBeenCalledWith({
assetId: 'test-id',
assetType: 'folder',
first: 20,
after: 'cursorToken123',
prefix: 'sub',
})
})
it('DELETE /folders', async () => {
@@ -310,4 +350,59 @@ describe('folder api', () => {
beforeIndex: 'index-1',
})
})
it('GET /folders/:folderId/agentsmd', async () => {
vi.spyOn(assetService, 'getAgentsMd').mockResolvedValue('# Folder Guidelines')
const app = new Hono().use('*', authMiddleware).route('/', folderRoute)
const res = await app.request('/folders/test-id/agentsmd')
expect(res.status).toBe(200)
const json = await res.json()
expect(json.content).toBe('# Folder Guidelines')
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user1', name: 'Test User' },
permission: Permission.Read,
type: ResourceType.Asset,
id: 'test-id',
})
expect(assetService.getAgentsMd).toHaveBeenCalledWith('test-id')
})
it('PATCH /folders/:folderId/agentsmd', async () => {
vi.spyOn(assetService, 'updateAgentsMd').mockResolvedValue({ content: '# Updated Guidelines' })
const app = new Hono().use('*', authMiddleware).route('/', folderRoute)
const res = await app.request('/folders/test-id/agentsmd', {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ content: '# Updated Guidelines' }),
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.content).toBe('# Updated Guidelines')
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: { id: 'user1', name: 'Test User' },
permission: Permission.Admin,
type: ResourceType.Asset,
id: 'test-id',
})
expect(assetService.updateAgentsMd).toHaveBeenCalledWith('test-id', '# Updated Guidelines')
})
it('PATCH /folders/:folderId/agentsmd rejects oversized content', async () => {
const mockUpdate = vi.spyOn(assetService, 'updateAgentsMd').mockResolvedValue({ content: '' })
const app = new Hono().use('*', authMiddleware).route('/', folderRoute)
const res = await app.request('/folders/test-id/agentsmd', {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ content: 'x'.repeat(100_001) }),
})
expect(res.status).toBe(400)
expect(mockUpdate).not.toHaveBeenCalled()
})
})
+45 -5
View File
@@ -8,6 +8,7 @@ import {
updateFolderRequestSchema,
deleteFoldersRequestSchema,
restoreFoldersRequestSchema,
updateAgentsMdRequestSchema,
} from '@shumai/dtos'
import { listChildrenRequestSchema, updateAssetOrderRequestSchema, AuditAction } from '@shumai/dtos'
import { searchRequestSchema } from '@shumai/dtos'
@@ -119,12 +120,8 @@ const route = new Hono<{ Variables: { user: User } }>()
})
const resp = await assetService.listChildren({
...req,
assetId: folderId,
assetType: req.assetType,
projectId: req.projectId,
showDeleted: req.showDeleted,
sort: req.sort,
order: req.order,
})
return c.json(resp)
})
@@ -189,5 +186,48 @@ const route = new Hono<{ Variables: { user: User } }>()
const result = await searchService.search(folderId, req)
return c.json(result)
})
.get('/folders/:folderId/agentsmd', async (c) => {
const folderId = c.req.param('folderId')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Asset,
id: folderId,
})
const content = await assetService.getAgentsMd(folderId)
return c.json({ content })
})
.patch(
'/folders/:folderId/agentsmd',
zValidator('json', updateAgentsMdRequestSchema),
async (c) => {
const folderId = c.req.param('folderId')
const user = c.get('user')
const req = c.req.valid('json')
await authzService.hasPermission({
user,
permission: Permission.Admin,
type: ResourceType.Asset,
id: folderId,
})
const res = await assetService.updateAgentsMd(folderId, req.content)
const ctx = await assetService.getAssetContext(folderId)
await auditLogService.logAction({
action: AuditAction.folder_update,
teamId: ctx.teamId,
userId: user.id,
projectId: ctx.projectId,
itemId: folderId,
})
return c.json(res)
},
)
export default route
+4
View File
@@ -21,9 +21,11 @@ import s3Route from './s3'
import shareRoute from './share'
import skillRoute from './skill'
import watermarkTemplateRoute from './watermark-template'
import quotaRoute from './quota'
import teamRoute from './team'
import uploadRoute, { localUploadRoute } from './upload'
import versionStackRoute from './versionStack'
import kanbanRoute from './kanban'
type User = Prisma.UserGetPayload<Record<string, never>>
@@ -65,6 +67,8 @@ const apiRoute = new Hono<{ Variables: { user: User } }>()
.route('/', skillRoute)
.route('/', mcpRoute)
.route('/', watermarkTemplateRoute)
.route('/', quotaRoute)
.route('/', kanbanRoute)
app.route('/files', s3Route)
+130
View File
@@ -0,0 +1,130 @@
import { zValidator } from '@hono/zod-validator'
import { Hono } from 'hono'
import { kanbanService } from '@shumai/core/src/kanban/kanban'
import { notificationService } from '@shumai/core/src/notification/notification'
import { auditLogService } from '@shumai/core/src/auditLog/auditLog'
import {
authzService,
Permission,
ResourceType,
resolveEffectiveRole,
} from '@shumai/core/src/authz/authz'
import { createKanbanCommentSchema, postAttachmentRequestSchema, AuditAction } from '@shumai/dtos'
import type { Prisma } from '@shumai/db'
type User = Prisma.UserGetPayload<Record<string, never>>
const route = new Hono<{ Variables: { user: User } }>()
.post(
'/teams/:teamId/kanban/attachments',
zValidator('json', postAttachmentRequestSchema),
async (c) => {
const { teamId } = c.req.param()
const req = c.req.valid('json')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Team,
id: teamId,
})
const attachment = await kanbanService.createAttachment(teamId, req)
return c.json(attachment)
},
)
.post(
'/teams/:teamId/kanban/tasks/:taskId/comments',
zValidator('json', createKanbanCommentSchema),
async (c) => {
const { teamId, taskId } = c.req.param()
const req = c.req.valid('json')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.KanbanTask,
id: taskId,
})
const comment = await kanbanService.addComment(taskId, user.id, req.body, req.attachments)
const task = await kanbanService.getTask(taskId)
await auditLogService.logAction({
action: AuditAction.kanban_comment_create,
teamId,
userId: user.id,
projectId: task.projectId ?? undefined,
itemId: comment.id,
})
notificationService.notifyKanbanTaskEvent({
type: 'kanban_task_comment_created',
teamId,
projectId: task.projectId,
creatorId: user.id,
kanbanTaskId: taskId,
stakeholderIds: [task.creator?.id, task.assignee?.id, task.reporter?.id],
commentMessage: req.body,
})
return c.json(comment)
},
)
.get('/teams/:teamId/kanban/tasks/:taskId/comments', async (c) => {
const { taskId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.KanbanTask,
id: taskId,
})
const comments = await kanbanService.listComments(taskId)
return c.json({ data: comments })
})
.delete('/teams/:teamId/kanban/tasks/:taskId/comments/:commentId', async (c) => {
const { teamId, taskId, commentId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.KanbanTask,
id: taskId,
})
const userRole = await resolveEffectiveRole(teamId, undefined, user.id)
const task = await kanbanService.getTask(taskId)
await kanbanService.deleteComment(taskId, commentId, user.id, userRole)
await auditLogService.logAction({
action: AuditAction.kanban_comment_delete,
teamId,
userId: user.id,
projectId: task.projectId ?? undefined,
itemId: commentId,
})
return c.json({ success: true })
})
.get('/teams/:teamId/kanban/tasks/:taskId/events', async (c) => {
const { taskId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.KanbanTask,
id: taskId,
})
const events = await kanbanService.listEvents(taskId)
return c.json({ data: events })
})
export default route
+131
View File
@@ -0,0 +1,131 @@
import { zValidator } from '@hono/zod-validator'
import { Hono } from 'hono'
import { kanbanService } from '@shumai/core/src/kanban/kanban'
import { auditLogService } from '@shumai/core/src/auditLog/auditLog'
import {
authzService,
Permission,
ResourceType,
resolveEffectiveRole,
} from '@shumai/core/src/authz/authz'
import {
createKanbanGoalSchema,
updateKanbanGoalSchema,
listKanbanGoalsRequestSchema,
AuditAction,
} from '@shumai/dtos'
import type { Prisma } from '@shumai/db'
type User = Prisma.UserGetPayload<Record<string, never>>
const route = new Hono<{ Variables: { user: User } }>()
.post('/teams/:teamId/kanban/goals', zValidator('json', createKanbanGoalSchema), async (c) => {
const { teamId } = c.req.param()
const req = c.req.valid('json')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Admin,
type: ResourceType.Team,
id: teamId,
})
const role = await resolveEffectiveRole(teamId, undefined, user.id)
const goal = await kanbanService.createGoal(teamId, req, user.id, role)
await auditLogService.logAction({
action: AuditAction.kanban_goal_create,
teamId,
userId: user.id,
itemId: goal.id,
})
return c.json(goal)
})
.get(
'/teams/:teamId/kanban/goals',
zValidator('query', listKanbanGoalsRequestSchema),
async (c) => {
const { teamId } = c.req.param()
const req = c.req.valid('query')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Team,
id: teamId,
})
const goals = await kanbanService.listGoals(teamId, req)
return c.json({ data: goals })
},
)
.get('/teams/:teamId/kanban/goals/:goalId', async (c) => {
const { goalId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.KanbanGoal,
id: goalId,
})
const goal = await kanbanService.getGoal(goalId)
return c.json(goal)
})
.patch(
'/teams/:teamId/kanban/goals/:goalId',
zValidator('json', updateKanbanGoalSchema),
async (c) => {
const { teamId, goalId } = c.req.param()
const req = c.req.valid('json')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Admin,
type: ResourceType.KanbanGoal,
id: goalId,
})
const role = await resolveEffectiveRole(teamId, undefined, user.id)
const goal = await kanbanService.updateGoal(goalId, req, role)
await auditLogService.logAction({
action: AuditAction.kanban_goal_update,
teamId,
userId: user.id,
itemId: goalId,
})
return c.json(goal)
},
)
.delete('/teams/:teamId/kanban/goals/:goalId', async (c) => {
const { teamId, goalId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Admin,
type: ResourceType.KanbanGoal,
id: goalId,
})
const role = await resolveEffectiveRole(teamId, undefined, user.id)
await kanbanService.deleteGoal(goalId, role)
await auditLogService.logAction({
action: AuditAction.kanban_goal_delete,
teamId,
userId: user.id,
itemId: goalId,
})
return c.json({ ok: true })
})
export default route
+14
View File
@@ -0,0 +1,14 @@
import { Hono } from 'hono'
import type { Prisma } from '@shumai/db'
import goalsRoute from './goals'
import tasksRoute from './tasks'
import commentsRoute from './comments'
type User = Prisma.UserGetPayload<Record<string, never>>
const kanbanRoute = new Hono<{ Variables: { user: User } }>()
.route('/', goalsRoute)
.route('/', tasksRoute)
.route('/', commentsRoute)
export default kanbanRoute
+689
View File
@@ -0,0 +1,689 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { Hono, type Context, type Next } from 'hono'
import { kanbanService } from '@shumai/core/src/kanban/kanban'
import { auditLogService } from '@shumai/core/src/auditLog/auditLog'
import { notificationService } from '@shumai/core/src/notification/notification'
import { KanbanTaskStatus, KanbanTaskPriority } from '@shumai/db/enums'
import kanbanRoute from './index'
vi.mock('@shumai/core/src/authz/authz', () => ({
authzService: {
hasPermission: vi.fn().mockResolvedValue(undefined),
},
resolveEffectiveRole: vi.fn().mockResolvedValue('owner'),
Permission: {
Read: 'Read',
Edit: 'Edit',
Admin: 'Admin',
},
ResourceType: {
Team: 'team',
Project: 'project',
KanbanGoal: 'kanbanGoal',
KanbanTask: 'kanbanTask',
},
}))
vi.mock('@shumai/core/src/auditLog/auditLog', () => ({
auditLogService: {
logAction: vi.fn().mockResolvedValue({}),
},
}))
vi.mock('@shumai/core/src/notification/notification', () => ({
notificationService: {
notifyKanbanTaskEvent: vi.fn().mockResolvedValue(undefined),
create: vi.fn().mockResolvedValue(undefined),
},
}))
describe('Kanban API Routes', () => {
const teamId = 'team-1'
const taskId = 'task-1'
const goalId = 'goal-1'
const authMiddleware = async (c: Context, next: Next) => {
c.set('user', { id: 'user-1', name: 'Test User' })
await next()
}
let app: Hono
beforeEach(() => {
vi.clearAllMocks()
vi.mocked(auditLogService.logAction).mockResolvedValue({} as any) // eslint-disable-line @typescript-eslint/no-explicit-any
vi.mocked(notificationService.notifyKanbanTaskEvent).mockResolvedValue(undefined as any) // eslint-disable-line @typescript-eslint/no-explicit-any
app = new Hono()
app.use('*', authMiddleware)
app.route('/', kanbanRoute)
})
// --------------------------------------------------------------------------
// Goals Endpoints
// --------------------------------------------------------------------------
describe('Goals Endpoints', () => {
it('POST /teams/:teamId/kanban/goals', async () => {
const mockCreate = vi.spyOn(kanbanService, 'createGoal').mockResolvedValue({
id: goalId,
title: 'New Goal',
description: 'Goal description',
teamId,
creatorId: 'user-1',
taskCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const res = await app.request(`/teams/${teamId}/kanban/goals`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ title: 'New Goal', description: 'Goal description' }),
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.id).toBe(goalId)
expect(mockCreate).toHaveBeenCalledWith(
teamId,
{ title: 'New Goal', description: 'Goal description' },
'user-1',
'owner',
)
})
it('GET /teams/:teamId/kanban/goals', async () => {
const mockList = vi.spyOn(kanbanService, 'listGoals').mockResolvedValue([
{
id: goalId,
title: 'Goal 1',
teamId,
creatorId: 'user-1',
taskCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
},
])
const res = await app.request(`/teams/${teamId}/kanban/goals`)
expect(res.status).toBe(200)
const json = await res.json()
expect(json.data).toHaveLength(1)
expect(mockList).toHaveBeenCalledWith(teamId, expect.any(Object))
})
it('PATCH /teams/:teamId/kanban/goals/:goalId', async () => {
const mockUpdate = vi.spyOn(kanbanService, 'updateGoal').mockResolvedValue({
id: goalId,
title: 'Updated Goal',
teamId,
creatorId: 'user-1',
taskCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const res = await app.request(`/teams/${teamId}/kanban/goals/${goalId}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ title: 'Updated Goal' }),
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.title).toBe('Updated Goal')
expect(mockUpdate).toHaveBeenCalledWith(goalId, { title: 'Updated Goal' }, 'owner')
})
it('DELETE /teams/:teamId/kanban/goals/:goalId', async () => {
const mockDelete = vi.spyOn(kanbanService, 'deleteGoal').mockResolvedValue(undefined)
const res = await app.request(`/teams/${teamId}/kanban/goals/${goalId}`, {
method: 'DELETE',
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.ok).toBe(true)
expect(mockDelete).toHaveBeenCalledWith(goalId, 'owner')
})
})
// --------------------------------------------------------------------------
// Tasks Endpoints
// --------------------------------------------------------------------------
describe('Tasks Endpoints', () => {
it('POST /teams/:teamId/kanban/tasks', async () => {
const mockCreate = vi.spyOn(kanbanService, 'createTask').mockResolvedValue({
id: taskId,
title: 'New Task',
isAgentTask: false,
status: KanbanTaskStatus.READY,
priority: KanbanTaskPriority.MEDIUM,
teamId,
creator: { id: 'user-1', name: 'User 1' },
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const res = await app.request(`/teams/${teamId}/kanban/tasks`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ title: 'New Task', isAgentTask: false }),
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.id).toBe(taskId)
expect(mockCreate).toHaveBeenCalledWith(teamId, expect.any(Object), 'user-1', 'owner')
})
it('GET /teams/:teamId/kanban/tasks (with isAgentTask and goal filters)', async () => {
const mockList = vi.spyOn(kanbanService, 'listTasks').mockResolvedValue({
data: [
{
id: taskId,
title: 'Task 1',
isAgentTask: false,
status: KanbanTaskStatus.READY,
priority: KanbanTaskPriority.MEDIUM,
teamId,
creator: { id: 'user-1', name: 'User 1' },
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
},
],
pageInfo: { total: 1 },
})
const res = await app.request(
`/teams/${teamId}/kanban/tasks?isAgentTask=false&status=READY&goalId=goal-1`,
)
expect(res.status).toBe(200)
const json = await res.json()
expect(json.data).toHaveLength(1)
expect(mockList).toHaveBeenCalledWith(
teamId,
expect.objectContaining({
isAgentTask: false,
status: 'READY',
goalId: 'goal-1',
}),
)
})
it('GET /teams/:teamId/kanban/tasks/:taskId', async () => {
const mockGet = vi.spyOn(kanbanService, 'getTask').mockResolvedValue({
id: taskId,
title: 'Task 1',
isAgentTask: false,
status: KanbanTaskStatus.READY,
priority: KanbanTaskPriority.MEDIUM,
teamId,
creator: { id: 'user-1', name: 'User 1' },
dependencies: [],
dependents: [],
assets: [],
comments: [],
events: [],
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}`)
expect(res.status).toBe(200)
const json = await res.json()
expect(json.id).toBe(taskId)
expect(mockGet).toHaveBeenCalledWith(taskId)
})
it('PATCH /teams/:teamId/kanban/tasks/:taskId', async () => {
vi.spyOn(kanbanService, 'getTask').mockResolvedValue({
id: taskId,
title: 'Old Task Title',
isAgentTask: false,
status: KanbanTaskStatus.READY,
priority: KanbanTaskPriority.MEDIUM,
teamId,
creator: { id: 'user-1', name: 'User 1' },
dependencies: [],
dependents: [],
assets: [],
comments: [],
events: [],
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const mockUpdate = vi.spyOn(kanbanService, 'updateTask').mockResolvedValue({
id: taskId,
title: 'Updated Task Title',
isAgentTask: false,
status: KanbanTaskStatus.READY,
priority: KanbanTaskPriority.HIGH,
teamId,
creator: { id: 'user-1', name: 'User 1' },
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ priority: 'HIGH', status: 'DONE', reason: 'All done' }),
})
expect(res.status).toBe(200)
expect(mockUpdate).toHaveBeenCalledWith(
taskId,
{ priority: 'HIGH', status: 'DONE', reason: 'All done' },
'user-1',
'owner',
)
})
it('PATCH /teams/:teamId/kanban/tasks/:taskId with beforeIndex and afterIndex', async () => {
vi.spyOn(kanbanService, 'getTask').mockResolvedValue({
id: taskId,
title: 'Task 1',
isAgentTask: false,
status: KanbanTaskStatus.READY,
priority: KanbanTaskPriority.MEDIUM,
sortIndex: 'a0V',
teamId,
creator: { id: 'user-1', name: 'User 1' },
dependencies: [],
dependents: [],
assets: [],
comments: [],
events: [],
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const mockUpdate = vi.spyOn(kanbanService, 'updateTask').mockResolvedValue({
id: taskId,
title: 'Task 1',
isAgentTask: false,
status: KanbanTaskStatus.READY,
priority: KanbanTaskPriority.MEDIUM,
sortIndex: 'a0V',
teamId,
creator: { id: 'user-1', name: 'User 1' },
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ status: 'READY', beforeIndex: 'a1', afterIndex: 'a0' }),
})
expect(res.status).toBe(200)
expect(mockUpdate).toHaveBeenCalledWith(
taskId,
{ status: 'READY', beforeIndex: 'a1', afterIndex: 'a0' },
'user-1',
'owner',
)
})
it('PATCH /teams/:teamId/kanban/tasks/:taskId with fromStatus for CAS', async () => {
vi.spyOn(kanbanService, 'getTask').mockResolvedValue({
id: taskId,
title: 'Task 1',
isAgentTask: false,
status: KanbanTaskStatus.TODO,
priority: KanbanTaskPriority.MEDIUM,
teamId,
creator: { id: 'user-1', name: 'User 1' },
dependencies: [],
dependents: [],
assets: [],
comments: [],
events: [],
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const mockUpdate = vi.spyOn(kanbanService, 'updateTask').mockResolvedValue({
id: taskId,
title: 'Task 1',
isAgentTask: false,
status: KanbanTaskStatus.IN_PROGRESS,
priority: KanbanTaskPriority.MEDIUM,
teamId,
creator: { id: 'user-1', name: 'User 1' },
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ fromStatus: 'TODO', status: 'IN_PROGRESS' }),
})
expect(res.status).toBe(200)
expect(mockUpdate).toHaveBeenCalledWith(
taskId,
{ fromStatus: 'TODO', status: 'IN_PROGRESS' },
'user-1',
'owner',
)
})
it('DELETE /teams/:teamId/kanban/tasks/:taskId', async () => {
vi.spyOn(kanbanService, 'getTask').mockResolvedValue({
id: taskId,
title: 'Task to delete',
isAgentTask: false,
status: KanbanTaskStatus.READY,
priority: KanbanTaskPriority.MEDIUM,
teamId,
creator: { id: 'user-1', name: 'User 1' },
dependencies: [],
dependents: [],
assets: [],
comments: [],
events: [],
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const mockDelete = vi.spyOn(kanbanService, 'deleteTask').mockResolvedValue({
id: taskId,
title: 'Task to delete',
isAgentTask: false,
status: KanbanTaskStatus.READY,
priority: KanbanTaskPriority.MEDIUM,
teamId,
creator: { id: 'user-1', name: 'User 1' },
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 0,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}`, {
method: 'DELETE',
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.success).toBe(true)
expect(mockDelete).toHaveBeenCalledWith(taskId, 'user-1', 'owner')
})
it('GET /teams/:teamId/kanban/tasks/:taskId/assets', async () => {
const mockListTaskAssets = vi.spyOn(kanbanService, 'listTaskAssets').mockResolvedValue([
{
id: 'asset-1',
name: 'Video.mp4',
type: 'file',
proxyType: 'video',
thumbnailUrl: 'http://example.com/thumb.jpg',
path: '/Project/Video.mp4',
creator: { id: 'user-1', name: 'User 1' },
createdAt: new Date().toISOString(),
},
])
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}/assets`)
expect(res.status).toBe(200)
const json = await res.json()
expect(json).toHaveLength(1)
expect(json[0].name).toBe('Video.mp4')
expect(mockListTaskAssets).toHaveBeenCalledWith(taskId)
})
it('POST /teams/:teamId/kanban/tasks/:taskId/assets', async () => {
const mockLinkAssets = vi.spyOn(kanbanService, 'linkAssets').mockResolvedValue(undefined)
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}/assets`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ assetIds: ['asset-1', 'asset-2'] }),
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.success).toBe(true)
expect(mockLinkAssets).toHaveBeenCalledWith(teamId, taskId, ['asset-1', 'asset-2'])
})
it('DELETE /teams/:teamId/kanban/tasks/:taskId/assets/:assetId', async () => {
const mockUnlinkAsset = vi.spyOn(kanbanService, 'unlinkAsset').mockResolvedValue(undefined)
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}/assets/asset-1`, {
method: 'DELETE',
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.success).toBe(true)
expect(mockUnlinkAsset).toHaveBeenCalledWith(teamId, taskId, 'asset-1')
})
it('GET /teams/:teamId/kanban/assets/:assetId/tasks', async () => {
const mockListTasksForAsset = vi.spyOn(kanbanService, 'listTasksForAsset').mockResolvedValue([
{
id: taskId,
title: 'Task linked to asset',
isAgentTask: false,
status: KanbanTaskStatus.READY,
priority: KanbanTaskPriority.MEDIUM,
teamId,
creator: { id: 'user-1', name: 'User 1' },
commentCount: 0,
dependencyCount: 0,
dependentCount: 0,
assetCount: 1,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
},
])
const res = await app.request(`/teams/${teamId}/kanban/assets/asset-1/tasks`)
expect(res.status).toBe(200)
const json = await res.json()
expect(json.total).toBe(1)
expect(json.data).toHaveLength(1)
expect(mockListTasksForAsset).toHaveBeenCalledWith(teamId, 'asset-1')
})
it('POST /teams/:teamId/kanban/assets/:assetId/tasks', async () => {
const mockLinkAsset = vi.spyOn(kanbanService, 'linkAsset').mockResolvedValue(undefined)
const res = await app.request(`/teams/${teamId}/kanban/assets/asset-1/tasks`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ taskId }),
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.success).toBe(true)
expect(mockLinkAsset).toHaveBeenCalledWith(teamId, taskId, 'asset-1')
})
it('DELETE /teams/:teamId/kanban/assets/:assetId/tasks/:taskId', async () => {
const mockUnlinkAsset = vi.spyOn(kanbanService, 'unlinkAsset').mockResolvedValue(undefined)
const res = await app.request(`/teams/${teamId}/kanban/assets/asset-1/tasks/${taskId}`, {
method: 'DELETE',
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.success).toBe(true)
expect(mockUnlinkAsset).toHaveBeenCalledWith(teamId, taskId, 'asset-1')
})
})
// --------------------------------------------------------------------------
// Comments, Attachments & Events
// --------------------------------------------------------------------------
describe('Comments, Attachments and Events Endpoints', () => {
it('POST /teams/:teamId/kanban/attachments', async () => {
const mockAttachment = vi.spyOn(kanbanService, 'createAttachment').mockResolvedValue({
id: 'att-1',
name: 'test.png',
key: 'kanban/attachments/test.png',
sizeByte: 1024,
uploadUrl: 'http://s3/upload',
proxyType: 'image',
})
const res = await app.request(`/teams/${teamId}/kanban/attachments`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ fileName: 'test.png', size: 1024, contentType: 'image/png' }),
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.id).toBe('att-1')
expect(mockAttachment).toHaveBeenCalledWith(teamId, {
fileName: 'test.png',
size: 1024,
contentType: 'image/png',
})
})
it('POST /teams/:teamId/kanban/tasks/:taskId/comments', async () => {
const mockAdd = vi.spyOn(kanbanService, 'addComment').mockResolvedValue({
id: 'c-1',
taskId,
author: { id: 'user-1', name: 'User 1' },
body: 'Here is a note',
attachments: [],
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}/comments`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
body: 'Here is a note',
attachments: [
{
id: 'att-1',
name: 'test.png',
key: 'key-1',
sizeByte: 1024,
proxyType: 'image',
},
],
}),
})
expect(res.status).toBe(200)
expect(mockAdd).toHaveBeenCalledWith(taskId, 'user-1', 'Here is a note', [
{
id: 'att-1',
name: 'test.png',
key: 'key-1',
sizeByte: 1024,
proxyType: 'image',
},
])
})
it('DELETE /teams/:teamId/kanban/tasks/:taskId/comments/:commentId', async () => {
const mockDelete = vi.spyOn(kanbanService, 'deleteComment').mockResolvedValue()
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}/comments/c-1`, {
method: 'DELETE',
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.success).toBe(true)
expect(mockDelete).toHaveBeenCalledWith(taskId, 'c-1', 'user-1', 'owner')
})
it('GET /teams/:teamId/kanban/tasks/:taskId/comments', async () => {
const mockList = vi.spyOn(kanbanService, 'listComments').mockResolvedValue([
{
id: 'c-1',
taskId,
author: { id: 'user-1', name: 'User 1' },
body: 'Note',
attachments: [],
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
},
])
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}/comments`)
expect(res.status).toBe(200)
const json = await res.json()
expect(json.data).toHaveLength(1)
expect(mockList).toHaveBeenCalledWith(taskId)
})
it('GET /teams/:teamId/kanban/tasks/:taskId/events', async () => {
const mockEvents = vi.spyOn(kanbanService, 'listEvents').mockResolvedValue([
{
id: 'e-1',
taskId,
actor: { id: 'user-1', name: 'User 1' },
// eslint-disable-next-line @typescript-eslint/no-explicit-any
type: 'CREATED' as any,
createdAt: new Date().toISOString(),
},
])
const res = await app.request(`/teams/${teamId}/kanban/tasks/${taskId}/events`)
expect(res.status).toBe(200)
const json = await res.json()
expect(json.data).toHaveLength(1)
expect(mockEvents).toHaveBeenCalledWith(taskId)
})
})
})
+331
View File
@@ -0,0 +1,331 @@
import { zValidator } from '@hono/zod-validator'
import { Hono } from 'hono'
import { kanbanService } from '@shumai/core/src/kanban/kanban'
import { notificationService } from '@shumai/core/src/notification/notification'
import { auditLogService } from '@shumai/core/src/auditLog/auditLog'
import {
authzService,
Permission,
ResourceType,
resolveEffectiveRole,
} from '@shumai/core/src/authz/authz'
import {
createKanbanTaskSchema,
updateKanbanTaskSchema,
listKanbanTasksRequestSchema,
linkTaskAssetsSchema,
linkAssetToTaskSchema,
AuditAction,
} from '@shumai/dtos'
import type { Prisma } from '@shumai/db'
import { HTTPException } from 'hono/http-exception'
type User = Prisma.UserGetPayload<Record<string, never>>
const route = new Hono<{ Variables: { user: User } }>()
.post('/teams/:teamId/kanban/tasks', zValidator('json', createKanbanTaskSchema), async (c) => {
const { teamId } = c.req.param()
const req = c.req.valid('json')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Team,
id: teamId,
})
const role = await resolveEffectiveRole(teamId, undefined, user.id)
if (!role) {
throw new HTTPException(403, { message: 'User is not a member of the team' })
}
const task = await kanbanService.createTask(teamId, req, user.id, role)
await auditLogService.logAction({
action: AuditAction.kanban_task_create,
teamId,
userId: user.id,
projectId: task.projectId ?? undefined,
itemId: task.id,
})
if (req.assigneeId && req.assigneeId !== user.id) {
notificationService.notifyKanbanTaskEvent({
type: 'kanban_task_assigned',
teamId,
projectId: task.projectId,
creatorId: user.id,
kanbanTaskId: task.id,
targetUserId: req.assigneeId,
})
}
if (req.reporterId && req.reporterId !== user.id && req.reporterId !== req.assigneeId) {
notificationService.notifyKanbanTaskEvent({
type: 'kanban_task_created',
teamId,
projectId: task.projectId,
creatorId: user.id,
kanbanTaskId: task.id,
targetUserId: req.reporterId,
})
}
return c.json(task)
})
.get(
'/teams/:teamId/kanban/tasks',
zValidator('query', listKanbanTasksRequestSchema),
async (c) => {
const { teamId } = c.req.param()
const req = c.req.valid('query')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Team,
id: teamId,
})
const result = await kanbanService.listTasks(teamId, req)
return c.json(result)
},
)
.get('/teams/:teamId/kanban/tasks/:taskId', async (c) => {
const { taskId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.KanbanTask,
id: taskId,
})
const task = await kanbanService.getTask(taskId)
return c.json(task)
})
.patch(
'/teams/:teamId/kanban/tasks/:taskId',
zValidator('json', updateKanbanTaskSchema),
async (c) => {
const { teamId, taskId } = c.req.param()
const req = c.req.valid('json')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Edit,
type: ResourceType.KanbanTask,
id: taskId,
})
const existing = await kanbanService.getTask(taskId)
const role = await resolveEffectiveRole(teamId, existing.projectId ?? undefined, user.id)
const task = await kanbanService.updateTask(taskId, req, user.id, role)
const hasMeaningfulChanges =
req.title !== undefined ||
req.description !== undefined ||
req.priority !== undefined ||
req.startDate !== undefined ||
req.dueDate !== undefined ||
req.goalId !== undefined ||
req.assigneeId !== undefined ||
req.reporterId !== undefined ||
req.targetFolderId !== undefined ||
(req.status !== undefined && req.status !== existing.status)
if (hasMeaningfulChanges) {
await auditLogService.logAction({
action: AuditAction.kanban_task_update,
teamId,
userId: user.id,
projectId: task.projectId ?? undefined,
itemId: task.id,
})
}
if (req.status !== undefined && req.status !== existing.status) {
notificationService.notifyKanbanTaskEvent({
type: 'kanban_task_status_updated',
teamId,
projectId: task.projectId,
creatorId: user.id,
kanbanTaskId: task.id,
stakeholderIds: [task.creator?.id, task.assignee?.id, task.reporter?.id],
})
}
if (
req.assigneeId !== undefined &&
req.assigneeId !== existing.assignee?.id &&
req.assigneeId !== null &&
req.assigneeId !== user.id
) {
notificationService.notifyKanbanTaskEvent({
type: 'kanban_task_assigned',
teamId,
projectId: task.projectId,
creatorId: user.id,
kanbanTaskId: task.id,
targetUserId: req.assigneeId,
})
}
const hasOtherDetailsChanged =
req.title !== undefined ||
req.description !== undefined ||
req.priority !== undefined ||
req.startDate !== undefined ||
req.dueDate !== undefined ||
req.goalId !== undefined ||
req.reporterId !== undefined ||
req.targetFolderId !== undefined
if (hasOtherDetailsChanged && req.status === undefined) {
notificationService.notifyKanbanTaskEvent({
type: 'kanban_task_updated',
teamId,
projectId: task.projectId,
creatorId: user.id,
kanbanTaskId: task.id,
stakeholderIds: [task.creator?.id, task.assignee?.id, task.reporter?.id],
})
}
return c.json(task)
},
)
.delete('/teams/:teamId/kanban/tasks/:taskId', async (c) => {
const { teamId, taskId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Edit,
type: ResourceType.KanbanTask,
id: taskId,
})
const existing = await kanbanService.getTask(taskId)
const role = await resolveEffectiveRole(teamId, existing.projectId ?? undefined, user.id)
notificationService.notifyKanbanTaskEvent({
type: 'kanban_task_deleted',
teamId,
projectId: existing.projectId,
creatorId: user.id,
kanbanTaskId: taskId,
stakeholderIds: [existing.creator?.id, existing.assignee?.id, existing.reporter?.id],
})
await auditLogService.logAction({
action: AuditAction.kanban_task_delete,
teamId,
userId: user.id,
projectId: existing.projectId ?? undefined,
itemId: taskId,
})
await kanbanService.deleteTask(taskId, user.id, role)
return c.json({ success: true })
})
.get('/teams/:teamId/kanban/tasks/:taskId/assets', async (c) => {
const { taskId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.KanbanTask,
id: taskId,
})
const assets = await kanbanService.listTaskAssets(taskId)
return c.json(assets)
})
.post(
'/teams/:teamId/kanban/tasks/:taskId/assets',
zValidator('json', linkTaskAssetsSchema),
async (c) => {
const { teamId, taskId } = c.req.param()
const req = c.req.valid('json')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Edit,
type: ResourceType.KanbanTask,
id: taskId,
})
await kanbanService.linkAssets(teamId, taskId, req.assetIds)
return c.json({ success: true })
},
)
.delete('/teams/:teamId/kanban/tasks/:taskId/assets/:assetId', async (c) => {
const { teamId, taskId, assetId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Edit,
type: ResourceType.KanbanTask,
id: taskId,
})
await kanbanService.unlinkAsset(teamId, taskId, assetId)
return c.json({ success: true })
})
.get('/teams/:teamId/kanban/assets/:assetId/tasks', async (c) => {
const { teamId, assetId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Read,
type: ResourceType.Team,
id: teamId,
})
const tasks = await kanbanService.listTasksForAsset(teamId, assetId)
return c.json({ data: tasks, total: tasks.length })
})
.post(
'/teams/:teamId/kanban/assets/:assetId/tasks',
zValidator('json', linkAssetToTaskSchema),
async (c) => {
const { teamId, assetId } = c.req.param()
const req = c.req.valid('json')
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Edit,
type: ResourceType.KanbanTask,
id: req.taskId,
})
await kanbanService.linkAsset(teamId, req.taskId, assetId)
return c.json({ success: true })
},
)
.delete('/teams/:teamId/kanban/assets/:assetId/tasks/:taskId', async (c) => {
const { teamId, assetId, taskId } = c.req.param()
const user = c.get('user')
await authzService.hasPermission({
user,
permission: Permission.Edit,
type: ResourceType.KanbanTask,
id: taskId,
})
await kanbanService.unlinkAsset(teamId, taskId, assetId)
return c.json({ success: true })
})
export default route
+7 -13
View File
@@ -46,10 +46,9 @@ describe('metadata api', () => {
const mockField = {
key: 'field1',
scope: 'TEAM',
config: { name: 'Test Field', type: 'text' },
config: { name: 'Test Field', type: 'text', autofillSource: 'CONTENT' },
readOnly: false,
description: 'desc',
aiAutofill: true,
}
// Using any here because mocking complex service return types or Hono context is overly verbose for this test.
@@ -62,7 +61,7 @@ describe('metadata api', () => {
expect(json).toHaveLength(1)
expect(json[0].id).toBe('field1')
expect(json[0].config.name).toBe('Test Field')
expect(json[0].aiAutofill).toBe(true)
expect(json[0].config.autofillSource).toBe('CONTENT')
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: expect.anything(),
permission: Permission.Read,
@@ -75,10 +74,9 @@ describe('metadata api', () => {
const mockField = {
key: 'newfield',
scope: 'TEAM',
config: { name: 'New Field', type: 'text' },
config: { name: 'New Field', type: 'text', autofillSource: 'CONTENT' },
readOnly: false,
description: 'desc',
aiAutofill: true,
}
// Using any here because mocking complex service return types or Hono context is overly verbose for this test.
@@ -88,8 +86,7 @@ describe('metadata api', () => {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
config: { name: 'New Field', type: 'text' },
aiAutofill: true,
config: { name: 'New Field', type: 'text', autofillSource: 'CONTENT' },
description: 'desc',
}),
})
@@ -115,10 +112,9 @@ describe('metadata api', () => {
const mockField = {
key: 'field1',
scope: 'PROJECT',
config: { name: 'Project Field', type: 'text' },
config: { name: 'Project Field', type: 'text', autofillSource: 'NONE' },
readOnly: false,
description: 'desc',
aiAutofill: false,
}
vi.mocked(metadataService.listProjectFields).mockResolvedValue([
@@ -198,10 +194,9 @@ describe('metadata api', () => {
const mockField = {
key: 'f1',
scope: 'TEAM',
config: { name: 'Updated Field', type: 'text' },
config: { name: 'Updated Field', type: 'text', autofillSource: 'CONTENT' },
readOnly: false,
description: 'desc',
aiAutofill: true,
teamId: 't1',
}
@@ -214,8 +209,7 @@ describe('metadata api', () => {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
config: { name: 'Updated Field', type: 'text' },
aiAutofill: true,
config: { name: 'Updated Field', type: 'text', autofillSource: 'CONTENT' },
description: 'desc',
}),
})
-1
View File
@@ -27,7 +27,6 @@ function toFieldInfo(
readOnly: f.readOnly,
visible: visible,
description: f.description,
aiAutofill: f.aiAutofill,
}
}
+12
View File
@@ -31,6 +31,7 @@ describe('authMiddleware', () => {
.get('/test', (c) => c.text('ok'))
.post('/test', (c) => c.text('ok'))
.post('/test/search', (c) => c.text('ok'))
.post('/test/recents/view', (c) => c.text('ok'))
beforeEach(() => {
vi.resetAllMocks()
@@ -81,6 +82,17 @@ describe('authMiddleware', () => {
expect(await res.json()).toEqual({ error: 'System is in read-only mode' })
})
it('allows POST /recents/view for reviewer', async () => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
vi.mocked(auth.api.getSession).mockResolvedValue({ user: { id: 'user1' } } as any)
// eslint-disable-next-line @typescript-eslint/no-explicit-any
vi.mocked(userService.getUserById).mockResolvedValue({ id: 'user1' } as any)
vi.mocked(teamService.hasWritableRoleInAnyTeam).mockResolvedValue(false)
const res = await app.request('/test/recents/view', { method: 'POST' })
expect(res.status).toBe(200)
})
it('allows POST /search for reviewer', async () => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
vi.mocked(auth.api.getSession).mockResolvedValue({ user: { id: 'user1' } } as any)
+3 -1
View File
@@ -36,7 +36,9 @@ export const authMiddleware = createMiddleware<{
if (process.env.SHUMAI_DEMO_MODE === '1') {
const method = c.req.method.toUpperCase()
const path = c.req.path
if (['POST', 'PUT', 'PATCH', 'DELETE'].includes(method) && !path.endsWith('/search')) {
const isAllowedDemoWrite =
method === 'POST' && (path.endsWith('/search') || path.endsWith('/recents/view'))
if (['POST', 'PUT', 'PATCH', 'DELETE'].includes(method) && !isAllowedDemoWrite) {
const hasWritable = await teamService.hasWritableRoleInAnyTeam(user.id)
if (!hasWritable) {
+2
View File
@@ -150,6 +150,8 @@ describe('notification api', () => {
yourUploads: true,
otherUploads: false,
statusUpdates: true,
kanbanTasks: true,
kanbanComments: true,
},
)
})
+112
View File
@@ -35,6 +35,7 @@ vi.mock('@shumai/core/src/project/project', () => ({
},
}))
vi.mock('@shumai/core/src/asset/asset')
vi.mock('@shumai/core/src/auditLog/auditLog', () => ({
auditLogService: {
logAction: vi.fn().mockResolvedValue({}),
@@ -264,6 +265,75 @@ describe('project api', () => {
expect(assetService.emptyTrash).toHaveBeenCalledWith('p1')
})
it('GET /projects/:projectId/members', async () => {
vi.mocked(projectService.listProjectMembers).mockResolvedValue([
{
id: 'user1',
name: 'Test User',
role: 'editor',
type: 'human',
},
])
const res = await app.request('/projects/p1/members')
expect(res.status).toBe(200)
const json = await res.json()
expect(json).toEqual([
{
id: 'user1',
name: 'Test User',
role: 'editor',
type: 'human',
},
])
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: expect.anything(),
permission: Permission.Read,
type: ResourceType.Project,
id: 'p1',
})
expect(projectService.listProjectMembers).toHaveBeenCalledWith({
projectId: 'p1',
includeAgents: false,
requesterUserId: 'user1',
})
})
it('GET /projects/:projectId/members?includeAgents=true', async () => {
vi.mocked(projectService.listProjectMembers).mockResolvedValue([
{
id: 'user1',
name: 'Test User',
role: 'editor',
type: 'human',
},
{
id: 'agent1',
name: 'Agent Bot',
role: 'bot',
type: 'agent',
},
])
const res = await app.request('/projects/p1/members?includeAgents=true')
expect(res.status).toBe(200)
const json = await res.json()
expect(json).toHaveLength(2)
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: expect.anything(),
permission: Permission.Read,
type: ResourceType.Project,
id: 'p1',
})
expect(projectService.listProjectMembers).toHaveBeenCalledWith({
projectId: 'p1',
includeAgents: true,
requesterUserId: 'user1',
})
})
it('POST /projects/:projectId/members', async () => {
vi.mocked(projectService.addProjectMember).mockResolvedValue(undefined)
@@ -341,4 +411,46 @@ describe('project api', () => {
})
expect(projectService.getUserProjects).toHaveBeenCalledWith('user1', 10)
})
it('GET /projects/:projectId/recents', async () => {
vi.mocked(assetService.listRecents).mockResolvedValue({
data: [{ id: 'file1', name: 'video.mp4', type: 'file' }] as any, // eslint-disable-line @typescript-eslint/no-explicit-any
pageInfo: { total: 1 },
})
const res = await app.request('/projects/p1/recents?first=20')
expect(res.status).toBe(200)
const json = await res.json()
expect(json.data).toHaveLength(1)
expect(json.data[0].id).toBe('file1')
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: expect.anything(),
permission: Permission.Read,
type: ResourceType.Project,
id: 'p1',
})
expect(assetService.listRecents).toHaveBeenCalledWith('user1', 'p1', { first: 20 })
})
it('POST /projects/:projectId/recents/view', async () => {
vi.mocked(assetService.recordRecentView).mockResolvedValue(undefined)
const res = await app.request('/projects/p1/recents/view', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ assetId: 'file1' }),
})
expect(res.status).toBe(200)
const json = await res.json()
expect(json.success).toBe(true)
expect(authzService.hasPermission).toHaveBeenCalledWith({
user: expect.anything(),
permission: Permission.Read,
type: ResourceType.Project,
id: 'p1',
})
expect(assetService.recordRecentView).toHaveBeenCalledWith('user1', 'p1', 'file1')
})
})

Some files were not shown because too many files have changed in this diff Show More