mirror of
https://github.com/shumaiOne/shumai.git
synced 2026-11-19 22:56:54 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d8c20d6a11 | ||
|
|
8697481d43 | ||
|
|
4ac274cdd4 | ||
|
|
eea844320d | ||
|
|
d255e76806 | ||
|
|
2be1093b91 | ||
|
|
83e902862c |
@@ -12,6 +12,21 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
### Changed
|
||||
|
||||
## [0.7.0] - 2026-11-18
|
||||
|
||||
### Added
|
||||
|
||||
- **i18n**: Add Korean as a display language, selectable on the sign-up page and in team settings.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **cli**: Fix release publication failing to find staged standalone downloads.
|
||||
|
||||
### Changed
|
||||
|
||||
- **agent**: ⚠️ **Breaking Change**: Replaced the built-in server-side Bash sandbox with external Runners and removed the team-level sandbox domain allowlist settings. The Bash tool is now disabled by default. If your Agent or Skills require Bash execution, you'll need to set up and connect an external Runner using shumai-cli. The Shumai CLI includes built-in sandbox support, allowing you to configure filesystem access and network domain allowlists for Bash execution directly on the Runner. See [Agent Runners](https://docs.shumai.one/using-shumai/agent-sandbox) for setup instructions and configuration details. The Docker Compose configuration has also been updated to improve security following the removal of the server-side sandbox. We strongly recommend upgrading to the latest Shumai image and updating your Docker Compose file to apply these security improvements.
|
||||
- **cli**: Show copyable commands and shell configuration instructions when the standalone CLI installation directory is missing from PATH.
|
||||
|
||||
## [0.6.6] - 2026-11-13
|
||||
|
||||
### Added
|
||||
|
||||
@@ -46,9 +46,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ripgrep \
|
||||
procps \
|
||||
lsof \
|
||||
socat \
|
||||
ca-certificates \
|
||||
bubblewrap \
|
||||
gosu \
|
||||
poppler-utils \
|
||||
imagemagick \
|
||||
|
||||
+42
-2
@@ -32,12 +32,14 @@ powershell -NoProfile -ExecutionPolicy Bypass -Command "irm https://github.com/s
|
||||
|
||||
Native builds use Bun 1.4.4's runtime; x64 CPUs need SSE4.2. See [Bun's system requirements](https://bun.com/docs/installation). Alpine/musl is not included in these native downloads. Native downloads are available for releases containing the installer assets; use npm for earlier releases.
|
||||
|
||||
On macOS/Linux the default directory is `~/.local/bin`. Add it to your shell's PATH if needed:
|
||||
On macOS/Linux the default directory is `~/.local/bin`. If it is missing from your current PATH, the installer prints a command to run in your current Bash/Zsh terminal:
|
||||
|
||||
```sh
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
```
|
||||
|
||||
To keep the CLI available in future terminals, add the same export line to `~/.bashrc` (Bash) or `~/.zshrc` (Zsh). For Bash login shells, use your existing login profile, such as `~/.bash_profile` or `~/.profile`. Open a new terminal after saving. If you already added it, you do not need to add it again. For a custom installation directory, use the exact command printed by the installer.
|
||||
|
||||
On Windows the installer uses `%LOCALAPPDATA%\Shumai\bin` and updates the user PATH. Open a new terminal afterward. Administrator access is not required.
|
||||
|
||||
Re-run the installer to upgrade. To pin a version, roll back, or choose an installation directory, download the installer and pass its options (replace `X.Y.Z` with a release containing native binaries):
|
||||
@@ -84,6 +86,44 @@ export SHUMAI_ALLOW_DELETE="true"
|
||||
|
||||
---
|
||||
|
||||
## Agent runner
|
||||
|
||||
Create a runner in **Settings → Agent Runners** and copy its one-time key. On macOS, Linux, or Windows, start it in an existing dedicated workspace:
|
||||
|
||||
```sh
|
||||
export SHUMAI_API_SERVER="https://shumai.example.com"
|
||||
export SHUMAI_RUNNER_KEY="your-runner-key"
|
||||
shumai-cli runner start --cwd /path/to/workspace
|
||||
```
|
||||
|
||||
For Windows PowerShell:
|
||||
|
||||
```powershell
|
||||
$env:SHUMAI_API_SERVER = "https://shumai.example.com"
|
||||
$env:SHUMAI_RUNNER_KEY = "your-runner-key"
|
||||
shumai-cli runner start --cwd C:\workspace
|
||||
```
|
||||
|
||||
Use `--url` or `--key` to override those variables. Pass `-v` or `--verbose` to log executed commands, output sizes, and file transfer paths (by default, only lifecycle events, execution IDs, exit status, and durations are printed to stderr). A runner key is separate from `SHUMAI_API_KEY`. The runner stays in the foreground; Ctrl+C stops it and its active commands. Remote servers require HTTPS; localhost and `host.docker.internal` support HTTP for development. No inbound port is opened.
|
||||
|
||||
Commands use `/bin/sh` on macOS and Linux. On Windows, the runner checks for Git Bash in the standard installation directories, then for `bash.exe` on PATH, then falls back to `cmd.exe`. It prints the selected shell at startup and reports it to the agent. Commands run with the local user's permissions unless SRT is requested.
|
||||
|
||||
[Anthropic Sandbox Runtime](https://github.com/anthropics/sandbox-runtime) is bundled with the CLI. Install its platform prerequisites (Linux: bubblewrap, socat, and ripgrep; macOS: ripgrep), then pass its native JSON settings:
|
||||
|
||||
```sh
|
||||
shumai-cli runner start --srt sandbox.json
|
||||
```
|
||||
|
||||
Windows sandboxing uses SRT's alpha Windows support. Use Git Bash if your selected shell is the legacy WSL Bash launcher. Run the one-time SRT setup before starting a sandboxed runner (it requests administrator permission):
|
||||
|
||||
```powershell
|
||||
npx @anthropic-ai/sandbox-runtime windows-install
|
||||
```
|
||||
|
||||
SRT settings are read and validated at startup. Invalid settings or unavailable sandbox prerequisites stop startup; commands never fall back to unrestricted execution. Only shell commands and their children are sandboxed. Runner connections and server-directed file transfers stay outside SRT, and no server domain is automatically allowed. The shared SRT runtime is cleaned up when the runner stops.
|
||||
|
||||
See [Agent Runners](../../docs/using-shumai/agent-sandbox.mdx) for configuration, Docker usage, limits, and security boundaries.
|
||||
|
||||
## Usage
|
||||
|
||||
> [!IMPORTANT]
|
||||
@@ -328,7 +368,7 @@ bun run build:cli --target darwin-arm64 # or darwin-x64, linux-arm64, linux-x64,
|
||||
bun run test:cli-release
|
||||
```
|
||||
|
||||
Without `--target`, the build produces all six targets. Output defaults to `out/cli-release`; both commands accept `--out /absolute/path`. Unix archives require `tar`; creating Windows archives on Unix also requires `zip`. The test runs the generated installer against local HTTP fixtures, then authenticates, creates a folder, uploads and downloads random bytes, and checks failure recovery with no runtime on the executable's PATH.
|
||||
Without `--target`, the build produces all six targets. Output defaults to `out/cli-release`. Set `--out /absolute/path` on either command to change it. Unix archives require `tar`. Creating Windows archives on Unix also requires `zip`. The test exercises installation, runner shell quoting, authenticated API calls, file transfers, and failure recovery. Install Git Bash to run the quoting check on Windows. With SRT configured, pass `--srt /absolute/path/to/settings.json` to repeat the quoting check in the sandbox.
|
||||
|
||||
The Native CLI workflow builds and tests each target on a matching runner. The npm release workflow assembles those tested archives, stages a draft GitHub Release, publishes npm packages, then publishes the completed release. Its `validate_only` dispatch input builds and tests without publishing. Published assets are verified and preserved on retries; draft assets can be replaced while resuming a failed run. Prerelease npm packages use the `next` tag and prerelease GitHub Releases do not become latest.
|
||||
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
The Windows Bash discovery and related tests include material adapted from Pi.
|
||||
https://github.com/earendil-works/pi
|
||||
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mario Zechner
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
+11
-1
@@ -1,19 +1,29 @@
|
||||
{
|
||||
"name": "@shumai/cli",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"bin": {
|
||||
"shumai-cli": "./src/index.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@anthropic-ai/sandbox-runtime": "0.0.83",
|
||||
"@shumai/api": "workspace:*",
|
||||
"@shumai/dtos": "workspace:*",
|
||||
"@shumai/operations": "workspace:*",
|
||||
"@shumai/tableprinter": "workspace:*",
|
||||
"commander": "15.0.0",
|
||||
"execa": "10.2.1",
|
||||
"hono": "4.13.5",
|
||||
"ulid": "3.0.2",
|
||||
"ws": "8.22.2",
|
||||
"yauzl": "3.4.0",
|
||||
"zod": "4.5.4"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/adm-zip": "0.5.8",
|
||||
"@types/ws": "8.18.2",
|
||||
"@types/yauzl": "3.4.0",
|
||||
"adm-zip": "0.6.1"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -33,8 +33,10 @@ function setup() {
|
||||
let stdout = ''
|
||||
let stderr = ''
|
||||
const callOperation = vi.fn<CallOperation>()
|
||||
const startRunner = vi.fn(async () => {})
|
||||
const program = createProgram({
|
||||
callOperation,
|
||||
startRunner,
|
||||
output: { out: { write: (s) => void (stdout += s) }, isTty: true, columns: 120 },
|
||||
})
|
||||
const configure = (cmd: Command) => {
|
||||
@@ -47,6 +49,7 @@ function setup() {
|
||||
configure(program)
|
||||
return {
|
||||
callOperation,
|
||||
startRunner,
|
||||
run: (...argv: string[]) => program.parseAsync(['node', 'shumai-cli', ...argv]),
|
||||
stdout: () => stdout,
|
||||
stderr: () => stderr,
|
||||
@@ -857,6 +860,23 @@ describe('shumai-cli', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('runner start', () => {
|
||||
it('documents and forwards the -v, --verbose flag', async () => {
|
||||
const cli = setup()
|
||||
await expect(cli.run('runner', 'start', '--help')).rejects.toThrow()
|
||||
expect(cli.stdout()).toContain('-v, --verbose')
|
||||
|
||||
await cli.run('runner', 'start')
|
||||
expect(cli.startRunner).toHaveBeenLastCalledWith({})
|
||||
|
||||
await cli.run('runner', 'start', '-v')
|
||||
expect(cli.startRunner).toHaveBeenLastCalledWith({ verbose: true })
|
||||
|
||||
await cli.run('runner', 'start', '--verbose')
|
||||
expect(cli.startRunner).toHaveBeenLastCalledWith({ verbose: true })
|
||||
})
|
||||
})
|
||||
|
||||
it('has no upload Operation commands: upload and create-version stream local files', () => {
|
||||
const program = createProgram({ callOperation: vi.fn<CallOperation>() })
|
||||
const names = program.commands.map((c) => c.name())
|
||||
|
||||
@@ -7,10 +7,13 @@ import { cliExtensions } from './operations/extensions'
|
||||
import { registerOperationCommands } from './operations/generate'
|
||||
import type { Output } from './operations/render'
|
||||
import { version } from '../package.json'
|
||||
import { startRunner } from './runner/client'
|
||||
import type { RunnerOptions } from './runner/config'
|
||||
|
||||
export interface ProgramOptions {
|
||||
callOperation?: CallOperation
|
||||
output?: Output
|
||||
startRunner?: (options: RunnerOptions) => Promise<void>
|
||||
}
|
||||
|
||||
function isDeleteAllowed(): boolean {
|
||||
@@ -43,6 +46,18 @@ export function createProgram(options: ProgramOptions = {}): Command {
|
||||
extensions: cliExtensions,
|
||||
})
|
||||
|
||||
program
|
||||
.command('runner')
|
||||
.description('Connect a local command runner to Shumai.')
|
||||
.command('start')
|
||||
.description('Run in the foreground using a key created in Settings → Agent Runners.')
|
||||
.option('--url <url>', 'Shumai server URL (or SHUMAI_API_SERVER).')
|
||||
.option('--key <key>', 'Runner key (prefer SHUMAI_RUNNER_KEY to avoid shell history).')
|
||||
.option('--cwd <directory>', 'Runner workspace directory (defaults to the current directory).')
|
||||
.option('--srt <file>', 'Sandbox commands with bundled SRT using this JSON settings file.')
|
||||
.option('-v, --verbose', 'Show command details, output sizes, and file transfer paths.')
|
||||
.action((opts: RunnerOptions) => (options.startRunner ?? startRunner)(opts))
|
||||
|
||||
program
|
||||
.command('upload')
|
||||
.description('Upload a local file or folder to a parent folder.')
|
||||
|
||||
@@ -0,0 +1,446 @@
|
||||
import { createServer } from 'node:http'
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { WebSocketServer, type WebSocket } from 'ws'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { runnerToServerMessageSchema, type RunnerToServerMessage } from '@shumai/dtos'
|
||||
import { RunnerClient } from './client'
|
||||
import { RunnerFiles } from './files'
|
||||
import { RunnerShell } from './shell'
|
||||
|
||||
describe('runner WebSocket client', () => {
|
||||
let cwd: string
|
||||
let server: WebSocketServer
|
||||
let runner: RunnerClient
|
||||
let running: Promise<void>
|
||||
let socket: WebSocket
|
||||
let messages: RunnerToServerMessage[]
|
||||
let welcomeInFlight: string[]
|
||||
let connections: number
|
||||
let replacePrevious: boolean
|
||||
let connectRunner: (options?: { verbose?: boolean; log?: (line: string) => void }) => RunnerClient
|
||||
beforeEach(async () => {
|
||||
cwd = await mkdtemp(path.join(tmpdir(), 'shumai-client-test-'))
|
||||
messages = []
|
||||
welcomeInFlight = []
|
||||
connections = 0
|
||||
replacePrevious = false
|
||||
server = new WebSocketServer({ host: '127.0.0.1', port: 0 })
|
||||
await new Promise<void>((resolve) => server.once('listening', resolve))
|
||||
const address = server.address()
|
||||
if (!address || typeof address === 'string') throw new Error('No WebSocket address')
|
||||
server.on('connection', (connection, request) => {
|
||||
expect(request.headers.authorization).toBe('Bearer runner-key')
|
||||
if (replacePrevious) socket.close(4000, 'Replaced by a newer connection')
|
||||
socket = connection
|
||||
connections++
|
||||
connection.on('message', (raw) =>
|
||||
messages.push(runnerToServerMessageSchema.parse(JSON.parse(raw.toString()))),
|
||||
)
|
||||
connection.send(
|
||||
JSON.stringify({
|
||||
type: 'welcome',
|
||||
runnerId: 'runner',
|
||||
runnerName: 'Test',
|
||||
serverTime: Date.now(),
|
||||
inFlightExecutionIds: welcomeInFlight,
|
||||
}),
|
||||
)
|
||||
})
|
||||
connectRunner = (options) => {
|
||||
const files = new RunnerFiles(cwd)
|
||||
return new RunnerClient(
|
||||
{
|
||||
url: new URL(`ws://127.0.0.1:${address.port}/api/runners/ws`),
|
||||
key: 'runner-key',
|
||||
cwd,
|
||||
srt: undefined,
|
||||
verbose: options?.verbose ?? false,
|
||||
},
|
||||
files,
|
||||
new RunnerShell(files),
|
||||
options?.log ?? (() => {}),
|
||||
)
|
||||
}
|
||||
runner = connectRunner()
|
||||
running = runner.run()
|
||||
// Attach an error handler immediately, including for tests exercising terminal errors.
|
||||
void running.catch(() => {})
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'reconcile')).toBe(true),
|
||||
)
|
||||
})
|
||||
afterEach(async () => {
|
||||
runner.stop()
|
||||
await running.catch(() => {})
|
||||
for (const connection of server.clients) connection.terminate()
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()))
|
||||
await rm(cwd, { recursive: true, force: true })
|
||||
})
|
||||
const send = (message: unknown) => socket.send(JSON.stringify(message))
|
||||
|
||||
it('responds to heartbeat, executes a command once and streams byte offsets', async () => {
|
||||
expect(messages).toContainEqual(
|
||||
expect.objectContaining({
|
||||
type: 'reconcile',
|
||||
environment: { platform: process.platform, shell: '/bin/sh' },
|
||||
}),
|
||||
)
|
||||
send({ type: 'ping', ts: 123 })
|
||||
const command = {
|
||||
type: 'exec.start',
|
||||
executionId: 'one',
|
||||
command: 'printf "世界"; printf "error" >&2',
|
||||
}
|
||||
send(command)
|
||||
send(command)
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'exec.result')).toBe(true),
|
||||
)
|
||||
expect(messages).toContainEqual({ type: 'pong', ts: 123 })
|
||||
expect(messages.filter((message) => message.type === 'exec.result')).toHaveLength(1)
|
||||
const chunks = messages.filter((message) => message.type === 'exec.output')
|
||||
expect(chunks.map((chunk) => chunk.data).join('')).toContain('世界')
|
||||
let offset = 0
|
||||
for (const chunk of chunks) {
|
||||
expect(chunk.offset).toBe(offset)
|
||||
offset += Buffer.byteLength(chunk.data)
|
||||
}
|
||||
})
|
||||
|
||||
it('cancels an active command through the protocol', async () => {
|
||||
send({ type: 'exec.start', executionId: 'cancel', command: 'echo ready; sleep 30' })
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'exec.output')).toBe(true),
|
||||
)
|
||||
send({ type: 'exec.cancel', executionId: 'cancel' })
|
||||
await vi.waitFor(() =>
|
||||
expect(messages).toContainEqual(
|
||||
expect.objectContaining({ type: 'exec.result', executionId: 'cancel', aborted: true }),
|
||||
),
|
||||
)
|
||||
})
|
||||
|
||||
it('reconnects and replays output before completing an in-flight command', async () => {
|
||||
send({
|
||||
type: 'exec.start',
|
||||
executionId: 'reconnect',
|
||||
command: 'echo before; sleep 0.2; echo after',
|
||||
})
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'exec.output')).toBe(true),
|
||||
)
|
||||
welcomeInFlight = ['reconnect']
|
||||
socket.terminate()
|
||||
messages = []
|
||||
await vi.waitFor(() => expect(connections).toBe(2), { timeout: 5000 })
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'reconcile')).toBe(true),
|
||||
)
|
||||
const reconciliation = messages.findIndex((message) => message.type === 'reconcile')
|
||||
expect(
|
||||
messages
|
||||
.slice(0, reconciliation)
|
||||
.filter((message) => message.type === 'exec.output')
|
||||
.map((message) => message.data)
|
||||
.join(''),
|
||||
).toContain('after')
|
||||
expect(messages[reconciliation]).toMatchObject({
|
||||
completedExecutions: [expect.objectContaining({ executionId: 'reconnect', exitCode: 0 })],
|
||||
})
|
||||
})
|
||||
|
||||
it('handles file requests and returns a cached result for a duplicate request', async () => {
|
||||
const request = { type: 'file.stat', executionId: 'stat', srcPath: '.pi/missing' }
|
||||
send(request)
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'file.stat.result')).toBe(true),
|
||||
)
|
||||
send(request)
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.filter((message) => message.type === 'file.stat.result')).toHaveLength(2),
|
||||
)
|
||||
expect(messages.at(-1)).toMatchObject({ ok: true, exists: false })
|
||||
})
|
||||
|
||||
it('stops instead of reconnecting when the runner key is revoked', async () => {
|
||||
socket.close(1008, 'revoked')
|
||||
await expect(running).rejects.toThrow('revoked')
|
||||
expect(connections).toBe(1)
|
||||
})
|
||||
|
||||
it('stops the superseded client when a second runner connects with the same key', async () => {
|
||||
replacePrevious = true
|
||||
let replaced: unknown
|
||||
void running.catch((error: unknown) => {
|
||||
replaced = error
|
||||
})
|
||||
const replacement = connectRunner()
|
||||
const replacementRun = replacement.run()
|
||||
void replacementRun.catch(() => {})
|
||||
try {
|
||||
await vi.waitFor(() => expect(replaced).toBeInstanceOf(Error))
|
||||
await expect(running).rejects.toThrow('replaced')
|
||||
expect(connections).toBe(2)
|
||||
} finally {
|
||||
replacement.stop()
|
||||
await replacementRun.catch(() => {})
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects invalid messages and path-like execution IDs', async () => {
|
||||
send({ type: 'exec.start', executionId: '../escape', command: 'echo unsafe' })
|
||||
await expect(running).rejects.toThrow('invalid runner protocol')
|
||||
expect(messages.some((message) => message.type === 'exec.output')).toBe(false)
|
||||
})
|
||||
|
||||
it('logs compact lifecycle, bash, and transfer summaries in default mode', async () => {
|
||||
runner.stop()
|
||||
await running.catch(() => {})
|
||||
const logs: string[] = []
|
||||
messages = []
|
||||
runner = connectRunner({ verbose: false, log: (line) => logs.push(line) })
|
||||
running = runner.run()
|
||||
void running.catch(() => {})
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'reconcile')).toBe(true),
|
||||
)
|
||||
|
||||
send({ type: 'exec.start', executionId: '01JTEST0000A8F2', command: 'printf "hello"' })
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'exec.result')).toBe(true),
|
||||
)
|
||||
|
||||
send({
|
||||
type: 'exec.start',
|
||||
executionId: '01JTEST0000TOUT',
|
||||
command: 'sleep 5',
|
||||
timeoutSec: 0.1,
|
||||
})
|
||||
await vi.waitFor(() =>
|
||||
expect(
|
||||
messages.some(
|
||||
(message) => message.type === 'exec.result' && message.executionId === '01JTEST0000TOUT',
|
||||
),
|
||||
).toBe(true),
|
||||
)
|
||||
|
||||
send({ type: 'file.stat', executionId: '01JTEST0000STAT', srcPath: '.pi/missing' })
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'file.stat.result')).toBe(true),
|
||||
)
|
||||
|
||||
const httpServer = createServer((_request, response) => {
|
||||
response.end('payload')
|
||||
})
|
||||
await new Promise<void>((resolve) => httpServer.listen(0, '127.0.0.1', resolve))
|
||||
try {
|
||||
const httpAddress = httpServer.address()
|
||||
if (!httpAddress || typeof httpAddress === 'string') throw new Error('No HTTP address')
|
||||
send({
|
||||
type: 'file.download',
|
||||
executionId: '01JTEST0000C3D9',
|
||||
url: `http://127.0.0.1:${httpAddress.port}/file`,
|
||||
destPath: '.pi/assets/file.txt',
|
||||
})
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'file.download.result')).toBe(true),
|
||||
)
|
||||
} finally {
|
||||
httpServer.closeAllConnections()
|
||||
await new Promise<void>((resolve) => httpServer.close(() => resolve()))
|
||||
}
|
||||
|
||||
expect(
|
||||
logs.some((line) =>
|
||||
/^\d{2}:\d{2}:\d{2} INFO {2}Runner connected: Test \(runner\)$/.test(line),
|
||||
),
|
||||
).toBe(true)
|
||||
expect(
|
||||
logs.some((line) => /^\d{2}:\d{2}:\d{2} INFO {2}Bash started \[a8f2\]$/.test(line)),
|
||||
).toBe(true)
|
||||
expect(
|
||||
logs.some((line) =>
|
||||
/^\d{2}:\d{2}:\d{2} INFO {2}Bash completed \[a8f2\] exit=0 duration=\d+\.\ds$/.test(line),
|
||||
),
|
||||
).toBe(true)
|
||||
expect(
|
||||
logs.some((line) =>
|
||||
/^\d{2}:\d{2}:\d{2} WARN {2}Bash timed out \[tout\] duration=\d+\.\ds$/.test(line),
|
||||
),
|
||||
).toBe(true)
|
||||
expect(
|
||||
logs.some((line) =>
|
||||
/^\d{2}:\d{2}:\d{2} INFO {2}File download completed \[c3d9\] size=7B duration=\d+\.\ds$/.test(
|
||||
line,
|
||||
),
|
||||
),
|
||||
).toBe(true)
|
||||
expect(logs.join('\n')).not.toContain('command=')
|
||||
expect(logs.join('\n')).not.toContain('File stat')
|
||||
expect(logs.join('\n')).not.toContain('.pi/assets/file.txt')
|
||||
})
|
||||
|
||||
it('logs commands, stream sizes, and file paths in verbose mode', async () => {
|
||||
runner.stop()
|
||||
await running.catch(() => {})
|
||||
const logs: string[] = []
|
||||
messages = []
|
||||
runner = connectRunner({ verbose: true, log: (line) => logs.push(line) })
|
||||
running = runner.run()
|
||||
void running.catch(() => {})
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'reconcile')).toBe(true),
|
||||
)
|
||||
|
||||
send({
|
||||
type: 'exec.start',
|
||||
executionId: '01JTEST0000B9E1',
|
||||
command: 'printf "hi"; printf "err" >&2',
|
||||
})
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'exec.result')).toBe(true),
|
||||
)
|
||||
|
||||
send({ type: 'file.stat', executionId: '01JTEST0000B1A4', srcPath: '.pi/missing' })
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'file.stat.result')).toBe(true),
|
||||
)
|
||||
|
||||
const httpServer = createServer((_request, response) => {
|
||||
response.writeHead(500)
|
||||
response.end()
|
||||
})
|
||||
await new Promise<void>((resolve) => httpServer.listen(0, '127.0.0.1', resolve))
|
||||
try {
|
||||
const httpAddress = httpServer.address()
|
||||
if (!httpAddress || typeof httpAddress === 'string') throw new Error('No HTTP address')
|
||||
send({
|
||||
type: 'file.download',
|
||||
executionId: '01JTEST0000E5F1',
|
||||
url: `http://127.0.0.1:${httpAddress.port}/fail`,
|
||||
destPath: '.pi/assets/fail.txt',
|
||||
})
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'file.download.result')).toBe(true),
|
||||
)
|
||||
} finally {
|
||||
httpServer.closeAllConnections()
|
||||
await new Promise<void>((resolve) => httpServer.close(() => resolve()))
|
||||
}
|
||||
|
||||
expect(
|
||||
logs.some((line) =>
|
||||
/^\d{2}:\d{2}:\d{2} INFO {2}Bash started \[b9e1\] command="printf \\"hi\\"; printf \\"err\\" >&2"$/.test(
|
||||
line,
|
||||
),
|
||||
),
|
||||
).toBe(true)
|
||||
expect(
|
||||
logs.some((line) =>
|
||||
/^\d{2}:\d{2}:\d{2} INFO {2}Bash completed \[b9e1\] exit=0 duration=\d+\.\ds stdout=2B stderr=3B$/.test(
|
||||
line,
|
||||
),
|
||||
),
|
||||
).toBe(true)
|
||||
expect(
|
||||
logs.some((line) =>
|
||||
/^\d{2}:\d{2}:\d{2} INFO {2}File stat \[b1a4\] path="\.pi\/missing" exists=false$/.test(
|
||||
line,
|
||||
),
|
||||
),
|
||||
).toBe(true)
|
||||
expect(
|
||||
logs.some((line) =>
|
||||
/^\d{2}:\d{2}:\d{2} INFO {2}File download started \[e5f1\] path="\.pi\/assets\/fail\.txt"$/.test(
|
||||
line,
|
||||
),
|
||||
),
|
||||
).toBe(true)
|
||||
expect(
|
||||
logs.some((line) =>
|
||||
/^\d{2}:\d{2}:\d{2} WARN {2}File download failed \[e5f1\] path="\.pi\/assets\/fail\.txt" duration=\d+\.\ds error="Download failed \(HTTP 500\)\."$/.test(
|
||||
line,
|
||||
),
|
||||
),
|
||||
).toBe(true)
|
||||
})
|
||||
|
||||
it.each([false, true])(
|
||||
'logs a single busy rejection without false starts at capacity (verbose=%s)',
|
||||
async (verbose) => {
|
||||
runner.stop()
|
||||
await running.catch(() => {})
|
||||
const logs: string[] = []
|
||||
messages = []
|
||||
runner = connectRunner({ verbose, log: (line) => logs.push(line) })
|
||||
running = runner.run()
|
||||
void running.catch(() => {})
|
||||
await vi.waitFor(() =>
|
||||
expect(messages.some((message) => message.type === 'reconcile')).toBe(true),
|
||||
)
|
||||
|
||||
for (let index = 0; index < 8; index++) {
|
||||
send({
|
||||
type: 'exec.start',
|
||||
executionId: `01JACTIVE00000${index}`,
|
||||
command: 'sleep 30',
|
||||
})
|
||||
}
|
||||
await vi.waitFor(() =>
|
||||
expect(logs.filter((line) => line.includes('Bash started'))).toHaveLength(8),
|
||||
)
|
||||
|
||||
send({
|
||||
type: 'exec.start',
|
||||
executionId: '01JBUSYCMD00B123',
|
||||
command: 'echo rejected',
|
||||
})
|
||||
send({
|
||||
type: 'file.upload',
|
||||
executionId: '01JBUSYFILE0U456',
|
||||
srcPath: '.pi/out.bin',
|
||||
url: 'http://127.0.0.1:9/upload',
|
||||
})
|
||||
|
||||
await vi.waitFor(() => {
|
||||
expect(messages).toContainEqual(
|
||||
expect.objectContaining({
|
||||
type: 'exec.result',
|
||||
executionId: '01JBUSYCMD00B123',
|
||||
error: 'Runner is busy. Try again later.',
|
||||
}),
|
||||
)
|
||||
expect(messages).toContainEqual(
|
||||
expect.objectContaining({
|
||||
type: 'file.upload.result',
|
||||
executionId: '01JBUSYFILE0U456',
|
||||
ok: false,
|
||||
error: 'Runner is busy. Try again later.',
|
||||
}),
|
||||
)
|
||||
})
|
||||
|
||||
const bashLogs = logs.filter((line) => line.includes('[b123]'))
|
||||
const uploadLogs = logs.filter((line) => line.includes('[u456]'))
|
||||
expect(bashLogs).toHaveLength(1)
|
||||
expect(uploadLogs).toHaveLength(1)
|
||||
if (verbose) {
|
||||
expect(bashLogs[0]).toMatch(
|
||||
/^\d{2}:\d{2}:\d{2} WARN {2}Bash rejected \[b123\] command="echo rejected" error="Runner is busy\. Try again later\."$/,
|
||||
)
|
||||
expect(uploadLogs[0]).toMatch(
|
||||
/^\d{2}:\d{2}:\d{2} WARN {2}File upload rejected \[u456\] path="\.pi\/out\.bin" error="Runner is busy\. Try again later\."$/,
|
||||
)
|
||||
} else {
|
||||
expect(bashLogs[0]).toMatch(
|
||||
/^\d{2}:\d{2}:\d{2} WARN {2}Bash rejected \[b123\] \(runner is busy\)$/,
|
||||
)
|
||||
expect(uploadLogs[0]).toMatch(
|
||||
/^\d{2}:\d{2}:\d{2} WARN {2}File upload rejected \[u456\] \(runner is busy\)$/,
|
||||
)
|
||||
}
|
||||
},
|
||||
)
|
||||
})
|
||||
@@ -0,0 +1,461 @@
|
||||
import WebSocket from 'ws'
|
||||
import { setTimeout as delay } from 'node:timers/promises'
|
||||
import {
|
||||
RUNNER_REPLACED_CLOSE_CODE,
|
||||
serverToRunnerMessageSchema,
|
||||
type RunnerExecOutputMessage,
|
||||
type RunnerExecResultMessage,
|
||||
type RunnerToServerMessage,
|
||||
type ServerToRunnerMessage,
|
||||
} from '@shumai/dtos'
|
||||
import { CliError } from '../operations/errors'
|
||||
import { resolveRunnerConfig, type RunnerConfig, type RunnerOptions } from './config'
|
||||
import { RunnerFiles, type FileMessage } from './files'
|
||||
import {
|
||||
createRunnerLogger,
|
||||
formatBytes,
|
||||
formatDuration,
|
||||
formatLogString,
|
||||
shortId,
|
||||
type RunnerLogger,
|
||||
} from './log'
|
||||
import { RunnerShell } from './shell'
|
||||
|
||||
const MAX_REPLAY_BYTES = 1024 * 1024
|
||||
const MAX_COMPLETED = 16
|
||||
const MAX_ACTIVE = 8
|
||||
const RUNNER_BUSY_ERROR = 'Runner is busy. Try again later.'
|
||||
const FILE_OPERATION_KINDS: Record<FileMessage['type'], 'download' | 'upload' | 'stat'> = {
|
||||
'file.download': 'download',
|
||||
'file.upload': 'upload',
|
||||
'file.stat': 'stat',
|
||||
}
|
||||
|
||||
interface Execution {
|
||||
controller: AbortController
|
||||
chunks: RunnerExecOutputMessage[]
|
||||
bytes: number
|
||||
offset: number
|
||||
startedAt: number
|
||||
stdoutBytes: number
|
||||
stderrBytes: number
|
||||
result?: RunnerExecResultMessage
|
||||
}
|
||||
|
||||
export class RunnerClient {
|
||||
private socket?: WebSocket
|
||||
private ready = false
|
||||
private stopped = new AbortController()
|
||||
private executions = new Map<string, Execution>()
|
||||
private transfers = new Map<string, AbortController>()
|
||||
private completedFiles = new Map<string, RunnerToServerMessage>()
|
||||
private pending = new Set<Promise<void>>()
|
||||
private readonly logger: RunnerLogger
|
||||
|
||||
constructor(
|
||||
private readonly config: RunnerConfig,
|
||||
private readonly files: RunnerFiles,
|
||||
private readonly shell: RunnerShell,
|
||||
log: (message: string) => void = console.error,
|
||||
) {
|
||||
this.logger = createRunnerLogger(log, Boolean(config.verbose))
|
||||
}
|
||||
|
||||
async run(signal?: AbortSignal): Promise<void> {
|
||||
const stop = () => this.stop()
|
||||
signal?.addEventListener('abort', stop, { once: true })
|
||||
if (signal?.aborted) this.stop()
|
||||
let attempt = 0
|
||||
try {
|
||||
while (!this.stopped.signal.aborted) {
|
||||
const wasReady = await this.connect()
|
||||
if (this.stopped.signal.aborted) break
|
||||
attempt = wasReady ? 0 : Math.min(attempt + 1, 5)
|
||||
this.logger.warn('Runner disconnected; reconnecting…')
|
||||
await delay(Math.min(1000 * 2 ** attempt, 30_000), undefined, {
|
||||
signal: this.stopped.signal,
|
||||
}).catch(() => {})
|
||||
}
|
||||
} finally {
|
||||
signal?.removeEventListener('abort', stop)
|
||||
this.stop()
|
||||
await Promise.allSettled(this.pending)
|
||||
}
|
||||
}
|
||||
|
||||
stop() {
|
||||
this.stopped.abort()
|
||||
this.ready = false
|
||||
this.socket?.terminate()
|
||||
for (const execution of this.executions.values()) execution.controller.abort()
|
||||
for (const transfer of this.transfers.values()) transfer.abort()
|
||||
}
|
||||
|
||||
private connect(): Promise<boolean> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const socket = new WebSocket(this.config.url, {
|
||||
headers: { authorization: `Bearer ${this.config.key}` },
|
||||
handshakeTimeout: 10_000,
|
||||
maxPayload: 1024 * 1024,
|
||||
perMessageDeflate: false,
|
||||
followRedirects: false,
|
||||
})
|
||||
this.socket = socket
|
||||
let welcomed = false
|
||||
let lastMessage = Date.now()
|
||||
const heartbeat = setInterval(() => {
|
||||
if (Date.now() - lastMessage > (welcomed ? 75_000 : 10_000)) socket.terminate()
|
||||
}, 5_000)
|
||||
socket.on('error', () => {
|
||||
/* Close drives reconnect; never print credential-bearing request errors. */
|
||||
})
|
||||
socket.on('unexpected-response', (_request, response) => {
|
||||
response.destroy()
|
||||
socket.terminate()
|
||||
if ([401, 403, 404, 426].includes(response.statusCode ?? 0))
|
||||
reject(
|
||||
new CliError(
|
||||
`Runner connection rejected (HTTP ${response.statusCode}). Check the server URL and runner key.`,
|
||||
),
|
||||
)
|
||||
})
|
||||
socket.on('close', (code) => {
|
||||
clearInterval(heartbeat)
|
||||
this.ready = false
|
||||
if ((code === 1008 || code === RUNNER_REPLACED_CLOSE_CODE) && !this.stopped.signal.aborted)
|
||||
reject(
|
||||
new CliError('Runner access was revoked or another connection replaced this runner.'),
|
||||
)
|
||||
else resolve(welcomed)
|
||||
})
|
||||
socket.on('message', (raw) => {
|
||||
try {
|
||||
const parsed = serverToRunnerMessageSchema.parse(JSON.parse(raw.toString()))
|
||||
if ('executionId' in parsed && !/^[A-Za-z0-9_-]{1,128}$/.test(parsed.executionId))
|
||||
throw new Error('Invalid execution ID.')
|
||||
lastMessage = Date.now()
|
||||
if (parsed.type === 'welcome') {
|
||||
if (welcomed) throw new Error('Duplicate welcome.')
|
||||
welcomed = true
|
||||
this.ready = true
|
||||
this.reconcile(parsed.inFlightExecutionIds)
|
||||
this.logger.info(`Runner connected: ${parsed.runnerName} (${parsed.runnerId})`)
|
||||
} else {
|
||||
if (!welcomed) throw new Error('Expected welcome before runner requests.')
|
||||
this.handle(parsed)
|
||||
}
|
||||
} catch {
|
||||
reject(new CliError('Server sent an invalid runner protocol message.'))
|
||||
socket.terminate()
|
||||
}
|
||||
})
|
||||
if (this.stopped.signal.aborted) socket.terminate()
|
||||
})
|
||||
}
|
||||
|
||||
private send(message: RunnerToServerMessage) {
|
||||
if (this.ready && this.socket?.readyState === WebSocket.OPEN)
|
||||
this.socket.send(JSON.stringify(message))
|
||||
}
|
||||
|
||||
private reconcile(inFlight: string[]) {
|
||||
const expected = new Set(inFlight)
|
||||
for (const [id, execution] of this.executions) {
|
||||
if (!expected.has(id)) {
|
||||
execution.controller.abort()
|
||||
if (execution.result) this.executions.delete(id)
|
||||
continue
|
||||
}
|
||||
// Replay before results: the server deduplicates chunks by UTF-8 byte offset.
|
||||
for (const chunk of execution.chunks) this.send(chunk)
|
||||
}
|
||||
this.send({
|
||||
type: 'reconcile',
|
||||
environment: this.shell.environment,
|
||||
activeExecutions: [...this.executions]
|
||||
.filter(([id, execution]) => expected.has(id) && !execution.result)
|
||||
.map(([executionId, execution]) => ({ executionId, nextOffset: execution.offset })),
|
||||
completedExecutions: [...this.executions]
|
||||
.filter(([id, execution]) => expected.has(id) && execution.result)
|
||||
.map(([, execution]) => execution.result!),
|
||||
})
|
||||
}
|
||||
|
||||
private handle(message: Exclude<ServerToRunnerMessage, { type: 'welcome' }>) {
|
||||
if (message.type === 'ping') {
|
||||
this.send({ type: 'pong', ts: message.ts })
|
||||
return
|
||||
}
|
||||
if (message.type === 'exec.cancel') {
|
||||
this.executions.get(message.executionId)?.controller.abort()
|
||||
return
|
||||
}
|
||||
if (message.type === 'exec.start') {
|
||||
const previous = this.executions.get(message.executionId)
|
||||
if (previous) {
|
||||
if (previous.result) this.send(previous.result)
|
||||
return
|
||||
}
|
||||
const id = shortId(message.executionId)
|
||||
if (this.activeCount() >= MAX_ACTIVE) {
|
||||
const result: RunnerExecResultMessage = {
|
||||
type: 'exec.result',
|
||||
executionId: message.executionId,
|
||||
exitCode: null,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
error: RUNNER_BUSY_ERROR,
|
||||
}
|
||||
this.recordCompletedExecution(message.executionId, result)
|
||||
this.logger.warn(
|
||||
this.logger.verbose
|
||||
? `Bash rejected [${id}] command=${formatLogString(message.command)} error=${formatLogString(RUNNER_BUSY_ERROR)}`
|
||||
: `Bash rejected [${id}] (runner is busy)`,
|
||||
)
|
||||
this.send(result)
|
||||
return
|
||||
}
|
||||
const execution: Execution = {
|
||||
controller: new AbortController(),
|
||||
chunks: [],
|
||||
bytes: 0,
|
||||
offset: 0,
|
||||
startedAt: performance.now(),
|
||||
stdoutBytes: 0,
|
||||
stderrBytes: 0,
|
||||
}
|
||||
this.executions.set(message.executionId, execution)
|
||||
this.logger.info(
|
||||
this.logger.verbose
|
||||
? `Bash started [${id}] command=${formatLogString(message.command)}`
|
||||
: `Bash started [${id}]`,
|
||||
)
|
||||
this.track(
|
||||
(async () => {
|
||||
const result = await this.shell.run(
|
||||
message,
|
||||
execution.controller.signal,
|
||||
(stream, data) => {
|
||||
// Small frames keep reconnect replay and WebSocket buffering bounded.
|
||||
for (const part of data.match(/[\s\S]{1,8192}/gu) ?? []) {
|
||||
const chunk: RunnerExecOutputMessage = {
|
||||
type: 'exec.output',
|
||||
executionId: message.executionId,
|
||||
stream,
|
||||
data: part,
|
||||
offset: execution.offset,
|
||||
}
|
||||
const bytes = Buffer.byteLength(part)
|
||||
execution.offset += bytes
|
||||
execution.bytes += bytes
|
||||
if (stream === 'stdout') execution.stdoutBytes += bytes
|
||||
else execution.stderrBytes += bytes
|
||||
execution.chunks.push(chunk)
|
||||
while (execution.bytes > MAX_REPLAY_BYTES) {
|
||||
const removed = execution.chunks.shift()!
|
||||
execution.bytes -= Buffer.byteLength(removed.data)
|
||||
if (!this.ready) execution.controller.abort()
|
||||
}
|
||||
if ((this.socket?.bufferedAmount ?? 0) > MAX_REPLAY_BYTES) {
|
||||
this.ready = false
|
||||
this.socket?.terminate()
|
||||
}
|
||||
this.send(chunk)
|
||||
}
|
||||
},
|
||||
)
|
||||
execution.result = result
|
||||
this.logExecResult(id, execution, result)
|
||||
this.send(result)
|
||||
this.pruneCompletedExecutions()
|
||||
})(),
|
||||
)
|
||||
return
|
||||
}
|
||||
this.handleFile(message)
|
||||
}
|
||||
|
||||
private recordCompletedExecution(executionId: string, result: RunnerExecResultMessage) {
|
||||
this.executions.set(executionId, {
|
||||
controller: new AbortController(),
|
||||
chunks: [],
|
||||
bytes: 0,
|
||||
offset: 0,
|
||||
startedAt: performance.now(),
|
||||
stdoutBytes: 0,
|
||||
stderrBytes: 0,
|
||||
result,
|
||||
})
|
||||
this.pruneCompletedExecutions()
|
||||
}
|
||||
|
||||
private pruneCompletedExecutions() {
|
||||
const completed = [...this.executions].filter(([, value]) => value.result)
|
||||
for (const [completedId] of completed.slice(0, -MAX_COMPLETED))
|
||||
this.executions.delete(completedId)
|
||||
}
|
||||
|
||||
private logExecResult(id: string, execution: Execution, result: RunnerExecResultMessage) {
|
||||
const duration = formatDuration(performance.now() - execution.startedAt)
|
||||
const sizes = this.logger.verbose
|
||||
? ` stdout=${formatBytes(execution.stdoutBytes)} stderr=${formatBytes(execution.stderrBytes)}`
|
||||
: ''
|
||||
if (result.timedOut) {
|
||||
const error =
|
||||
this.logger.verbose && result.error ? ` error=${formatLogString(result.error)}` : ''
|
||||
this.logger.warn(`Bash timed out [${id}] duration=${duration}${sizes}${error}`)
|
||||
return
|
||||
}
|
||||
if (result.aborted) {
|
||||
this.logger.info(`Bash cancelled [${id}] duration=${duration}${sizes}`)
|
||||
return
|
||||
}
|
||||
if (result.error) {
|
||||
const error = this.logger.verbose ? ` error=${formatLogString(result.error)}` : ''
|
||||
this.logger.warn(`Bash failed [${id}] duration=${duration}${sizes}${error}`)
|
||||
return
|
||||
}
|
||||
if (result.exitCode === null && result.signal) {
|
||||
this.logger.warn(`Bash killed [${id}] signal=${result.signal} duration=${duration}${sizes}`)
|
||||
return
|
||||
}
|
||||
this.logger.info(`Bash completed [${id}] exit=${result.exitCode} duration=${duration}${sizes}`)
|
||||
}
|
||||
|
||||
private activeCount() {
|
||||
return (
|
||||
[...this.executions.values()].filter((execution) => !execution.result).length +
|
||||
this.transfers.size
|
||||
)
|
||||
}
|
||||
|
||||
private handleFile(message: FileMessage) {
|
||||
const previous = this.completedFiles.get(message.executionId)
|
||||
if (previous) {
|
||||
this.send(previous)
|
||||
return
|
||||
}
|
||||
if (this.transfers.has(message.executionId)) return
|
||||
const id = shortId(message.executionId)
|
||||
const transferPath = message.type === 'file.download' ? message.destPath : message.srcPath
|
||||
if (this.activeCount() >= MAX_ACTIVE) {
|
||||
const result = this.fileBusy(message)
|
||||
this.recordCompletedFile(message.executionId, result)
|
||||
const kind = FILE_OPERATION_KINDS[message.type]
|
||||
this.logger.warn(
|
||||
this.logger.verbose
|
||||
? `File ${kind} rejected [${id}] path=${formatLogString(transferPath)} error=${formatLogString(RUNNER_BUSY_ERROR)}`
|
||||
: `File ${kind} rejected [${id}] (runner is busy)`,
|
||||
)
|
||||
this.send(result)
|
||||
return
|
||||
}
|
||||
const controller = new AbortController()
|
||||
this.transfers.set(message.executionId, controller)
|
||||
const timer = setTimeout(() => controller.abort(), 120_000)
|
||||
const startedAt = performance.now()
|
||||
if (this.logger.verbose && message.type !== 'file.stat') {
|
||||
const kind = message.type === 'file.download' ? 'download' : 'upload'
|
||||
this.logger.info(`File ${kind} started [${id}] path=${formatLogString(transferPath)}`)
|
||||
}
|
||||
this.track(
|
||||
(async () => {
|
||||
try {
|
||||
const result = await this.files.handle(message, controller.signal)
|
||||
this.recordCompletedFile(message.executionId, result)
|
||||
this.logFileResult(id, transferPath, startedAt, result)
|
||||
this.send(result)
|
||||
} finally {
|
||||
clearTimeout(timer)
|
||||
this.transfers.delete(message.executionId)
|
||||
}
|
||||
})(),
|
||||
)
|
||||
}
|
||||
|
||||
private recordCompletedFile(executionId: string, result: RunnerToServerMessage) {
|
||||
this.completedFiles.set(executionId, result)
|
||||
if (this.completedFiles.size > MAX_COMPLETED)
|
||||
this.completedFiles.delete(this.completedFiles.keys().next().value!)
|
||||
}
|
||||
|
||||
private logFileResult(
|
||||
id: string,
|
||||
transferPath: string,
|
||||
startedAt: number,
|
||||
result: RunnerToServerMessage,
|
||||
) {
|
||||
if (result.type === 'file.stat.result') {
|
||||
if (!this.logger.verbose) return
|
||||
const pathPart = `path=${formatLogString(transferPath)}`
|
||||
if (!result.ok) {
|
||||
this.logger.warn(
|
||||
`File stat failed [${id}] ${pathPart} error=${formatLogString(result.error ?? 'Unknown error')}`,
|
||||
)
|
||||
return
|
||||
}
|
||||
const sizePart = result.exists ? ` size=${formatBytes(result.size ?? 0)}` : ''
|
||||
this.logger.info(`File stat [${id}] ${pathPart} exists=${result.exists}${sizePart}`)
|
||||
return
|
||||
}
|
||||
if (result.type !== 'file.download.result' && result.type !== 'file.upload.result') return
|
||||
const kind = result.type === 'file.download.result' ? 'download' : 'upload'
|
||||
const duration = formatDuration(performance.now() - startedAt)
|
||||
const pathPart = this.logger.verbose ? ` path=${formatLogString(transferPath)}` : ''
|
||||
if (result.ok) {
|
||||
this.logger.info(
|
||||
`File ${kind} completed [${id}]${pathPart} size=${formatBytes(result.size ?? 0)} duration=${duration}`,
|
||||
)
|
||||
return
|
||||
}
|
||||
const errorPart = this.logger.verbose
|
||||
? ` error=${formatLogString(result.error ?? 'Unknown error')}`
|
||||
: ''
|
||||
this.logger.warn(`File ${kind} failed [${id}]${pathPart} duration=${duration}${errorPart}`)
|
||||
}
|
||||
|
||||
private fileBusy(message: FileMessage): RunnerToServerMessage {
|
||||
const result = {
|
||||
executionId: message.executionId,
|
||||
ok: false,
|
||||
error: RUNNER_BUSY_ERROR,
|
||||
}
|
||||
switch (message.type) {
|
||||
case 'file.download':
|
||||
return { ...result, type: 'file.download.result' }
|
||||
case 'file.upload':
|
||||
return { ...result, type: 'file.upload.result' }
|
||||
case 'file.stat':
|
||||
return { ...result, type: 'file.stat.result', exists: false }
|
||||
}
|
||||
}
|
||||
|
||||
private track(task: Promise<void>) {
|
||||
this.pending.add(task)
|
||||
void task.finally(() => this.pending.delete(task)).catch(() => this.stop())
|
||||
}
|
||||
}
|
||||
|
||||
export async function startRunner(options: RunnerOptions) {
|
||||
const config = await resolveRunnerConfig(options)
|
||||
const files = new RunnerFiles(config.cwd)
|
||||
const shell = new RunnerShell(files, config.srt)
|
||||
// SRT resolves relative policy paths against the runner workspace.
|
||||
process.chdir(config.cwd)
|
||||
await shell.initialize()
|
||||
createRunnerLogger(console.error, Boolean(config.verbose)).info(
|
||||
`Runner shell: ${shell.environment.shell}`,
|
||||
)
|
||||
const runner = new RunnerClient(config, files, shell)
|
||||
const controller = new AbortController()
|
||||
const shutdown = () => controller.abort()
|
||||
process.once('SIGINT', shutdown)
|
||||
process.once('SIGTERM', shutdown)
|
||||
try {
|
||||
await runner.run(controller.signal)
|
||||
} finally {
|
||||
process.removeListener('SIGINT', shutdown)
|
||||
process.removeListener('SIGTERM', shutdown)
|
||||
await shell.close()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { commandEnv, resolveRunnerConfig } from './config'
|
||||
|
||||
afterEach(() => vi.unstubAllEnvs())
|
||||
|
||||
describe('runner configuration', () => {
|
||||
it('preserves Windows OS settings, normalizes explicit overrides, and excludes credentials', () => {
|
||||
vi.stubEnv('Path', 'C:\\Windows\\System32')
|
||||
vi.stubEnv('SystemRoot', 'C:\\Windows')
|
||||
vi.stubEnv('USERPROFILE', 'C:\\Users\\runner')
|
||||
const env = commandEnv({ PATH: 'C:\\tools', ['shumai_runner_key']: 'secret' })
|
||||
expect(env).toMatchObject({
|
||||
PATH: 'C:\\tools',
|
||||
SystemRoot: 'C:\\Windows',
|
||||
USERPROFILE: 'C:\\Users\\runner',
|
||||
})
|
||||
expect(env).not.toHaveProperty('Path')
|
||||
expect(env).not.toHaveProperty('shumai_runner_key')
|
||||
})
|
||||
it('uses a dedicated runner key and preserves a server path prefix', async () => {
|
||||
vi.stubEnv('SHUMAI_API_SERVER', 'https://example.com/shumai/')
|
||||
vi.stubEnv('SHUMAI_RUNNER_KEY', 'runner-secret')
|
||||
const config = await resolveRunnerConfig({})
|
||||
expect(config.url.href).toBe('wss://example.com/shumai/api/runners/ws')
|
||||
expect(config.key).toBe('runner-secret')
|
||||
expect(config.cwd).toBe(process.cwd())
|
||||
expect(config.verbose).toBe(false)
|
||||
expect((await resolveRunnerConfig({ verbose: true })).verbose).toBe(true)
|
||||
})
|
||||
|
||||
it.each([
|
||||
'http://example.com',
|
||||
'ftp://localhost',
|
||||
'https://user:secret@example.com',
|
||||
'https://example.com/?key=secret',
|
||||
'not a URL',
|
||||
])('rejects unsafe server URL %s', async (url) => {
|
||||
await expect(resolveRunnerConfig({ url, key: 'key' })).rejects.toThrow()
|
||||
})
|
||||
|
||||
it('allows the Docker host development address', async () => {
|
||||
expect(
|
||||
(await resolveRunnerConfig({ url: 'http://host.docker.internal:3000', key: 'key' })).url.href,
|
||||
).toBe('ws://host.docker.internal:3000/api/runners/ws')
|
||||
})
|
||||
|
||||
it('does not accept a personal API token in place of a missing runner key', async () => {
|
||||
vi.stubEnv('SHUMAI_RUNNER_KEY', '')
|
||||
vi.stubEnv('SHUMAI_API_KEY', 'personal-secret')
|
||||
await expect(resolveRunnerConfig({ url: 'https://example.com' })).rejects.toThrow(
|
||||
'SHUMAI_RUNNER_KEY',
|
||||
)
|
||||
})
|
||||
|
||||
it('inherits only baseline environment and explicit skill variables', () => {
|
||||
vi.stubEnv('SHUMAI_RUNNER_KEY', 'runner-secret')
|
||||
vi.stubEnv('SHUMAI_API_KEY', 'personal-secret')
|
||||
vi.stubEnv('AWS_SECRET_ACCESS_KEY', 'aws-secret')
|
||||
vi.stubEnv('BASH_ENV', '/malicious-startup')
|
||||
expect(
|
||||
commandEnv({ SKILL_TOKEN: 'skill-secret', SHUMAI_RUNNER_KEY: 'injected' }),
|
||||
).toMatchObject({ SKILL_TOKEN: 'skill-secret' })
|
||||
for (const name of ['SHUMAI_RUNNER_KEY', 'SHUMAI_API_KEY', 'AWS_SECRET_ACCESS_KEY', 'BASH_ENV'])
|
||||
expect(commandEnv()).not.toHaveProperty(name)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,84 @@
|
||||
import { realpath, stat } from 'node:fs/promises'
|
||||
import path from 'node:path'
|
||||
import { CliError } from '../operations/errors'
|
||||
|
||||
export interface RunnerOptions {
|
||||
url?: string
|
||||
key?: string
|
||||
cwd?: string
|
||||
srt?: string
|
||||
verbose?: boolean
|
||||
}
|
||||
|
||||
export interface RunnerConfig {
|
||||
url: URL
|
||||
key: string
|
||||
cwd: string
|
||||
srt?: string
|
||||
verbose?: boolean
|
||||
}
|
||||
|
||||
export async function resolveRunnerConfig(options: RunnerOptions): Promise<RunnerConfig> {
|
||||
const address = options.url ?? process.env.SHUMAI_API_SERVER
|
||||
const key = options.key ?? process.env.SHUMAI_RUNNER_KEY
|
||||
if (!address) throw new CliError('Set SHUMAI_API_SERVER or pass --url.')
|
||||
if (!key?.trim()) throw new CliError('Set SHUMAI_RUNNER_KEY or pass --key with a runner key.')
|
||||
let url: URL
|
||||
try {
|
||||
url = new URL(address)
|
||||
} catch {
|
||||
throw new CliError('Invalid Shumai server URL.')
|
||||
}
|
||||
if (
|
||||
url.username ||
|
||||
url.password ||
|
||||
url.search ||
|
||||
url.hash ||
|
||||
!['http:', 'https:'].includes(url.protocol)
|
||||
) {
|
||||
throw new CliError(
|
||||
'Use an http:// or https:// server URL without credentials, query, or fragment.',
|
||||
)
|
||||
}
|
||||
const localHosts = ['localhost', '127.0.0.1', '[::1]', 'host.docker.internal']
|
||||
if (url.protocol === 'http:' && !localHosts.includes(url.hostname)) {
|
||||
throw new CliError(
|
||||
'Remote runners require HTTPS. HTTP is only supported for local development.',
|
||||
)
|
||||
}
|
||||
url.protocol = url.protocol === 'https:' ? 'wss:' : 'ws:'
|
||||
url.pathname = `${url.pathname.replace(/\/+$/, '')}/api/runners/ws`
|
||||
const cwd = await realpath(path.resolve(options.cwd ?? process.cwd()))
|
||||
if (!(await stat(cwd)).isDirectory()) throw new CliError('Runner workspace must be a directory.')
|
||||
return {
|
||||
url,
|
||||
key: key.trim(),
|
||||
cwd,
|
||||
srt: options.srt ? path.resolve(options.srt) : undefined,
|
||||
verbose: Boolean(options.verbose),
|
||||
}
|
||||
}
|
||||
|
||||
/** Commands inherit baseline OS settings and explicitly supplied skill variables, never CLI credentials. */
|
||||
export function commandEnv(extra: Record<string, string> = {}, inherit = true): NodeJS.ProcessEnv {
|
||||
const env: NodeJS.ProcessEnv = {}
|
||||
for (const [key, value] of Object.entries(inherit ? process.env : {})) {
|
||||
if (
|
||||
/^(PATH|PATHEXT|HOME|USER|LOGNAME|LANG|TERM|TMPDIR|TEMP|TMP|SYSTEMROOT|SYSTEMDRIVE|COMSPEC|USERPROFILE|USERNAME|APPDATA|LOCALAPPDATA|PROGRAMDATA|PROGRAMFILES|PROGRAMFILES\(X86\)|LC_.*)$/i.test(
|
||||
key,
|
||||
)
|
||||
)
|
||||
env[key] = value
|
||||
}
|
||||
for (const [key, value] of Object.entries(extra)) {
|
||||
if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(key))
|
||||
throw new Error('Invalid skill environment variable name.')
|
||||
if (['SHUMAI_RUNNER_KEY', 'SHUMAI_API_KEY'].includes(key.toUpperCase())) continue
|
||||
// Windows treats environment names as case-insensitive.
|
||||
for (const existing of Object.keys(env)) {
|
||||
if (existing.toUpperCase() === key.toUpperCase()) delete env[existing]
|
||||
}
|
||||
env[key] = value
|
||||
}
|
||||
return env
|
||||
}
|
||||
@@ -0,0 +1,364 @@
|
||||
import { createHash } from 'node:crypto'
|
||||
import { createServer, type Server } from 'node:http'
|
||||
import { mkdtemp, mkdir, readFile, readdir, rm, stat, symlink, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import path from 'node:path'
|
||||
import AdmZip from 'adm-zip'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import yauzl from 'yauzl'
|
||||
import { RunnerFiles } from './files'
|
||||
|
||||
describe('runner workspace file transfers', () => {
|
||||
let cwd: string
|
||||
let files: RunnerFiles
|
||||
let server: Server
|
||||
let url: string
|
||||
let body: Buffer
|
||||
let uploaded: Buffer
|
||||
beforeEach(async () => {
|
||||
cwd = await mkdtemp(path.join(tmpdir(), 'shumai-files-test-'))
|
||||
files = new RunnerFiles(cwd)
|
||||
body = Buffer.from('download contents')
|
||||
uploaded = Buffer.alloc(0)
|
||||
server = createServer(async (request, response) => {
|
||||
if (request.method === 'PUT') {
|
||||
const chunks: Buffer[] = []
|
||||
for await (const chunk of request) chunks.push(Buffer.from(chunk))
|
||||
uploaded = Buffer.concat(chunks)
|
||||
response.end()
|
||||
} else if (request.url === '/slow') {
|
||||
const timer = setInterval(() => response.write(Buffer.alloc(65536)), 20)
|
||||
response.once('close', () => clearInterval(timer))
|
||||
} else if (request.url === '/redirect') {
|
||||
response.writeHead(302, { location: '/file' })
|
||||
response.end()
|
||||
} else response.end(body)
|
||||
})
|
||||
await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve))
|
||||
const address = server.address()
|
||||
if (!address || typeof address === 'string') throw new Error('Missing test HTTP address')
|
||||
url = `http://127.0.0.1:${address.port}`
|
||||
})
|
||||
afterEach(async () => {
|
||||
server.closeAllConnections()
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()))
|
||||
await rm(cwd, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
const signal = () => new AbortController().signal
|
||||
|
||||
it('downloads, checks SHA-256, stats and uploads actual bytes', async () => {
|
||||
expect(
|
||||
await files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: 'download',
|
||||
url,
|
||||
destPath: '.pi/input/data.bin',
|
||||
expectedHash: createHash('sha256').update(body).digest('hex'),
|
||||
},
|
||||
signal(),
|
||||
),
|
||||
).toMatchObject({ ok: true, size: body.length })
|
||||
expect(await readFile(path.join(cwd, '.pi/input/data.bin'))).toEqual(body)
|
||||
expect(
|
||||
await files.handle(
|
||||
{ type: 'file.stat', executionId: 'stat', srcPath: '.pi/input/data.bin' },
|
||||
signal(),
|
||||
),
|
||||
).toMatchObject({ ok: true, exists: true, isFile: true, size: body.length })
|
||||
expect(
|
||||
await files.handle(
|
||||
{ type: 'file.upload', executionId: 'upload', srcPath: '.pi/input/data.bin', url },
|
||||
signal(),
|
||||
),
|
||||
).toMatchObject({ ok: true, size: body.length, name: 'data.bin' })
|
||||
expect(uploaded).toEqual(body)
|
||||
expect(
|
||||
await files.handle(
|
||||
{ type: 'file.stat', executionId: 'absent', srcPath: '.pi/absent' },
|
||||
signal(),
|
||||
),
|
||||
).toMatchObject({ ok: true, exists: false })
|
||||
})
|
||||
|
||||
it('does not publish mismatched downloads or follow redirects', async () => {
|
||||
expect(
|
||||
await files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: 'bad-hash',
|
||||
url,
|
||||
destPath: '.pi/data',
|
||||
expectedHash: 'bad',
|
||||
},
|
||||
signal(),
|
||||
),
|
||||
).toMatchObject({ ok: false, error: 'Downloaded file hash does not match.' })
|
||||
await expect(readFile(path.join(cwd, '.pi/data'))).rejects.toThrow()
|
||||
expect(
|
||||
await files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: 'redirect',
|
||||
url: `${url}/redirect`,
|
||||
destPath: '.pi/data',
|
||||
},
|
||||
signal(),
|
||||
),
|
||||
).toHaveProperty('ok', false)
|
||||
})
|
||||
|
||||
it.each([
|
||||
'../secret',
|
||||
'/etc/passwd',
|
||||
'.pi/../secret',
|
||||
'.pi/dir/../../secret',
|
||||
'.pi\\secret',
|
||||
'.pi//file',
|
||||
'.pi/a:b',
|
||||
'.pi/a\0b',
|
||||
'.pi/.. /outside',
|
||||
'.pi/folder./file',
|
||||
'.pi/CON.txt',
|
||||
'.pi/nul .txt',
|
||||
'.pi/COM1',
|
||||
'.pi/lpt¹.log',
|
||||
'.pi/CONOUT$',
|
||||
])('rejects unsafe protocol path %s', async (srcPath) => {
|
||||
expect(
|
||||
await files.handle({ type: 'file.stat', executionId: 'unsafe', srcPath }, signal()),
|
||||
).toHaveProperty('ok', false)
|
||||
})
|
||||
|
||||
it('rejects parent and final symlinks, including a replaced .pi root', async () => {
|
||||
await mkdir(path.join(cwd, '.pi'))
|
||||
await writeFile(path.join(cwd, 'secret'), 'secret')
|
||||
await symlink(cwd, path.join(cwd, '.pi/escape'))
|
||||
await symlink(path.join(cwd, 'secret'), path.join(cwd, '.pi/link'))
|
||||
for (const srcPath of ['.pi/escape/secret', '.pi/link']) {
|
||||
expect(
|
||||
await files.handle({ type: 'file.upload', executionId: 'symlink', srcPath, url }, signal()),
|
||||
).toHaveProperty('ok', false)
|
||||
}
|
||||
await rm(path.join(cwd, '.pi'), { recursive: true })
|
||||
await symlink(cwd, path.join(cwd, '.pi'))
|
||||
expect(
|
||||
await files.handle(
|
||||
{ type: 'file.download', executionId: 'root', destPath: '.pi/file', url },
|
||||
signal(),
|
||||
),
|
||||
).toHaveProperty('ok', false)
|
||||
})
|
||||
|
||||
it('extracts skills and safely replaces old contents', async () => {
|
||||
const zip = new AdmZip()
|
||||
zip.addFile('SKILL.md', Buffer.from('# Test skill'))
|
||||
zip.addFile('scripts/run.sh', Buffer.from('echo skill'))
|
||||
body = zip.toBuffer()
|
||||
for (let attempt = 0; attempt < 2; attempt++) {
|
||||
expect(
|
||||
await files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: `zip-${attempt}`,
|
||||
url,
|
||||
destPath: '.pi/skills/test',
|
||||
extractZip: true,
|
||||
},
|
||||
signal(),
|
||||
),
|
||||
).toHaveProperty('ok', true)
|
||||
}
|
||||
expect(await readFile(path.join(cwd, '.pi/skills/test/SKILL.md'), 'utf8')).toBe('# Test skill')
|
||||
})
|
||||
|
||||
it('rejects symlink ZIP entries and zip-slip paths without publishing them', async () => {
|
||||
const zip = new AdmZip()
|
||||
zip.addFile('link', Buffer.from('/etc/passwd'))
|
||||
const entry = zip.getEntry('link')!
|
||||
entry.header.attr = (0o120777 << 16) >>> 0
|
||||
body = zip.toBuffer()
|
||||
expect(
|
||||
await files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: 'zip-link',
|
||||
url,
|
||||
destPath: '.pi/skills/bad',
|
||||
extractZip: true,
|
||||
},
|
||||
signal(),
|
||||
),
|
||||
).toHaveProperty('ok', false)
|
||||
const normal = new AdmZip()
|
||||
normal.addFile('aa/secret', Buffer.from('escape'))
|
||||
body = Buffer.from(
|
||||
normal.toBuffer().toString('latin1').replaceAll('aa/secret', '../secret'),
|
||||
'latin1',
|
||||
)
|
||||
expect(
|
||||
await files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: 'zip-slip',
|
||||
url,
|
||||
destPath: '.pi/skills/bad',
|
||||
extractZip: true,
|
||||
},
|
||||
signal(),
|
||||
),
|
||||
).toHaveProperty('ok', false)
|
||||
await expect(readFile(path.join(cwd, '.pi/skills/bad/link'))).rejects.toThrow()
|
||||
})
|
||||
|
||||
it('preserves executable script permissions and rejects oversized ZIP entries', async () => {
|
||||
const zip = new AdmZip()
|
||||
zip.addFile('run.sh', Buffer.from('#!/bin/sh\necho executable\n'))
|
||||
zip.getEntry('run.sh')!.header.attr = (0o100755 << 16) >>> 0
|
||||
body = zip.toBuffer()
|
||||
expect(
|
||||
await files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: 'exec-zip',
|
||||
url,
|
||||
destPath: '.pi/skills/executable',
|
||||
extractZip: true,
|
||||
},
|
||||
signal(),
|
||||
),
|
||||
).toHaveProperty('ok', true)
|
||||
expect((await stat(path.join(cwd, '.pi/skills/executable/run.sh'))).mode & 0o7777).toBe(0o755)
|
||||
const central = body.indexOf(Buffer.from([0x50, 0x4b, 0x01, 0x02]))
|
||||
body.writeUInt32LE(256 * 1024 * 1024 + 1, central + 24)
|
||||
expect(
|
||||
await files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: 'large-zip',
|
||||
url,
|
||||
destPath: '.pi/skills/large',
|
||||
extractZip: true,
|
||||
},
|
||||
signal(),
|
||||
),
|
||||
).toMatchObject({ ok: false, error: 'Extracted ZIP exceeds 256 MiB.' })
|
||||
})
|
||||
|
||||
it('cancels a streaming download without publishing partial bytes or leaving staging files', async () => {
|
||||
await mkdir(path.join(cwd, '.pi/input'), { recursive: true })
|
||||
await writeFile(path.join(cwd, '.pi/input/data'), 'original')
|
||||
const controller = new AbortController()
|
||||
const transfer = files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: 'stream-cancel',
|
||||
url: `${url}/slow`,
|
||||
destPath: '.pi/input/data',
|
||||
},
|
||||
controller.signal,
|
||||
)
|
||||
await vi.waitFor(async () =>
|
||||
expect(
|
||||
(await readdir(path.join(cwd, '.pi/input'))).some((name) => name.startsWith('.shumai-')),
|
||||
).toBe(true),
|
||||
)
|
||||
controller.abort()
|
||||
expect(await transfer).toHaveProperty('ok', false)
|
||||
expect(await readFile(path.join(cwd, '.pi/input/data'), 'utf8')).toBe('original')
|
||||
expect(await readdir(path.join(cwd, '.pi/input'))).toEqual(['data'])
|
||||
})
|
||||
|
||||
it('cancels extraction without deleting the installed skill or leaving staging trees', async () => {
|
||||
await mkdir(path.join(cwd, '.pi/skills/installed'), { recursive: true })
|
||||
await writeFile(path.join(cwd, '.pi/skills/installed/SKILL.md'), 'original')
|
||||
const zip = new AdmZip()
|
||||
for (let index = 0; index < 2000; index++)
|
||||
zip.addFile(`file-${index}.txt`, Buffer.alloc(32768, 65))
|
||||
body = zip.toBuffer()
|
||||
const controller = new AbortController()
|
||||
const transfer = files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: 'extract-cancel',
|
||||
url,
|
||||
destPath: '.pi/skills/installed',
|
||||
extractZip: true,
|
||||
},
|
||||
controller.signal,
|
||||
)
|
||||
await vi.waitFor(
|
||||
async () => {
|
||||
const temporary = (await readdir(path.join(cwd, '.pi/skills'))).find((name) =>
|
||||
name.startsWith('.shumai-'),
|
||||
)
|
||||
expect(temporary).toBeDefined()
|
||||
expect(
|
||||
(await readdir(path.join(cwd, '.pi/skills', temporary!, 'extracted'))).length,
|
||||
).toBeGreaterThan(0)
|
||||
},
|
||||
{ interval: 5, timeout: 5000 },
|
||||
)
|
||||
controller.abort()
|
||||
expect(await transfer).toHaveProperty('ok', false)
|
||||
expect(await readFile(path.join(cwd, '.pi/skills/installed/SKILL.md'), 'utf8')).toBe('original')
|
||||
expect(await readdir(path.join(cwd, '.pi/skills'))).toEqual(['installed'])
|
||||
}, 10000)
|
||||
|
||||
it('waits for an in-flight ZIP entry read before closing on cancellation', async () => {
|
||||
const zip = new AdmZip()
|
||||
zip.addFile('SKILL.md', Buffer.from('skill'))
|
||||
body = zip.toBuffer()
|
||||
const controller = new AbortController()
|
||||
const pendingRead = Promise.withResolvers<() => void>()
|
||||
const readEntry = yauzl.ZipFile.prototype.readEntry
|
||||
const read = vi.spyOn(yauzl.ZipFile.prototype, 'readEntry').mockImplementationOnce(function (
|
||||
this: yauzl.ZipFile,
|
||||
) {
|
||||
pendingRead.resolve(() => readEntry.call(this))
|
||||
})
|
||||
const close = vi.spyOn(yauzl.ZipFile.prototype, 'close')
|
||||
try {
|
||||
const transfer = files.handle(
|
||||
{
|
||||
type: 'file.download',
|
||||
executionId: 'zip-entry-cancel',
|
||||
url,
|
||||
destPath: '.pi/skills/new',
|
||||
extractZip: true,
|
||||
},
|
||||
controller.signal,
|
||||
)
|
||||
const finishRead = await pendingRead.promise
|
||||
controller.abort()
|
||||
await new Promise<void>((resolve) => setImmediate(resolve))
|
||||
const closedDuringRead = close.mock.calls.length > 0
|
||||
if (!closedDuringRead) finishRead()
|
||||
expect(await transfer).toHaveProperty('ok', false)
|
||||
expect(closedDuringRead).toBe(false)
|
||||
expect(await readdir(path.join(cwd, '.pi/skills'))).toEqual([])
|
||||
} finally {
|
||||
read.mockRestore()
|
||||
close.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('handles cancelled transfers and refuses uploads of directories', async () => {
|
||||
const controller = new AbortController()
|
||||
controller.abort()
|
||||
expect(
|
||||
await files.handle(
|
||||
{ type: 'file.download', executionId: 'cancelled', url, destPath: '.pi/data' },
|
||||
controller.signal,
|
||||
),
|
||||
).toHaveProperty('ok', false)
|
||||
expect(
|
||||
await files.handle(
|
||||
{ type: 'file.upload', executionId: 'directory', url, srcPath: '.pi/transfers' },
|
||||
signal(),
|
||||
),
|
||||
).toHaveProperty('ok', false)
|
||||
expect(uploaded).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,282 @@
|
||||
import { createHash } from 'node:crypto'
|
||||
import { constants, createWriteStream } from 'node:fs'
|
||||
import { lstat, mkdir, mkdtemp, open, rename, rm } from 'node:fs/promises'
|
||||
import path from 'node:path'
|
||||
import { Readable, Transform } from 'node:stream'
|
||||
import { pipeline } from 'node:stream/promises'
|
||||
import yauzl from 'yauzl'
|
||||
import type {
|
||||
RunnerFileDownloadMessage,
|
||||
RunnerFileUploadMessage,
|
||||
RunnerFileStatMessage,
|
||||
RunnerToServerMessage,
|
||||
} from '@shumai/dtos'
|
||||
|
||||
const MAX_FILE_BYTES = 256 * 1024 * 1024
|
||||
const MAX_ZIP_ENTRIES = 10_000
|
||||
export type FileMessage =
|
||||
RunnerFileDownloadMessage | RunnerFileUploadMessage | RunnerFileStatMessage
|
||||
|
||||
function pathParts(value: string): string[] {
|
||||
const parts = value.split('/')
|
||||
if (
|
||||
value.includes('\\') ||
|
||||
value.includes('\0') ||
|
||||
parts.some(
|
||||
(part) =>
|
||||
!part ||
|
||||
part === '.' ||
|
||||
part === '..' ||
|
||||
part.includes(':') ||
|
||||
/[. ]$/.test(part) ||
|
||||
/^(?:con|prn|aux|nul|conin\$|conout\$|com[1-9¹²³]|lpt[1-9¹²³])(?: *\.|$)/i.test(part),
|
||||
)
|
||||
)
|
||||
throw new Error('Invalid workspace path.')
|
||||
return parts
|
||||
}
|
||||
|
||||
function isMissing(error: unknown): boolean {
|
||||
return error instanceof Error && 'code' in error && error.code === 'ENOENT'
|
||||
}
|
||||
|
||||
function transferUrl(address: string): URL {
|
||||
const url = new URL(address)
|
||||
if (!['http:', 'https:'].includes(url.protocol) || url.username || url.password)
|
||||
throw new Error('Invalid transfer URL.')
|
||||
return url
|
||||
}
|
||||
|
||||
export class RunnerFiles {
|
||||
constructor(readonly cwd: string) {}
|
||||
|
||||
/** Basic workspace validation. Concurrent hostile filesystem mutation is outside v1's scope. */
|
||||
private async resolve(relative: string, createParents = false): Promise<string> {
|
||||
const parts = pathParts(relative)
|
||||
if (parts[0] !== '.pi' || parts.length < 2) throw new Error('File paths must stay inside .pi/.')
|
||||
let current = this.cwd
|
||||
for (let index = 0; index < parts.length; index++) {
|
||||
current = path.join(current, parts[index])
|
||||
const parent = index < parts.length - 1
|
||||
if (parent && createParents) await mkdir(current, { recursive: true, mode: 0o700 })
|
||||
try {
|
||||
const info = await lstat(current)
|
||||
if (info.isSymbolicLink() || (parent && !info.isDirectory()))
|
||||
throw new Error('Workspace paths cannot contain symlinks or non-directory parents.')
|
||||
} catch (error) {
|
||||
if (!isMissing(error)) throw error
|
||||
}
|
||||
}
|
||||
return current
|
||||
}
|
||||
|
||||
async createWriter(relative: string) {
|
||||
const file = await open(
|
||||
await this.resolve(relative, true),
|
||||
constants.O_CREAT | constants.O_EXCL | constants.O_WRONLY | constants.O_NOFOLLOW,
|
||||
0o600,
|
||||
)
|
||||
return {
|
||||
write: async (data: string) => {
|
||||
await file.writeFile(data)
|
||||
},
|
||||
close: () => file.close(),
|
||||
}
|
||||
}
|
||||
|
||||
async handle(message: FileMessage, signal: AbortSignal): Promise<RunnerToServerMessage> {
|
||||
try {
|
||||
signal.throwIfAborted()
|
||||
if (message.type === 'file.download')
|
||||
return {
|
||||
type: 'file.download.result',
|
||||
executionId: message.executionId,
|
||||
ok: true,
|
||||
...(await this.download(message, signal)),
|
||||
}
|
||||
const filename = await this.resolve(message.srcPath)
|
||||
if (message.type === 'file.upload') return await this.upload(message, filename, signal)
|
||||
try {
|
||||
const info = await lstat(filename)
|
||||
return {
|
||||
type: 'file.stat.result',
|
||||
executionId: message.executionId,
|
||||
ok: true,
|
||||
exists: true,
|
||||
isFile: info.isFile(),
|
||||
size: info.size,
|
||||
name: path.basename(filename),
|
||||
}
|
||||
} catch (error) {
|
||||
if (!isMissing(error)) throw error
|
||||
return {
|
||||
type: 'file.stat.result',
|
||||
executionId: message.executionId,
|
||||
ok: true,
|
||||
exists: false,
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
const result = {
|
||||
executionId: message.executionId,
|
||||
ok: false,
|
||||
error: error instanceof Error ? error.message : 'File operation failed.',
|
||||
}
|
||||
switch (message.type) {
|
||||
case 'file.download':
|
||||
return { ...result, type: 'file.download.result' }
|
||||
case 'file.upload':
|
||||
return { ...result, type: 'file.upload.result' }
|
||||
case 'file.stat':
|
||||
return { ...result, type: 'file.stat.result', exists: false }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async upload(
|
||||
message: RunnerFileUploadMessage,
|
||||
filename: string,
|
||||
signal: AbortSignal,
|
||||
): Promise<RunnerToServerMessage> {
|
||||
const file = await open(
|
||||
filename,
|
||||
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
|
||||
)
|
||||
try {
|
||||
const info = await file.stat()
|
||||
if (!info.isFile() || info.size > MAX_FILE_BYTES)
|
||||
throw new Error('Upload must be a regular file no larger than 256 MiB.')
|
||||
// Node and Bun declare different DOM stream types for the same fetch-compatible stream.
|
||||
const body = info.size
|
||||
? (Readable.toWeb(
|
||||
file.createReadStream({ autoClose: false, end: info.size - 1 }),
|
||||
) as unknown as ReadableStream<Uint8Array>)
|
||||
: new Uint8Array()
|
||||
const response = await fetch(transferUrl(message.url), {
|
||||
method: 'PUT',
|
||||
body,
|
||||
...{ duplex: 'half' },
|
||||
headers: {
|
||||
'content-type': message.contentType ?? 'application/octet-stream',
|
||||
'content-length': String(info.size),
|
||||
},
|
||||
signal,
|
||||
redirect: 'error',
|
||||
})
|
||||
await response.body?.cancel()
|
||||
if (!response.ok) throw new Error(`Upload failed (HTTP ${response.status}).`)
|
||||
return {
|
||||
type: 'file.upload.result',
|
||||
executionId: message.executionId,
|
||||
ok: true,
|
||||
size: info.size,
|
||||
name: path.basename(filename),
|
||||
}
|
||||
} finally {
|
||||
await file.close()
|
||||
}
|
||||
}
|
||||
|
||||
private async download(message: RunnerFileDownloadMessage, signal: AbortSignal) {
|
||||
const destination = await this.resolve(message.destPath, true)
|
||||
const temporary = await mkdtemp(path.join(path.dirname(destination), '.shumai-'))
|
||||
let preserveBackup = false
|
||||
try {
|
||||
const response = await fetch(transferUrl(message.url), { signal, redirect: 'error' })
|
||||
if (!response.ok || !response.body) {
|
||||
await response.body?.cancel()
|
||||
throw new Error(`Download failed (HTTP ${response.status}).`)
|
||||
}
|
||||
const hash = createHash('sha256')
|
||||
let size = 0
|
||||
const archive = path.join(temporary, 'download')
|
||||
await pipeline(
|
||||
response.body,
|
||||
new Transform({
|
||||
transform(chunk: Buffer, _encoding, callback) {
|
||||
size += chunk.length
|
||||
if (size > MAX_FILE_BYTES) {
|
||||
callback(new Error('Download exceeds 256 MiB.'))
|
||||
return
|
||||
}
|
||||
hash.update(chunk)
|
||||
callback(null, chunk)
|
||||
},
|
||||
}),
|
||||
createWriteStream(archive, { flags: 'wx', mode: 0o600 }),
|
||||
{ signal },
|
||||
)
|
||||
if (message.expectedHash && hash.digest('hex') !== message.expectedHash)
|
||||
throw new Error('Downloaded file hash does not match.')
|
||||
let source = archive
|
||||
if (message.extractZip) {
|
||||
source = path.join(temporary, 'extracted')
|
||||
await mkdir(source, { mode: 0o700 })
|
||||
await this.extract(archive, source, signal)
|
||||
}
|
||||
signal.throwIfAborted()
|
||||
await this.resolve(message.destPath)
|
||||
if (message.extractZip) {
|
||||
const backup = path.join(temporary, 'previous')
|
||||
let backedUp = false
|
||||
try {
|
||||
await rename(destination, backup)
|
||||
backedUp = true
|
||||
} catch (error) {
|
||||
if (!isMissing(error)) throw error
|
||||
}
|
||||
try {
|
||||
await rename(source, destination)
|
||||
} catch (error) {
|
||||
if (backedUp) {
|
||||
preserveBackup = true
|
||||
await rename(backup, destination)
|
||||
preserveBackup = false
|
||||
}
|
||||
throw error
|
||||
}
|
||||
} else await rename(source, destination)
|
||||
return { size, cached: false }
|
||||
} finally {
|
||||
if (!preserveBackup) await rm(temporary, { recursive: true, force: true })
|
||||
}
|
||||
}
|
||||
|
||||
private async extract(archive: string, directory: string, signal: AbortSignal) {
|
||||
const zip = await yauzl.openPromise(archive, {
|
||||
lazyEntries: true,
|
||||
strictFileNames: true,
|
||||
autoClose: false,
|
||||
})
|
||||
let size = 0
|
||||
let count = 0
|
||||
try {
|
||||
signal.throwIfAborted()
|
||||
// Let each metadata read finish before cancellation can close the ZIP descriptor.
|
||||
for await (const entry of zip.eachEntry()) {
|
||||
signal.throwIfAborted()
|
||||
if (++count > MAX_ZIP_ENTRIES) throw new Error('ZIP contains too many entries.')
|
||||
const parts = pathParts(entry.fileName.replace(/\/$/, ''))
|
||||
const mode = entry.externalFileAttributes >>> 16
|
||||
const kind = mode & 0o170000
|
||||
if (kind && kind !== 0o100000 && kind !== 0o040000)
|
||||
throw new Error('ZIP links and special files are forbidden.')
|
||||
size += entry.uncompressedSize
|
||||
if (size > MAX_FILE_BYTES) throw new Error('Extracted ZIP exceeds 256 MiB.')
|
||||
const destination = path.join(directory, ...parts)
|
||||
if (entry.fileName.endsWith('/')) await mkdir(destination, { recursive: true, mode: 0o700 })
|
||||
else {
|
||||
await mkdir(path.dirname(destination), { recursive: true, mode: 0o700 })
|
||||
const input = await zip.openReadStreamPromise(entry)
|
||||
await pipeline(
|
||||
input,
|
||||
createWriteStream(destination, { flags: 'wx', mode: 0o600 | (mode & 0o155) }),
|
||||
{ signal },
|
||||
)
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
zip.close()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import {
|
||||
createRunnerLogger,
|
||||
formatBytes,
|
||||
formatDuration,
|
||||
formatLogString,
|
||||
formatTimestamp,
|
||||
shortId,
|
||||
} from './log'
|
||||
|
||||
describe('runner log helpers', () => {
|
||||
it('formats timestamps and aligned log levels', () => {
|
||||
const fixed = new Date(2026, 0, 1, 9, 4, 5)
|
||||
expect(formatTimestamp(fixed)).toBe('09:04:05')
|
||||
const lines: string[] = []
|
||||
const logger = createRunnerLogger(
|
||||
(line) => lines.push(line),
|
||||
false,
|
||||
() => fixed,
|
||||
)
|
||||
logger.info('Bash started [a8f2]')
|
||||
logger.warn('Runner disconnected; reconnecting…')
|
||||
logger.error('Fatal error')
|
||||
expect(lines).toEqual([
|
||||
'09:04:05 INFO Bash started [a8f2]',
|
||||
'09:04:05 WARN Runner disconnected; reconnecting…',
|
||||
'09:04:05 ERROR Fatal error',
|
||||
])
|
||||
})
|
||||
|
||||
it('formats short IDs, durations, byte counts, and single-line quoted strings', () => {
|
||||
expect(shortId('01JABCDEF1234567890A8F2')).toBe('a8f2')
|
||||
expect(shortId('one')).toBe('one')
|
||||
expect(formatDuration(1840)).toBe('1.8s')
|
||||
expect(formatDuration(-10)).toBe('0.0s')
|
||||
expect(formatBytes(0)).toBe('0')
|
||||
expect(formatBytes(512)).toBe('512B')
|
||||
expect(formatBytes(1536)).toBe('1.5KB')
|
||||
expect(formatBytes(12 * 1024)).toBe('12KB')
|
||||
expect(formatBytes(Math.round(2.4 * 1024 * 1024))).toBe('2.4MB')
|
||||
expect(formatLogString('echo "hi"\nls')).toBe('"echo \\"hi\\"\\nls"')
|
||||
expect(formatLogString('x'.repeat(210))).toBe(`"${'x'.repeat(200)}…"`)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,59 @@
|
||||
export type RunnerLogLevel = 'INFO' | 'WARN' | 'ERROR'
|
||||
|
||||
export interface RunnerLogger {
|
||||
readonly verbose: boolean
|
||||
info(message: string): void
|
||||
warn(message: string): void
|
||||
error(message: string): void
|
||||
}
|
||||
|
||||
/** Formats a local wall-clock timestamp as HH:MM:SS for runner stderr logs. */
|
||||
export function formatTimestamp(date = new Date()): string {
|
||||
const pad = (value: number) => String(value).padStart(2, '0')
|
||||
return `${pad(date.getHours())}:${pad(date.getMinutes())}:${pad(date.getSeconds())}`
|
||||
}
|
||||
|
||||
/** Builds a prefixed runner logger writing uniform timestamped lines to stderr. */
|
||||
export function createRunnerLogger(
|
||||
write: (message: string) => void = console.error,
|
||||
verbose = false,
|
||||
now: () => Date = () => new Date(),
|
||||
): RunnerLogger {
|
||||
const emit = (level: RunnerLogLevel, message: string) => {
|
||||
write(`${formatTimestamp(now())} ${level.padEnd(5)} ${message}`)
|
||||
}
|
||||
return {
|
||||
verbose,
|
||||
info: (message) => emit('INFO', message),
|
||||
warn: (message) => emit('WARN', message),
|
||||
error: (message) => emit('ERROR', message),
|
||||
}
|
||||
}
|
||||
|
||||
/** Extracts the trailing 4-character tail of an execution ID for compact log labels. */
|
||||
export function shortId(executionId: string): string {
|
||||
return executionId.slice(-4).toLowerCase()
|
||||
}
|
||||
|
||||
/** Formats a millisecond duration as seconds with one decimal place (e.g. `1.8s`). */
|
||||
export function formatDuration(ms: number): string {
|
||||
return `${(Math.max(0, ms) / 1000).toFixed(1)}s`
|
||||
}
|
||||
|
||||
/** Formats a byte count concisely for runner logs (`0`, `128B`, `12KB`, `2.4MB`). */
|
||||
export function formatBytes(bytes: number): string {
|
||||
if (bytes <= 0) return '0'
|
||||
if (bytes < 1024) return `${bytes}B`
|
||||
if (bytes < 1024 * 1024) {
|
||||
const kb = bytes / 1024
|
||||
return kb < 10 ? `${kb.toFixed(1).replace(/\.0$/, '')}KB` : `${Math.round(kb)}KB`
|
||||
}
|
||||
const mb = bytes / (1024 * 1024)
|
||||
return `${mb.toFixed(1).replace(/\.0$/, '')}MB`
|
||||
}
|
||||
|
||||
/** Formats an arbitrary string (command, path, error) as a bounded single-line JSON string. */
|
||||
export function formatLogString(value: string, maxLength = 200): string {
|
||||
const text = value.length > maxLength ? `${value.slice(0, maxLength)}…` : value
|
||||
return JSON.stringify(text)
|
||||
}
|
||||
@@ -0,0 +1,106 @@
|
||||
import { spawnSync, type SpawnSyncReturns } from 'node:child_process'
|
||||
import { accessSync, constants } from 'node:fs'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { resolveRunnerShell } from './shell-config'
|
||||
|
||||
vi.mock('node:child_process', () => ({ spawnSync: vi.fn() }))
|
||||
vi.mock('node:fs', { spy: true })
|
||||
|
||||
const access = vi.mocked(accessSync)
|
||||
const where = vi.mocked(spawnSync)
|
||||
const gitBash = 'D:\\Program Files\\Git\\bin\\bash.exe'
|
||||
const gitBashX86 = 'D:\\Program Files (x86)\\Git\\bin\\bash.exe'
|
||||
|
||||
function whereResult(stdout = '', status = 0): SpawnSyncReturns<string> {
|
||||
return { stdout, stderr: '', status, output: [null, stdout, ''], pid: 1, signal: null }
|
||||
}
|
||||
|
||||
function available(...paths: string[]) {
|
||||
access.mockImplementation((executable) => {
|
||||
if (!paths.some((path) => executable === path)) throw new Error('File not found')
|
||||
})
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.stubEnv('ProgramFiles', 'D:\\Program Files')
|
||||
vi.stubEnv('ProgramFiles(x86)', 'D:\\Program Files (x86)')
|
||||
vi.stubEnv('SystemRoot', 'E:\\WinNT')
|
||||
available()
|
||||
where.mockReturnValue(whereResult('', 1))
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.resetAllMocks()
|
||||
vi.unstubAllEnvs()
|
||||
})
|
||||
|
||||
describe('runner shell selection', () => {
|
||||
it.each(['linux', 'darwin'] as const)('uses /bin/sh on %s', (platform) => {
|
||||
expect(resolveRunnerShell(platform)).toEqual({
|
||||
executable: '/bin/sh',
|
||||
commandTransport: 'shell',
|
||||
})
|
||||
expect(where).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it.each([
|
||||
[gitBash, [gitBash, gitBashX86]],
|
||||
[gitBashX86, [gitBashX86]],
|
||||
] as const)('prefers installed Git Bash at %s before PATH', (executable, paths) => {
|
||||
available(...paths)
|
||||
expect(resolveRunnerShell('win32')).toEqual({ executable, commandTransport: 'argv' })
|
||||
expect(where).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
// Adapted from Pi's shell-path-exists.test.ts. See apps/cli/THIRD-PARTY-NOTICES.txt.
|
||||
it('accepts a Windows Store Bash alias found on PATH via access(F_OK)', () => {
|
||||
const alias = 'D:\\Users\\runner\\AppData\\Local\\Microsoft\\WindowsApps\\bash.exe'
|
||||
available(alias)
|
||||
where.mockReturnValue(whereResult(alias + '\r\n'))
|
||||
expect(resolveRunnerShell('win32')).toEqual({ executable: alias, commandTransport: 'argv' })
|
||||
expect(access).toHaveBeenCalledWith(alias, constants.F_OK)
|
||||
expect(where).toHaveBeenCalledWith('E:\\WinNT\\System32\\where.exe', ['$PATH:bash.exe'], {
|
||||
encoding: 'utf8',
|
||||
timeout: 5000,
|
||||
windowsHide: true,
|
||||
})
|
||||
})
|
||||
|
||||
it('skips relative and inaccessible PATH results', () => {
|
||||
const executable = 'D:\\MSYS2\\usr\\bin\\bash.exe'
|
||||
available(executable)
|
||||
where.mockReturnValue(
|
||||
whereResult(['bash.exe', 'D:\\missing\\bash.exe', executable].join('\r\n')),
|
||||
)
|
||||
expect(resolveRunnerShell('win32')).toEqual({ executable, commandTransport: 'argv' })
|
||||
})
|
||||
|
||||
it.each([
|
||||
['E:\\WinNT', 'E:\\WinNT\\System32\\cmd.exe'],
|
||||
[undefined, 'C:\\Windows\\System32\\cmd.exe'],
|
||||
] as const)(
|
||||
'falls back to cmd.exe when no Bash is available (SystemRoot %s)',
|
||||
(root, executable) => {
|
||||
vi.stubEnv('SystemRoot', root)
|
||||
expect(resolveRunnerShell('win32')).toEqual({ executable, commandTransport: 'shell' })
|
||||
},
|
||||
)
|
||||
|
||||
// Adapted from Pi's windows-shell-config.test.ts.
|
||||
it.each([
|
||||
['D:\\WinNT\\', 'd:/WINNT/System32/BASH.EXE', 'stdin'],
|
||||
['D:\\WinNT', 'D:\\WinNT\\Sysnative\\bash.exe', 'stdin'],
|
||||
[undefined, 'E:\\Windows\\System32\\bash.exe', 'stdin'],
|
||||
['D:\\WinNT', 'D:\\Git\\bin\\bash.exe', 'argv'],
|
||||
['D:\\WinNT', 'E:\\Windows\\System32\\bash.exe', 'argv'],
|
||||
['WinNT', 'C:\\Windows\\System32\\bash.exe', 'argv'],
|
||||
] as const)(
|
||||
'uses the correct command transport for SystemRoot %s and %s',
|
||||
(root, executable, commandTransport) => {
|
||||
vi.stubEnv('SystemRoot', root)
|
||||
available(executable)
|
||||
where.mockReturnValue(whereResult(executable + '\r\n'))
|
||||
expect(resolveRunnerShell('win32')).toEqual({ executable, commandTransport })
|
||||
},
|
||||
)
|
||||
})
|
||||
@@ -0,0 +1,64 @@
|
||||
import { spawnSync } from 'node:child_process'
|
||||
import { accessSync, constants } from 'node:fs'
|
||||
import { win32 } from 'node:path'
|
||||
|
||||
export interface RunnerShellConfig {
|
||||
executable: string
|
||||
commandTransport: 'shell' | 'argv' | 'stdin'
|
||||
}
|
||||
|
||||
// Windows Bash discovery and legacy WSL detection adapted from pi-mono.
|
||||
// See apps/cli/THIRD-PARTY-NOTICES.txt.
|
||||
function isLegacyWslBashPath(executable: string, systemRoot: string | undefined): boolean {
|
||||
const normalized = win32.normalize(executable).toLowerCase()
|
||||
if (systemRoot) {
|
||||
if (!/^[a-z]:[\\/]/i.test(systemRoot)) return false
|
||||
return ['System32', 'Sysnative'].some(
|
||||
(directory) => normalized === win32.join(systemRoot, directory, 'bash.exe').toLowerCase(),
|
||||
)
|
||||
}
|
||||
return /^[a-z]:\\windows\\(?:system32|sysnative)\\bash\.exe$/.test(normalized)
|
||||
}
|
||||
|
||||
function shellPathExists(executable: string): boolean {
|
||||
// access(F_OK) accepts runnable Windows Store aliases that fail an extra stat check.
|
||||
try {
|
||||
accessSync(executable, constants.F_OK)
|
||||
return true
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
function findWindowsBash(system32: string): string | undefined {
|
||||
for (const directory of [process.env.ProgramFiles, process.env['ProgramFiles(x86)']]) {
|
||||
if (!directory || !win32.isAbsolute(directory)) continue
|
||||
const executable = win32.join(directory, 'Git', 'bin', 'bash.exe')
|
||||
if (shellPathExists(executable)) return executable
|
||||
}
|
||||
|
||||
// Use the system tool and limit its search to PATH. The workspace may contain a bash.exe.
|
||||
const found = spawnSync(win32.join(system32, 'where.exe'), ['$PATH:bash.exe'], {
|
||||
encoding: 'utf8',
|
||||
timeout: 5000,
|
||||
windowsHide: true,
|
||||
})
|
||||
if (found.status !== 0 || !found.stdout) return
|
||||
return found.stdout
|
||||
.trim()
|
||||
.split(/\r?\n/)
|
||||
.find((executable) => win32.isAbsolute(executable) && shellPathExists(executable))
|
||||
}
|
||||
|
||||
export function resolveRunnerShell(platform = process.platform): RunnerShellConfig {
|
||||
if (platform !== 'win32') return { executable: '/bin/sh', commandTransport: 'shell' }
|
||||
|
||||
const systemRoot = process.env.SystemRoot
|
||||
const system32 = win32.join(systemRoot ?? 'C:\\Windows', 'System32')
|
||||
const executable = findWindowsBash(system32)
|
||||
if (!executable) return { executable: win32.join(system32, 'cmd.exe'), commandTransport: 'shell' }
|
||||
return {
|
||||
executable,
|
||||
commandTransport: isLegacyWslBashPath(executable, systemRoot) ? 'stdin' : 'argv',
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,352 @@
|
||||
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { SandboxManager } from '@anthropic-ai/sandbox-runtime'
|
||||
import { RunnerFiles } from './files'
|
||||
import { RunnerShell } from './shell'
|
||||
|
||||
describe('runner bash execution', () => {
|
||||
let cwd: string
|
||||
let shell: RunnerShell
|
||||
beforeEach(async () => {
|
||||
cwd = await mkdtemp(path.join(tmpdir(), 'shumai-shell-test-'))
|
||||
shell = new RunnerShell(new RunnerFiles(cwd))
|
||||
vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
|
||||
vi.spyOn(SandboxManager, 'reset').mockResolvedValue()
|
||||
vi.spyOn(SandboxManager, 'wrapWithSandboxArgv').mockImplementation(async (command) => ({
|
||||
argv: ['/bin/sh', '-c', command],
|
||||
env: process.env,
|
||||
}))
|
||||
})
|
||||
afterEach(async () => {
|
||||
await shell.close()
|
||||
vi.restoreAllMocks()
|
||||
vi.unstubAllEnvs()
|
||||
await rm(cwd, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
const request = (command: string, executionId = 'test') => ({
|
||||
type: 'exec.start' as const,
|
||||
executionId,
|
||||
command,
|
||||
})
|
||||
|
||||
it('runs with no executables on PATH when SRT is not requested', async () => {
|
||||
vi.stubEnv('PATH', '')
|
||||
await shell.initialize()
|
||||
let output = ''
|
||||
const result = await shell.run(
|
||||
request('printf no-extra-runtime'),
|
||||
new AbortController().signal,
|
||||
(_stream, data) => {
|
||||
output += data
|
||||
},
|
||||
)
|
||||
expect(result).toMatchObject({ exitCode: 0 })
|
||||
expect(output).toBe('no-extra-runtime')
|
||||
})
|
||||
|
||||
// Adapted from Pi's tools.test.ts. See apps/cli/THIRD-PARTY-NOTICES.txt.
|
||||
it('sends the complete command over stdin for the legacy Bash launcher', async () => {
|
||||
shell = new RunnerShell(new RunnerFiles(cwd), undefined, {
|
||||
executable: '/bin/sh',
|
||||
commandTransport: 'stdin',
|
||||
})
|
||||
const command = [
|
||||
'name="Hello, 世界"',
|
||||
'for choice in "one two" \'three "four"\'; do',
|
||||
' printf \'%s / %s\\n\' "$name" "$choice"',
|
||||
'done',
|
||||
].join('\n')
|
||||
let output = ''
|
||||
const result = await shell.run(
|
||||
request(command),
|
||||
new AbortController().signal,
|
||||
(_stream, data) => {
|
||||
output += data
|
||||
},
|
||||
)
|
||||
expect(result).toMatchObject({ exitCode: 0 })
|
||||
expect(output).toBe('Hello, 世界 / one two\nHello, 世界 / three "four"\n')
|
||||
})
|
||||
|
||||
it('passes a command with shell quoting directly to the selected executable', async () => {
|
||||
shell = new RunnerShell(new RunnerFiles(cwd), undefined, {
|
||||
executable: '/bin/sh',
|
||||
commandTransport: 'argv',
|
||||
})
|
||||
const command = "printf '%s' '$literal \"quotes\"'"
|
||||
let output = ''
|
||||
const result = await shell.run(
|
||||
request(command),
|
||||
new AbortController().signal,
|
||||
(_stream, data) => {
|
||||
output += data
|
||||
},
|
||||
)
|
||||
expect(result).toMatchObject({ exitCode: 0 })
|
||||
expect(output).toBe('$literal "quotes"')
|
||||
})
|
||||
|
||||
it('rejects the legacy WSL launcher before starting SRT', async () => {
|
||||
shell = new RunnerShell(new RunnerFiles(cwd), path.join(cwd, 'sandbox.json'), {
|
||||
executable: 'C:\\Windows\\System32\\bash.exe',
|
||||
commandTransport: 'stdin',
|
||||
})
|
||||
await expect(shell.initialize()).rejects.toThrow('SRT cannot run the legacy WSL Bash launcher')
|
||||
expect(SandboxManager.initialize).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('streams both outputs, returns the exit code, and saves a log in the runner workspace', async () => {
|
||||
const output: string[] = []
|
||||
const result = await shell.run(
|
||||
request('printf "hello 世界"; printf "failure" >&2; exit 7'),
|
||||
new AbortController().signal,
|
||||
(stream, data) => output.push(`${stream}:${data}`),
|
||||
)
|
||||
expect(result).toMatchObject({
|
||||
exitCode: 7,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
fullOutputPath: '.pi/logs/bash-test.log',
|
||||
})
|
||||
expect(output.join('|')).toContain('stdout:hello 世界')
|
||||
expect(output.join('|')).toContain('stderr:failure')
|
||||
expect(await readFile(path.join(cwd, result.fullOutputPath!), 'utf8')).toContain('hello 世界')
|
||||
})
|
||||
|
||||
it('passes skill env without inheriting runner or unrelated credentials', async () => {
|
||||
vi.stubEnv('SHUMAI_RUNNER_KEY', 'runner-secret')
|
||||
vi.stubEnv('AWS_SECRET_ACCESS_KEY', 'aws-secret')
|
||||
let output = ''
|
||||
await shell.run(
|
||||
{
|
||||
...request(
|
||||
'printf "%s|%s|%s" "$SKILL_TOKEN" "$SHUMAI_RUNNER_KEY" "$AWS_SECRET_ACCESS_KEY"',
|
||||
),
|
||||
env: { SKILL_TOKEN: 'skill-secret' },
|
||||
},
|
||||
new AbortController().signal,
|
||||
(_stream, data) => {
|
||||
output += data
|
||||
},
|
||||
)
|
||||
expect(output).toBe('skill-secret||')
|
||||
})
|
||||
|
||||
it('times out and terminates background children', async () => {
|
||||
const result = await shell.run(
|
||||
{ ...request('sleep 30 & echo $!; wait'), timeoutSec: 0.1 },
|
||||
new AbortController().signal,
|
||||
() => {},
|
||||
)
|
||||
expect(result).toMatchObject({ timedOut: true, signal: 'SIGKILL' })
|
||||
const pid = Number((await readFile(path.join(cwd, '.pi/logs/bash-test.log'), 'utf8')).trim())
|
||||
expect(() => process.kill(pid, 0)).toThrow()
|
||||
})
|
||||
|
||||
it('cancels a running command and never starts an already cancelled one', async () => {
|
||||
const controller = new AbortController()
|
||||
const running = shell.run(request('echo ready; sleep 30'), controller.signal, () =>
|
||||
controller.abort(),
|
||||
)
|
||||
expect(await running).toMatchObject({ aborted: true, signal: 'SIGKILL' })
|
||||
expect(
|
||||
await shell.run(request('touch unwanted', 'cancelled'), controller.signal, () => {}),
|
||||
).toMatchObject({ aborted: true })
|
||||
await expect(readFile(path.join(cwd, 'unwanted'))).rejects.toThrow()
|
||||
})
|
||||
|
||||
it('finishes draining after a timely root exit even if a background process holds the pipes', async () => {
|
||||
const source = `const {spawn} = require('node:child_process'); const child = spawn(process.execPath, ['-e', 'setTimeout(() => {}, 30000)'], {detached: true, stdio: ['ignore', 1, 2]}); console.log(child.pid); child.unref()`
|
||||
const quote = (value: string) => `'${value.replaceAll("'", "'\\''")}'`
|
||||
const result = await shell.run(
|
||||
{ ...request(`${quote(process.execPath)} -e ${quote(source)}`), timeoutSec: 0.5 },
|
||||
new AbortController().signal,
|
||||
() => {},
|
||||
)
|
||||
const pid = Number((await readFile(path.join(cwd, '.pi/logs/bash-test.log'), 'utf8')).trim())
|
||||
try {
|
||||
expect(result).toMatchObject({ exitCode: 0, timedOut: false, aborted: false })
|
||||
expect(result.error).toBeUndefined()
|
||||
} finally {
|
||||
if (pid > 0) process.kill(pid, 'SIGKILL')
|
||||
}
|
||||
})
|
||||
|
||||
it('reports launch failure without including command text or skill secrets', async () => {
|
||||
shell = new RunnerShell(new RunnerFiles(cwd), undefined, {
|
||||
executable: path.join(cwd, 'missing-shell'),
|
||||
commandTransport: 'shell',
|
||||
})
|
||||
const result = await shell.run(
|
||||
request('echo private-command'),
|
||||
new AbortController().signal,
|
||||
() => {},
|
||||
)
|
||||
expect(result).toMatchObject({
|
||||
exitCode: null,
|
||||
signal: null,
|
||||
error: 'Command execution failed (ENOENT).',
|
||||
})
|
||||
expect(result.error).not.toContain('private-command')
|
||||
})
|
||||
|
||||
it('terminates a running process if the output consumer fails', async () => {
|
||||
let pid = 0
|
||||
const result = await shell.run(
|
||||
request('printf "%s\\n" "$$"; sleep 30'),
|
||||
new AbortController().signal,
|
||||
(_stream, data) => {
|
||||
pid = Number(data.trim())
|
||||
throw new Error('Output consumer failed')
|
||||
},
|
||||
)
|
||||
expect(result.error).toBe('Output consumer failed')
|
||||
expect(pid).toBeGreaterThan(0)
|
||||
expect(() => process.kill(pid, 0)).toThrow()
|
||||
})
|
||||
|
||||
it('rejects invalid timeouts and caps runaway output', async () => {
|
||||
expect(
|
||||
await shell.run(
|
||||
{ ...request('sleep 30'), timeoutSec: -1 },
|
||||
new AbortController().signal,
|
||||
() => {},
|
||||
),
|
||||
).toHaveProperty('error')
|
||||
expect(
|
||||
await shell.run(
|
||||
request('head -c 70000000 /dev/zero', 'large'),
|
||||
new AbortController().signal,
|
||||
() => {},
|
||||
),
|
||||
).toMatchObject({ error: 'Command exceeded the 64 MiB output limit.' })
|
||||
})
|
||||
|
||||
it('uses the exact SRT JSON snapshot and validates SRT before connecting', async () => {
|
||||
const settings = path.join(cwd, 'sandbox.json')
|
||||
const config = {
|
||||
network: { allowedDomains: [], deniedDomains: [] },
|
||||
filesystem: { denyRead: [], allowWrite: ['.pi'], denyWrite: [] },
|
||||
}
|
||||
const json = JSON.stringify(config)
|
||||
await writeFile(settings, json)
|
||||
vi.stubEnv('PATH', '')
|
||||
shell = new RunnerShell(new RunnerFiles(cwd), settings)
|
||||
await shell.initialize()
|
||||
await writeFile(settings, '{}')
|
||||
expect(
|
||||
await shell.run(request('echo sandboxed'), new AbortController().signal, () => {}),
|
||||
).toMatchObject({ exitCode: 0 })
|
||||
expect(SandboxManager.initialize).toHaveBeenCalledExactlyOnceWith(
|
||||
expect.objectContaining(config),
|
||||
)
|
||||
expect(SandboxManager.wrapWithSandboxArgv).toHaveBeenCalledWith(
|
||||
expect.stringContaining('echo sandboxed'),
|
||||
'/bin/sh',
|
||||
undefined,
|
||||
expect.any(AbortSignal),
|
||||
cwd,
|
||||
{ commandId: 'test', commandText: 'echo sandboxed' },
|
||||
)
|
||||
expect(SandboxManager.reset).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('keeps skill PATH and Node startup settings out of the SRT launcher', async () => {
|
||||
const settings = path.join(cwd, 'sandbox.json')
|
||||
const marker = path.join(cwd, 'injected')
|
||||
const evil = path.join(cwd, 'evil.cjs')
|
||||
await writeFile(
|
||||
settings,
|
||||
JSON.stringify({
|
||||
network: { allowedDomains: [], deniedDomains: [] },
|
||||
filesystem: { denyRead: [], allowWrite: ['.pi'], denyWrite: [] },
|
||||
}),
|
||||
)
|
||||
await writeFile(evil, `require('node:fs').writeFileSync('${marker}', 'escaped')`)
|
||||
shell = new RunnerShell(new RunnerFiles(cwd), settings)
|
||||
await shell.initialize()
|
||||
const result = await shell.run(
|
||||
{
|
||||
...request('printf secure'),
|
||||
env: { PATH: '/evil-srt', NODE_OPTIONS: `--require=${evil}` },
|
||||
},
|
||||
new AbortController().signal,
|
||||
() => {},
|
||||
)
|
||||
expect(result).toMatchObject({ exitCode: 0 })
|
||||
await expect(readFile(marker)).rejects.toThrow()
|
||||
})
|
||||
|
||||
it('initializes one sandbox for parallel commands and resets only at shutdown', async () => {
|
||||
const settings = path.join(cwd, 'sandbox.json')
|
||||
await writeFile(
|
||||
settings,
|
||||
JSON.stringify({
|
||||
network: { allowedDomains: [], deniedDomains: [] },
|
||||
filesystem: { denyRead: [], allowWrite: ['.pi'], denyWrite: [] },
|
||||
}),
|
||||
)
|
||||
shell = new RunnerShell(new RunnerFiles(cwd), settings)
|
||||
await shell.initialize()
|
||||
const value = "quoted ' value\n$(echo should-not-expand)"
|
||||
let output = ''
|
||||
const results = await Promise.all(
|
||||
['one', 'two'].map((id) =>
|
||||
shell.run(
|
||||
{ ...request('printf "%s" "$SKILL_TOKEN"', id), env: { SKILL_TOKEN: value } },
|
||||
new AbortController().signal,
|
||||
(_stream, data) => {
|
||||
output += data
|
||||
},
|
||||
),
|
||||
),
|
||||
)
|
||||
expect(results.every((result) => result.exitCode === 0)).toBe(true)
|
||||
expect(output).toBe(value.repeat(2))
|
||||
expect(SandboxManager.initialize).toHaveBeenCalledTimes(1)
|
||||
expect(SandboxManager.reset).not.toHaveBeenCalled()
|
||||
await shell.close()
|
||||
expect(SandboxManager.reset).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('fails startup and cleans up if sandbox initialization or execution fails', async () => {
|
||||
const settings = path.join(cwd, 'sandbox.json')
|
||||
await writeFile(
|
||||
settings,
|
||||
JSON.stringify({
|
||||
network: { allowedDomains: [], deniedDomains: [] },
|
||||
filesystem: { denyRead: [], allowWrite: ['.pi'], denyWrite: [] },
|
||||
}),
|
||||
)
|
||||
shell = new RunnerShell(new RunnerFiles(cwd), settings)
|
||||
vi.mocked(SandboxManager.initialize).mockRejectedValueOnce(
|
||||
new Error('Run windows-install first'),
|
||||
)
|
||||
await expect(shell.initialize()).rejects.toThrow('windows-install')
|
||||
expect(SandboxManager.reset).toHaveBeenCalledTimes(1)
|
||||
expect(
|
||||
await shell.run(request('echo unsafe'), new AbortController().signal, () => {}),
|
||||
).toHaveProperty('error', 'SRT has not been initialized.')
|
||||
vi.mocked(SandboxManager.wrapWithSandboxArgv).mockRejectedValueOnce(
|
||||
new Error('sandbox unavailable'),
|
||||
)
|
||||
await expect(shell.initialize()).rejects.toThrow('SRT preflight failed: sandbox unavailable')
|
||||
})
|
||||
|
||||
it('fails closed for missing or malformed SRT settings', async () => {
|
||||
const settings = path.join(cwd, 'sandbox.json')
|
||||
await expect(new RunnerShell(new RunnerFiles(cwd), settings).initialize()).rejects.toThrow()
|
||||
await writeFile(settings, 'invalid json')
|
||||
await expect(new RunnerShell(new RunnerFiles(cwd), settings).initialize()).rejects.toThrow()
|
||||
await writeFile(settings, '[]')
|
||||
await expect(new RunnerShell(new RunnerFiles(cwd), settings).initialize()).rejects.toThrow(
|
||||
/object/,
|
||||
)
|
||||
await writeFile(settings, JSON.stringify({ network: { allowedDomains: 42 } }))
|
||||
await expect(new RunnerShell(new RunnerFiles(cwd), settings).initialize()).rejects.toThrow()
|
||||
expect(SandboxManager.initialize).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,232 @@
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import { execa } from 'execa'
|
||||
import { SandboxManager } from '@anthropic-ai/sandbox-runtime'
|
||||
import {
|
||||
runnerEnvironmentSchema,
|
||||
type RunnerEnvironment,
|
||||
type RunnerExecResultMessage,
|
||||
type RunnerExecStartMessage,
|
||||
} from '@shumai/dtos'
|
||||
import { commandEnv } from './config'
|
||||
import { RunnerFiles } from './files'
|
||||
import { resolveRunnerShell } from './shell-config'
|
||||
import { loadSandbox } from './srt'
|
||||
|
||||
const MAX_OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
const MAX_TIMEOUT_SEC = 3600
|
||||
|
||||
export class RunnerShell {
|
||||
readonly environment: RunnerEnvironment
|
||||
private sandbox?: Awaited<ReturnType<typeof loadSandbox>>
|
||||
private initialized = false
|
||||
private active = 0
|
||||
|
||||
constructor(
|
||||
private readonly files: RunnerFiles,
|
||||
private readonly srtPath?: string,
|
||||
private readonly config = resolveRunnerShell(),
|
||||
) {
|
||||
this.environment = runnerEnvironmentSchema.parse({
|
||||
platform: process.platform,
|
||||
shell: config.executable,
|
||||
})
|
||||
}
|
||||
|
||||
async initialize() {
|
||||
if (!this.srtPath) return
|
||||
if (this.config.commandTransport === 'stdin')
|
||||
throw new Error(
|
||||
'SRT cannot run the legacy WSL Bash launcher. Install Git for Windows to use Bash with SRT.',
|
||||
)
|
||||
// Read and validate once; commands cannot weaken the policy by editing the file.
|
||||
this.sandbox = await loadSandbox(this.srtPath)
|
||||
try {
|
||||
await SandboxManager.initialize(this.sandbox.config)
|
||||
this.initialized = true
|
||||
let diagnostic = ''
|
||||
const probe = await this.run(
|
||||
{ type: 'exec.start', executionId: randomUUID(), command: 'exit 0', timeoutSec: 15 },
|
||||
new AbortController().signal,
|
||||
(_stream, data) => {
|
||||
diagnostic = (diagnostic + data).slice(-4096)
|
||||
},
|
||||
)
|
||||
if (probe.exitCode !== 0 || probe.error || probe.timedOut)
|
||||
throw new Error(
|
||||
`SRT preflight failed: ${probe.error || diagnostic.trim() || 'Check settings and local SRT prerequisites.'}`,
|
||||
)
|
||||
} catch (error) {
|
||||
await this.close()
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async close() {
|
||||
if (!this.sandbox) return
|
||||
try {
|
||||
await SandboxManager.reset()
|
||||
} finally {
|
||||
await this.sandbox.cleanup()
|
||||
this.sandbox = undefined
|
||||
this.initialized = false
|
||||
}
|
||||
}
|
||||
|
||||
async run(
|
||||
message: RunnerExecStartMessage,
|
||||
signal: AbortSignal,
|
||||
output: (stream: 'stdout' | 'stderr', data: string) => void,
|
||||
): Promise<RunnerExecResultMessage> {
|
||||
const result: RunnerExecResultMessage = {
|
||||
type: 'exec.result',
|
||||
executionId: message.executionId,
|
||||
exitCode: null,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
}
|
||||
if (this.srtPath) this.active++
|
||||
try {
|
||||
if (signal.aborted) return { ...result, aborted: true }
|
||||
const timeout = message.timeoutSec ?? 120
|
||||
if (!Number.isFinite(timeout) || timeout <= 0 || timeout > MAX_TIMEOUT_SEC)
|
||||
throw new Error(`Command timeout must be between 0 and ${MAX_TIMEOUT_SEC} seconds.`)
|
||||
const logPath = `.pi/logs/bash-${message.executionId}.log`
|
||||
const log = await this.files.createWriter(logPath)
|
||||
try {
|
||||
const windows = process.platform === 'win32'
|
||||
const { commandTransport } = this.config
|
||||
const useShell = commandTransport === 'shell'
|
||||
const sendCommandOnStdin = commandTransport === 'stdin'
|
||||
let executable = useShell ? message.command : this.config.executable
|
||||
let args = useShell ? [] : sendCommandOnStdin ? ['-s'] : ['-c', message.command]
|
||||
let shell: string | false = useShell && this.config.executable
|
||||
let env = commandEnv(message.env)
|
||||
if (this.srtPath) {
|
||||
if (!this.initialized) throw new Error('SRT has not been initialized.')
|
||||
const skillEnv = commandEnv(message.env, false)
|
||||
const command = windows
|
||||
? message.command
|
||||
: [
|
||||
...Object.entries(skillEnv).map(
|
||||
([key, value]) => `export ${key}='${value!.replaceAll("'", "'\\''")}'`,
|
||||
),
|
||||
message.command,
|
||||
].join('\n')
|
||||
const wrapped = await SandboxManager.wrapWithSandboxArgv(
|
||||
command,
|
||||
this.config.executable,
|
||||
undefined,
|
||||
signal,
|
||||
this.files.cwd,
|
||||
{ commandId: message.executionId, commandText: message.command },
|
||||
)
|
||||
;[executable, ...args] = wrapped.argv
|
||||
shell = false
|
||||
if (windows) {
|
||||
// SRT's broker starts a fresh sandbox-user environment. Explicit skill
|
||||
// values belong in its child overlay, never in the broker's startup env.
|
||||
const separator = args.indexOf('--')
|
||||
if (separator < 0) throw new Error('SRT returned an invalid Windows launch command.')
|
||||
args.splice(
|
||||
separator,
|
||||
0,
|
||||
...Object.entries(skillEnv).flatMap(([key, value]) => ['--env', `${key}=${value}`]),
|
||||
)
|
||||
env = wrapped.env
|
||||
} else env = commandEnv()
|
||||
}
|
||||
if (signal.aborted) return { ...result, aborted: true }
|
||||
const deadline = Date.now() + Math.ceil(timeout * 1000)
|
||||
const child = execa(executable, args, {
|
||||
cwd: this.files.cwd,
|
||||
env,
|
||||
extendEnv: false,
|
||||
shell,
|
||||
stdin: sendCommandOnStdin ? [Buffer.from(message.command)] : 'ignore',
|
||||
buffer: false,
|
||||
reject: false,
|
||||
stripFinalNewline: false,
|
||||
verbose: 'none',
|
||||
windowsHide: true,
|
||||
killDescendants: true,
|
||||
killSignal: 'SIGKILL',
|
||||
cancelSignal: signal,
|
||||
timeout: Math.ceil(timeout * 1000),
|
||||
})
|
||||
let bytes = 0
|
||||
let writes = Promise.resolve()
|
||||
let drainTimer: ReturnType<typeof setTimeout> | undefined
|
||||
let drainExpired = false
|
||||
let exitedWithinDeadline = false
|
||||
child.nodeChildProcess.once('exit', (exitCode) => {
|
||||
exitedWithinDeadline = exitCode !== null && Date.now() < deadline
|
||||
// Execa cleans up descendants where the OS still permits it. Bound
|
||||
// draining if a background child keeps inherited output pipes open.
|
||||
child.kill('SIGKILL')
|
||||
drainTimer = setTimeout(() => {
|
||||
drainExpired = true
|
||||
child.stdout.destroy()
|
||||
child.stderr.destroy()
|
||||
}, 1000)
|
||||
})
|
||||
try {
|
||||
const readers = (['stdout', 'stderr'] as const).map(async (stream) => {
|
||||
child[stream].setEncoding('utf8')
|
||||
try {
|
||||
for await (const data of child[stream]) {
|
||||
bytes += Buffer.byteLength(data)
|
||||
if (bytes > MAX_OUTPUT_BYTES)
|
||||
throw new Error('Command exceeded the 64 MiB output limit.')
|
||||
await (writes = writes.then(async () => {
|
||||
await log.write(data)
|
||||
output(stream, data)
|
||||
}))
|
||||
}
|
||||
} catch (error) {
|
||||
if (
|
||||
drainExpired &&
|
||||
error instanceof Error &&
|
||||
'code' in error &&
|
||||
error.code === 'ERR_STREAM_PREMATURE_CLOSE'
|
||||
)
|
||||
return
|
||||
result.error ??= error instanceof Error ? error.message : 'Output streaming failed.'
|
||||
child.kill('SIGKILL')
|
||||
}
|
||||
})
|
||||
const [completed] = await Promise.all([child, Promise.all(readers)])
|
||||
await writes
|
||||
result.exitCode = completed.exitCode ?? null
|
||||
result.signal = completed.signal ?? null
|
||||
// Execa's timeout includes stream draining. Report the command's
|
||||
// runtime, so a timely exit does not become a timeout while draining.
|
||||
result.timedOut = completed.timedOut && !exitedWithinDeadline
|
||||
result.aborted = completed.isCanceled
|
||||
if (
|
||||
completed.failed &&
|
||||
completed.exitCode === undefined &&
|
||||
!result.timedOut &&
|
||||
!result.aborted &&
|
||||
!completed.signal
|
||||
)
|
||||
result.error = `Command execution failed (${completed.code ?? 'unknown error'}).`
|
||||
result.fullOutputPath = logPath
|
||||
} finally {
|
||||
clearTimeout(drainTimer)
|
||||
}
|
||||
} finally {
|
||||
await log.close()
|
||||
}
|
||||
} catch (error) {
|
||||
result.aborted = signal.aborted
|
||||
result.error = error instanceof Error ? error.message : 'Command execution failed.'
|
||||
} finally {
|
||||
// SRT's temporary mount resources are shared: clean up only after every
|
||||
// concurrent command has finished, and keep the proxies until shutdown.
|
||||
if (this.srtPath && --this.active === 0 && this.initialized)
|
||||
SandboxManager.cleanupAfterCommand()
|
||||
}
|
||||
return result
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
// Release builds embed SRT's native helpers through scripts/srt-bundle.ts.
|
||||
export const bundledSrtVendor: Record<string, string> | undefined = undefined
|
||||
@@ -0,0 +1,59 @@
|
||||
import { mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
import { loadSandbox } from './srt'
|
||||
|
||||
describe('runner SRT configuration', () => {
|
||||
let directory: string
|
||||
let settings: string
|
||||
const config = {
|
||||
network: { allowedDomains: [], deniedDomains: [] },
|
||||
filesystem: { denyRead: [], allowWrite: ['.pi'], denyWrite: [] },
|
||||
}
|
||||
beforeEach(async () => {
|
||||
directory = await mkdtemp(path.join(tmpdir(), 'shumai-srt-settings-'))
|
||||
settings = path.join(directory, 'sandbox.json')
|
||||
})
|
||||
afterEach(() => rm(directory, { recursive: true, force: true }))
|
||||
|
||||
it('validates upstream settings, supplies packaged helpers, and removes them on cleanup', async () => {
|
||||
await writeFile(settings, JSON.stringify(config))
|
||||
const sandbox = await loadSandbox(settings)
|
||||
const helper = sandbox.config.javaAgentJarPath!
|
||||
expect(sandbox.config).toMatchObject(config)
|
||||
expect((await readFile(helper)).subarray(0, 2).toString()).toBe('PK')
|
||||
if (process.platform === 'linux')
|
||||
expect(await stat(sandbox.config.seccomp!.applyPath!)).toHaveProperty(
|
||||
'size',
|
||||
expect.any(Number),
|
||||
)
|
||||
if (process.platform === 'win32')
|
||||
expect(await stat(sandbox.config.windows!.srtWin!.path!)).toHaveProperty(
|
||||
'size',
|
||||
expect.any(Number),
|
||||
)
|
||||
await sandbox.cleanup()
|
||||
await expect(stat(helper)).rejects.toThrow()
|
||||
})
|
||||
|
||||
it('preserves explicit upstream helper paths', async () => {
|
||||
await writeFile(
|
||||
settings,
|
||||
JSON.stringify({
|
||||
...config,
|
||||
javaAgentJarPath: '/custom/proxy.jar',
|
||||
seccomp: { applyPath: '/custom/seccomp' },
|
||||
windows: { srtWin: { path: 'C:\\custom\\srt-win.exe' } },
|
||||
}),
|
||||
)
|
||||
const sandbox = await loadSandbox(settings)
|
||||
try {
|
||||
expect(sandbox.config.javaAgentJarPath).toBe('/custom/proxy.jar')
|
||||
expect(sandbox.config.seccomp?.applyPath).toBe('/custom/seccomp')
|
||||
expect(sandbox.config.windows?.srtWin?.path).toBe('C:\\custom\\srt-win.exe')
|
||||
} finally {
|
||||
await sandbox.cleanup()
|
||||
}
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,59 @@
|
||||
import { createRequire } from 'node:module'
|
||||
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import path from 'node:path'
|
||||
import {
|
||||
SandboxRuntimeConfigSchema,
|
||||
type SandboxRuntimeConfig,
|
||||
} from '@anthropic-ai/sandbox-runtime'
|
||||
import { bundledSrtVendor } from './srt-vendor'
|
||||
|
||||
/** Load the upstream JSON format and supply the helpers bundled with this CLI. */
|
||||
export async function loadSandbox(configPath: string) {
|
||||
const config = SandboxRuntimeConfigSchema.parse(JSON.parse(await readFile(configPath, 'utf8')))
|
||||
const directory = await mkdtemp(path.join(tmpdir(), 'shumai-srt-'))
|
||||
const cleanup = () => rm(directory, { recursive: true, force: true })
|
||||
const asset = async (name: string) => {
|
||||
const encoded = bundledSrtVendor?.[name]
|
||||
const bytes =
|
||||
encoded !== undefined
|
||||
? Buffer.from(encoded, 'base64')
|
||||
: await readFile(
|
||||
path.join(
|
||||
path.dirname(
|
||||
createRequire(import.meta.url).resolve(
|
||||
'@anthropic-ai/sandbox-runtime/package.json',
|
||||
),
|
||||
),
|
||||
'vendor',
|
||||
name,
|
||||
),
|
||||
)
|
||||
const destination = path.join(directory, path.basename(name))
|
||||
await writeFile(destination, bytes, { mode: 0o500 })
|
||||
return destination
|
||||
}
|
||||
try {
|
||||
if (process.platform === 'linux' && !config.seccomp?.applyPath) {
|
||||
config.seccomp = {
|
||||
...config.seccomp,
|
||||
applyPath: await asset(`seccomp/${process.arch}/apply-seccomp`),
|
||||
}
|
||||
}
|
||||
if (process.platform === 'win32' && !config.windows?.srtWin?.path) {
|
||||
config.windows = {
|
||||
...config.windows,
|
||||
srtWin: {
|
||||
...config.windows?.srtWin,
|
||||
path: await asset(`srt-win/${process.arch}/srt-win.exe`),
|
||||
},
|
||||
}
|
||||
}
|
||||
if (!config.javaAgentJarPath)
|
||||
config.javaAgentJarPath = await asset('java-proxy-agent/srt-proxy-agent.jar')
|
||||
return { config: config satisfies SandboxRuntimeConfig, cleanup }
|
||||
} catch (error) {
|
||||
await cleanup()
|
||||
throw error
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@shumai/premiere-uxp",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"private": true,
|
||||
"type": "commonjs",
|
||||
"scripts": {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@shumai/transcode-app",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"bin": {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@shumai/web-app",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"dependencies": {
|
||||
|
||||
+16
-5
@@ -6,7 +6,7 @@ loadEnvConfig(process.cwd(), process.env.NODE_ENV !== 'production')
|
||||
import index from '@shumai/webui/index.html'
|
||||
|
||||
import { closeAgentSessions, initAgentWorkflows } from '@shumai/agent'
|
||||
import { app } from '@shumai/api'
|
||||
import { app, runnerWebsocket } from '@shumai/api'
|
||||
import { agentService } from '@shumai/core/src/agent/agent'
|
||||
import { migrateLegacyAgentAvatars } from '@shumai/core/src/agent/migration'
|
||||
import { assetService } from '@shumai/core/src/asset/asset'
|
||||
@@ -15,7 +15,7 @@ import { validateArchiveStorageConfig } from '@shumai/core/src/s3/archive-storag
|
||||
import { initTranscodeWorkflows } from '@shumai/transcode'
|
||||
import { workflowService } from '@shumai/workflow-core'
|
||||
|
||||
import { sandboxService } from '@shumai/core'
|
||||
import { runnerService } from '@shumai/core'
|
||||
import { autoArchiveJobService } from '@shumai/core/src/asset/auto-archive-job'
|
||||
import { authService } from '@shumai/core/src/auth/auth'
|
||||
import { notificationJobService } from '@shumai/core/src/notification/notification-job'
|
||||
@@ -81,12 +81,20 @@ if (regenCmdIndex !== -1) {
|
||||
|
||||
async function run() {
|
||||
validateArchiveStorageConfig()
|
||||
const runnerLease = await runnerService.startServerTopology()
|
||||
runnerLease.addEventListener(
|
||||
'abort',
|
||||
() => {
|
||||
console.error('Lost the runner server lease:', runnerLease.reason)
|
||||
process.exit(1)
|
||||
},
|
||||
{ once: true },
|
||||
)
|
||||
runnerLease.throwIfAborted()
|
||||
// Initialize workflows and activities for local executor mode
|
||||
initAgentWorkflows()
|
||||
initTranscodeWorkflows()
|
||||
await sandboxService.ensureInitialized().catch((err) => {
|
||||
console.error('Failed to eagerly initialize sandbox in web app:', err)
|
||||
})
|
||||
runnerService.startHeartbeatMonitor()
|
||||
|
||||
// Start services
|
||||
await metadataService.syncSystemFields().catch(console.error)
|
||||
@@ -210,6 +218,7 @@ async function run() {
|
||||
: 1024 * 1024 * 1024 * 20, // Default 20GB
|
||||
development: !isProd,
|
||||
fetch: app.fetch,
|
||||
websocket: runnerWebsocket,
|
||||
routes,
|
||||
})
|
||||
|
||||
@@ -218,10 +227,12 @@ async function run() {
|
||||
const shutdown = async () => {
|
||||
console.log('\nShutting down gracefully...')
|
||||
workflowService.stopStaleTaskRecovery()
|
||||
runnerService.stopHeartbeatMonitor()
|
||||
await closeAgentSessions()
|
||||
await workflowService.shutdownWorkers().catch(() => {})
|
||||
assetService.stopCleanupJob()
|
||||
server.stop(true)
|
||||
await runnerService.stopServerTopology().catch(console.error)
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
|
||||
@@ -32,24 +32,34 @@
|
||||
},
|
||||
"apps/cli": {
|
||||
"name": "@shumai/cli",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"bin": {
|
||||
"shumai-cli": "./src/index.ts",
|
||||
},
|
||||
"dependencies": {
|
||||
"@anthropic-ai/sandbox-runtime": "0.0.83",
|
||||
"@shumai/api": "workspace:*",
|
||||
"@shumai/dtos": "workspace:*",
|
||||
"@shumai/operations": "workspace:*",
|
||||
"@shumai/tableprinter": "workspace:*",
|
||||
"commander": "15.0.0",
|
||||
"execa": "10.2.1",
|
||||
"hono": "4.13.5",
|
||||
"ulid": "3.0.2",
|
||||
"ws": "8.22.2",
|
||||
"yauzl": "3.4.0",
|
||||
"zod": "4.5.4",
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/adm-zip": "0.5.8",
|
||||
"@types/ws": "8.18.2",
|
||||
"@types/yauzl": "3.4.0",
|
||||
"adm-zip": "0.6.1",
|
||||
},
|
||||
},
|
||||
"apps/premiere-uxp": {
|
||||
"name": "@shumai/premiere-uxp",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@shumai/api": "workspace:*",
|
||||
"@shumai/dtos": "workspace:*",
|
||||
@@ -103,7 +113,7 @@
|
||||
},
|
||||
"apps/transcode": {
|
||||
"name": "@shumai/transcode-app",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"bin": {
|
||||
"shumai-transcode": "./src/index.ts",
|
||||
},
|
||||
@@ -115,7 +125,7 @@
|
||||
},
|
||||
"apps/web": {
|
||||
"name": "@shumai/web-app",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@shumai/agent": "workspace:*",
|
||||
"@shumai/api": "workspace:*",
|
||||
@@ -135,9 +145,8 @@
|
||||
},
|
||||
"packages/agent": {
|
||||
"name": "@shumai/agent",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@anthropic-ai/sandbox-runtime": "0.0.74",
|
||||
"@earendil-works/chord": "1.5.0",
|
||||
"@earendil-works/pi-ai": "1.5.0",
|
||||
"@earendil-works/pi-durable": "1.5.0",
|
||||
@@ -160,7 +169,7 @@
|
||||
},
|
||||
"packages/api": {
|
||||
"name": "@shumai/api",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@better-auth/mcp": "1.7.12",
|
||||
"@earendil-works/pi-durable": "1.5.0",
|
||||
@@ -181,7 +190,7 @@
|
||||
},
|
||||
"packages/core": {
|
||||
"name": "@shumai/core",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@ai-sdk/alibaba": "2.0.38",
|
||||
"@ai-sdk/amazon-bedrock": "5.0.67",
|
||||
@@ -201,7 +210,6 @@
|
||||
"@ai-sdk/replicate": "3.0.34",
|
||||
"@ai-sdk/togetherai": "3.0.41",
|
||||
"@ai-sdk/xai": "4.0.49",
|
||||
"@anthropic-ai/sandbox-runtime": "0.0.74",
|
||||
"@aws-sdk/client-s3": "3.1121.0",
|
||||
"@aws-sdk/s3-request-presigner": "3.1121.0",
|
||||
"@better-auth/mcp": "1.7.12",
|
||||
@@ -246,7 +254,7 @@
|
||||
},
|
||||
"packages/db": {
|
||||
"name": "@shumai/db",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@prisma/adapter-pg": "7.10.0",
|
||||
"@prisma/client": "7.10.0",
|
||||
@@ -261,7 +269,7 @@
|
||||
},
|
||||
"packages/dtos": {
|
||||
"name": "@shumai/dtos",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@earendil-works/pi-ai": "1.5.0",
|
||||
"@earendil-works/pi-durable": "1.5.0",
|
||||
@@ -270,7 +278,7 @@
|
||||
},
|
||||
"packages/e2e": {
|
||||
"name": "@shumai/e2e",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@earendil-works/pi-ai": "1.5.0",
|
||||
"@earendil-works/pi-durable": "1.5.0",
|
||||
@@ -286,18 +294,18 @@
|
||||
},
|
||||
"packages/operations": {
|
||||
"name": "@shumai/operations",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"zod": "4.5.4",
|
||||
},
|
||||
},
|
||||
"packages/tableprinter": {
|
||||
"name": "@shumai/tableprinter",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
},
|
||||
"packages/transcode": {
|
||||
"name": "@shumai/transcode",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@shumai/core": "workspace:*",
|
||||
"@shumai/db": "workspace:*",
|
||||
@@ -308,7 +316,7 @@
|
||||
},
|
||||
"packages/webui": {
|
||||
"name": "@shumai/webui",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@base-ui/react": "1.7.0",
|
||||
"@better-auth/oauth-provider": "1.7.12",
|
||||
@@ -413,7 +421,7 @@
|
||||
},
|
||||
"packages/workflow-core": {
|
||||
"name": "@shumai/workflow-core",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"dependencies": {
|
||||
"@dbos-inc/dbos-sdk": "5.2.11",
|
||||
"@shumai/core": "workspace:*",
|
||||
@@ -476,7 +484,7 @@
|
||||
|
||||
"@alloc/quick-lru": ["@alloc/quick-lru@5.2.0", "", {}, "sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw=="],
|
||||
|
||||
"@anthropic-ai/sandbox-runtime": ["@anthropic-ai/sandbox-runtime@0.0.74", "", { "dependencies": { "@pondwader/socks5-server": "^1.0.10", "commander": "^12.1.0", "node-forge": "^1.4.0", "zod": "^3.24.1" }, "bin": { "srt": "dist/cli.js" } }, "sha512-k6xD9HPp4gCwIAQtAloTBSaKGGNQhH7o1ck0057iBxu3iHclM2vx+JzIXLDB9j4oaP9hwGcNlWSP5K7bgc1CIQ=="],
|
||||
"@anthropic-ai/sandbox-runtime": ["@anthropic-ai/sandbox-runtime@0.0.83", "", { "dependencies": { "@pondwader/socks5-server": "^1.0.10", "commander": "^12.1.0", "node-forge": "^1.4.0", "zod": "^3.25.0" }, "peerDependencies": { "@microsoft/mxc-sdk": "^1.0.1" }, "optionalPeers": ["@microsoft/mxc-sdk"], "bin": { "srt": "dist/cli.js" } }, "sha512-yEusORnVJrNseF/9fvjVYyk6tr79/aFbIB3T/X8p2puLZdaaiGNeYN50v0CE+1DT70v7XuzKOAnTivh1f5cksA=="],
|
||||
|
||||
"@anthropic-ai/sdk": ["@anthropic-ai/sdk@0.129.0", "", { "dependencies": { "json-schema-to-ts": "^3.1.1", "standardwebhooks": "^1.0.0" }, "peerDependencies": { "zod": "^3.25.0 || ^4.0.0" }, "optionalPeers": ["zod"], "bin": { "anthropic-ai-sdk": "bin/cli" } }, "sha512-MH7LB20kNpGLUpPTe2OG5XvL/KhX8HUO9XyBaFjgFk6TLS4Xjt0VPIuMKywe7POKKPslX+QxSlVok9e+8kG5Nw=="],
|
||||
|
||||
@@ -1100,6 +1108,8 @@
|
||||
|
||||
"@rolldown/pluginutils": ["@rolldown/pluginutils@1.0.1", "", {}, "sha512-2j9bGt5Jh8hj+vPtgzPtl72j0yRxHAyumoo6TNfAjsLB04UtpSvPbPcDcBMxz7n+9CYB0c1GxQFxYRg2jimqGw=="],
|
||||
|
||||
"@sec-ant/readable-stream": ["@sec-ant/readable-stream@0.4.1", "", {}, "sha512-831qok9r2t8AlxLko40y2ebgSDhenenCatLVeW/uBtnHPyhHOvG0C7TvfgecV+wHzIm5KUICgzmVpWS+IMEAeg=="],
|
||||
|
||||
"@shumai/agent": ["@shumai/agent@workspace:packages/agent"],
|
||||
|
||||
"@shumai/api": ["@shumai/api@workspace:packages/api"],
|
||||
@@ -1136,6 +1146,8 @@
|
||||
|
||||
"@sindresorhus/is": ["@sindresorhus/is@4.6.0", "", {}, "sha512-t09vSN3MdfsyCHoFcTRCH/iUtG7OJ0CsjzB8cjAmKc/va/kIgeDI/TxsigdncE/4be734m0cvIYwNaV4i2XqAw=="],
|
||||
|
||||
"@sindresorhus/merge-streams": ["@sindresorhus/merge-streams@4.0.0", "", {}, "sha512-tlqY9xq5ukxTUZBmoOp+m61cqwQD5pHJtFY3Mn8CA8ps6yghLH/Hw8UPdqg4OLmFW3IFlcXnQNmo/dh8HzXYIQ=="],
|
||||
|
||||
"@smithy/core": ["@smithy/core@3.33.3", "", { "dependencies": { "@smithy/types": "^4.17.2", "tslib": "^2.6.2" } }, "sha512-CsOeKq/9kA3y6VJHt+/+VTCtBaxJ4OTFpgrjIUhPpDIKxBci1k2bJaQASF2h/ELWrulGp+t97DZ0mevfAD8idg=="],
|
||||
|
||||
"@smithy/credential-provider-imds": ["@smithy/credential-provider-imds@4.5.2", "", { "dependencies": { "@smithy/core": "^3.33.2", "@smithy/types": "^4.17.2", "tslib": "^2.6.2" } }, "sha512-A9uSdn72ozbRUSit0eib0TW7nXuNPlaeM0zcGkJ+nE6tFcSDbnmtwoxbTCFBukVQcszDAyvsd7+rTduPTXpygg=="],
|
||||
@@ -1520,9 +1532,9 @@
|
||||
|
||||
"@types/whatwg-mimetype": ["@types/whatwg-mimetype@3.0.2", "", {}, "sha512-c2AKvDT8ToxLIOUlN51gTiHXflsfIFisS4pO7pDPoKouJCESkhZnEy623gwP9laCy5lnLDAw1vAzu2vM2YLOrA=="],
|
||||
|
||||
"@types/ws": ["@types/ws@8.18.1", "", { "dependencies": { "@types/node": "*" } }, "sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg=="],
|
||||
"@types/ws": ["@types/ws@8.18.2", "", { "dependencies": { "@types/node": "*" } }, "sha512-67MQl+fpWKVTT1NYdnmo3U4sc/xPo/zQBncVnI74qmQa0z/b+1g6iYqNmGCPbxO+zz2aklb08a0oHfegiVd0/w=="],
|
||||
|
||||
"@types/yauzl": ["@types/yauzl@2.10.3", "", { "dependencies": { "@types/node": "*" } }, "sha512-oJoftv0LSuaDZE3Le4DbKX+KS9G36NzOeSap90UIK0yMA/NhKJhqlSGtNDORNRaIbQfzjXDrQa0ytJ6mNRGz/Q=="],
|
||||
"@types/yauzl": ["@types/yauzl@3.4.0", "", { "dependencies": { "@types/node": "*" } }, "sha512-NRPn5w6h8dhcnmx3YIRQcqMywY/+nND/uOkJessedcrowO3C0AssHp3tMJpxKAwOhFOo0OV1y9VtsC5hbKKBAw=="],
|
||||
|
||||
"@typescript-eslint/eslint-plugin": ["@typescript-eslint/eslint-plugin@8.68.0", "", { "dependencies": { "@eslint-community/regexpp": "^4.12.2", "@typescript-eslint/scope-manager": "8.68.0", "@typescript-eslint/type-utils": "8.68.0", "@typescript-eslint/utils": "8.68.0", "@typescript-eslint/visitor-keys": "8.68.0", "ignore": "^7.0.5", "natural-compare": "^1.4.0", "ts-api-utils": "^2.5.0" }, "peerDependencies": { "@typescript-eslint/parser": "^8.68.0", "eslint": "^8.57.0 || ^9.0.0 || ^10.0.0", "typescript": ">=4.8.4 <6.1.0" } }, "sha512-WASHDpCm6qO5jj9g1a+8NiW5+GCkAyLReR56/4VruYmNgfUmqpxOfZ2Yfb8xGfJPWv5Qi6LSD8sXdces3vbp/Q=="],
|
||||
|
||||
@@ -1980,6 +1992,8 @@
|
||||
|
||||
"eventsource-parser": ["eventsource-parser@3.1.1", "", {}, "sha512-EKN1vKAMcZ8MlYMpaNuxN6R9yakzH6uajHcHVTqWJzvu5pWw9DyhbP35HH8MVBQ+dZjAfDxk+A8NiR9KWaXiyQ=="],
|
||||
|
||||
"execa": ["execa@10.2.1", "", { "dependencies": { "@sindresorhus/merge-streams": "^4.0.0", "figures": "^6.1.0", "get-stream": "^9.0.1", "human-signals": "^8.0.1", "is-plain-obj": "^4.1.0", "is-stream": "^4.0.1", "npm-run-path": "^6.0.0", "pretty-ms": "^9.3.1", "signal-exit": "^4.1.0", "strip-final-newline": "^4.0.0", "which-command": "^0.1.0", "yoctocolors": "^2.2.0" } }, "sha512-/Bofl3ZBh2Q4/PG0W63qFzP5vDAypi8Zc90UWTBzaDNIYiX7apPOd0N6L8hG8CojUdq4V51adu8BObP+iVsSTw=="],
|
||||
|
||||
"exiftool-vendored": ["exiftool-vendored@38.1.0", "", { "dependencies": { "@photostructure/tz-lookup": "^11.7.0", "@types/luxon": "^3.7.5", "batch-cluster": "^19.2.0", "he": "^1.2.0", "luxon": "^3.7.2" }, "optionalDependencies": { "exiftool-vendored.exe": "13.59.2", "exiftool-vendored.pl": "13.59.2" } }, "sha512-LIERqM/Qb0oqJlSswLFKWsxGVH81skp7omsX2BPKA/dEzfLP5syNO5KL7k5UlMCf+m0jqEXrKTQr/xV5vScPow=="],
|
||||
|
||||
"exiftool-vendored.exe": ["exiftool-vendored.exe@13.59.2", "", { "os": "win32" }, "sha512-U0l0CApIGO+AjpFYCiNCApQsMdlf6sqCyEl4wlKKqvwVAEVcvMRw91k7zj32OPFZE+/+XlLFsc6876MpOqBLiw=="],
|
||||
@@ -2024,6 +2038,8 @@
|
||||
|
||||
"fflate": ["fflate@0.8.3", "", {}, "sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA=="],
|
||||
|
||||
"figures": ["figures@6.1.0", "", { "dependencies": { "is-unicode-supported": "^2.0.0" } }, "sha512-d+l3qxjSesT4V7v2fh+QnmFnUWv9lSpjarhShNTgBOfA0ttejbQUAlHLitbjkoRiDulW0OPoQPYIGhIC8ohejg=="],
|
||||
|
||||
"file-entry-cache": ["file-entry-cache@8.0.0", "", { "dependencies": { "flat-cache": "^4.0.0" } }, "sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ=="],
|
||||
|
||||
"find-my-way": ["find-my-way@9.7.0", "", { "dependencies": { "fast-deep-equal": "^3.1.3", "fast-querystring": "^1.0.0", "safe-regex2": "^5.0.0" } }, "sha512-f2JHn75x2JlwUwLenZypgczR7YWMb/uO9BvUXtus+JMgkbIkLADd38cI4EiV+OQqrGo1Zlq6V8wnqMJ8e62wUQ=="],
|
||||
@@ -2080,7 +2096,7 @@
|
||||
|
||||
"get-proto": ["get-proto@1.0.1", "", { "dependencies": { "dunder-proto": "^1.0.1", "es-object-atoms": "^1.0.0" } }, "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g=="],
|
||||
|
||||
"get-stream": ["get-stream@5.2.0", "", { "dependencies": { "pump": "^3.0.0" } }, "sha512-nBF+F1rAZVCu/p7rjzgA+Yb4lfYXrpl7a6VmJrU8wF9I1CKvP/QwPNZHnOlwbTkY6dvtFIzFMSyQXbLoTQPRpA=="],
|
||||
"get-stream": ["get-stream@9.0.1", "", { "dependencies": { "@sec-ant/readable-stream": "^0.4.1", "is-stream": "^4.0.1" } }, "sha512-kVCxPF3vQM/N0B1PmoqVUqgHP+EeVjmZSQn+1oCRPxd2P21P2F19lIgbR3HBosbB1PUhOAoctJnfEn2GbN2eZA=="],
|
||||
|
||||
"get-symbol-description": ["get-symbol-description@1.1.0", "", { "dependencies": { "call-bound": "^1.0.3", "es-errors": "^1.3.0", "get-intrinsic": "^1.2.6" } }, "sha512-w9UMqWwJxHNOvoNzSJ2oPF5wvYcvP7jUvYzhp67yEhTi17ZDBBC1z9pTdGuzjD+EFIqLSYRweZjqfiPzQ06Ebg=="],
|
||||
|
||||
@@ -2140,6 +2156,8 @@
|
||||
|
||||
"https-proxy-agent": ["https-proxy-agent@9.1.0", "", { "dependencies": { "agent-base": "9.0.0", "debug": "^4.3.4", "proxy-agent-negotiate": "1.1.0" } }, "sha512-ag87y7cJJ9/3+GxFr8Oy4O5faDsGRGnBGsJj/YjOSsSx/5eadKLYTMPlzuR6obgoCDDm0abAAZitXXQkMOPSpA=="],
|
||||
|
||||
"human-signals": ["human-signals@8.0.1", "", {}, "sha512-eKCa6bwnJhvxj14kZk5NCPc6Hb6BdsU9DZcOnmQKSnO1VKrfV0zCvtttPZUsBvjmNDn8rpcJfpwSYnHBjc95MQ=="],
|
||||
|
||||
"iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="],
|
||||
|
||||
"ieee754": ["ieee754@1.2.1", "", {}, "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA=="],
|
||||
@@ -2214,7 +2232,7 @@
|
||||
|
||||
"is-shared-array-buffer": ["is-shared-array-buffer@1.0.4", "", { "dependencies": { "call-bound": "^1.0.3" } }, "sha512-ISWac8drv4ZGfwKl5slpHG9OwPNty4jOWPRIhBpxOoD+hqITiwuipOQ2bNthAzwA3B4fIjO4Nln74N0S9byq8A=="],
|
||||
|
||||
"is-stream": ["is-stream@2.0.1", "", {}, "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg=="],
|
||||
"is-stream": ["is-stream@4.0.1", "", {}, "sha512-Dnz92NInDqYckGEUJv689RbRiTSEHCQ7wOVeALbkOz999YpqT46yMRIGtSNl2iCL1waAZSx40+h59NV/EwzV/A=="],
|
||||
|
||||
"is-string": ["is-string@1.1.1", "", { "dependencies": { "call-bound": "^1.0.3", "has-tostringtag": "^1.0.2" } }, "sha512-BtEeSsoaQjlSPBemMQIrY1MY0uM6vnS1g5fmufYOtnxLGUZM2178PKbhsk7Ffv58IX+ZtcvoGwccYsh0PglkAA=="],
|
||||
|
||||
@@ -2222,6 +2240,8 @@
|
||||
|
||||
"is-typed-array": ["is-typed-array@1.1.15", "", { "dependencies": { "which-typed-array": "^1.1.16" } }, "sha512-p3EcsicXjit7SaskXHs1hA91QxgTw46Fv6EFKKGS5DRFLD8yKnohjF3hxoju94b/OcMZoQukzpPpBE9uLVKzgQ=="],
|
||||
|
||||
"is-unicode-supported": ["is-unicode-supported@2.1.0", "", {}, "sha512-mE00Gnza5EEB3Ds0HfMyllZzbBrmLOX3vfWoj9A9PEnTfratQ/BcaJOuMhnkhjXvb2+FkY3VuHqtAGpTPmglFQ=="],
|
||||
|
||||
"is-weakmap": ["is-weakmap@2.0.2", "", {}, "sha512-K5pXYOm9wqY1RgjpL3YTkF39tni1XajUIkawTLUo9EZEVUFga5gSQJF8nNS7ZwJQ02y+1YCNYcMh+HIf1ZqE+w=="],
|
||||
|
||||
"is-weakref": ["is-weakref@1.1.1", "", { "dependencies": { "call-bound": "^1.0.3" } }, "sha512-6i9mGWSlqzNMEqpCp93KwRS1uUOodk2OJ6b+sq7ZPDSy2WuI5NFIxp/254TytR8ftefexkWn5xNiHUNpPOfSew=="],
|
||||
@@ -2446,7 +2466,7 @@
|
||||
|
||||
"node-fetch": ["node-fetch@3.3.2", "", { "dependencies": { "data-uri-to-buffer": "^4.0.0", "fetch-blob": "^3.1.4", "formdata-polyfill": "^4.0.10" } }, "sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA=="],
|
||||
|
||||
"node-forge": ["node-forge@1.4.0", "", {}, "sha512-LarFH0+6VfriEhqMMcLX2F7SwSXeWwnEAJEsYm5QKWchiVYVvJyV9v7UDvUv+w5HO23ZpQTXDv/GxdDdMyOuoQ=="],
|
||||
"node-forge": ["node-forge@1.4.3", "", {}, "sha512-P6wzfh6t70avmV8FrVEAd/36sOPJQ+hkaJVX1Laap9J/77m7M0eyozDvc8SwnDf4P7MP94gwbAAo0rg3YVXT+g=="],
|
||||
|
||||
"node-releases": ["node-releases@2.0.54", "", {}, "sha512-YHs7BmmcsdAI5Ozuf8JZo6PT0mv2GIWC9vMfvUC3dp65M8hn7Ux8CPL+2oBI7juNuj9d0ndhTcznq2ODBps9cQ=="],
|
||||
|
||||
@@ -2454,6 +2474,8 @@
|
||||
|
||||
"normalize-path": ["normalize-path@3.0.0", "", {}, "sha512-6eZs5Ls3WtCisHWp9S2GUy8dqkpGi4BVSz3GaqiE6ezub0512ESztXUwUB6C6IKbQkY2Pnb/mD4WYojCRwcwLA=="],
|
||||
|
||||
"npm-run-path": ["npm-run-path@6.0.0", "", { "dependencies": { "path-key": "^4.0.0", "unicorn-magic": "^0.3.0" } }, "sha512-9qny7Z9DsQU8Ou39ERsPU4OZQlSTP47ShQzuKZ6PRXpYLtIFgl/DEBYEXKlvcEa+9tHVcK8CF81Y2V72qaZhWA=="],
|
||||
|
||||
"object-assign": ["object-assign@4.1.1", "", {}, "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg=="],
|
||||
|
||||
"object-inspect": ["object-inspect@1.13.4", "", {}, "sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew=="],
|
||||
@@ -2498,6 +2520,8 @@
|
||||
|
||||
"parse-entities": ["parse-entities@4.0.2", "", { "dependencies": { "@types/unist": "^2.0.0", "character-entities-legacy": "^3.0.0", "character-reference-invalid": "^2.0.0", "decode-named-character-reference": "^1.0.0", "is-alphanumerical": "^2.0.0", "is-decimal": "^2.0.0", "is-hexadecimal": "^2.0.0" } }, "sha512-GG2AQYWoLgL877gQIKeRPGO1xF9+eG1ujIb5soS5gPvLQ1y2o8FL90w2QWNdf9I361Mpp7726c+lj3U0qK1uGw=="],
|
||||
|
||||
"parse-ms": ["parse-ms@4.0.0", "", {}, "sha512-TXfryirbmq34y8QBwgqCVLi+8oA3oWx2eAnSn62ITyEhEYaWRlVZ2DvMM9eZbMs/RfxPu/PK/aBLyGj4IrqMHw=="],
|
||||
|
||||
"partial-json": ["partial-json@0.1.7", "", {}, "sha512-Njv/59hHaokb/hRUjce3Hdv12wd60MtM9Z5Olmn+nehe0QDAsRtRbJPvJ0Z91TusF0SuZRIvnM+S4l6EIP8leA=="],
|
||||
|
||||
"path-exists": ["path-exists@4.0.0", "", {}, "sha512-ak9Qy5Q7jYb2Wwcey5Fpvg2KoAc/ZIhLSLOSBmRmygPsGwkVVt0fZa0qrtMz+m6tJTAHfZQ8FnmB4MG4LWy7/w=="],
|
||||
@@ -2580,6 +2604,8 @@
|
||||
|
||||
"pretty-format": ["pretty-format@27.5.1", "", { "dependencies": { "ansi-regex": "^5.0.1", "ansi-styles": "^5.0.0", "react-is": "^17.0.1" } }, "sha512-Qb1gy5OrP5+zDf2Bvnzdl3jsTf1qXVMazbvCoKhtKqVs4/YK4ozX4gKQJJVyNe+cajNPn0KoC0MC3FUmaHWEmQ=="],
|
||||
|
||||
"pretty-ms": ["pretty-ms@9.3.1", "", { "dependencies": { "parse-ms": "^4.0.0" } }, "sha512-HzMy3Geq23nVALD/M2LliU+F+M+gVNsvkQWWqeBZ8HDiCgzo6YPJ/Omrmtq24EFrIsk0a3EkQGEd7bDOo+IhGA=="],
|
||||
|
||||
"prisma": ["prisma@7.10.0", "", { "dependencies": { "@prisma/config": "7.10.0", "@prisma/dev": "0.24.17", "@prisma/engines": "7.10.0", "@prisma/studio-core": "0.33.0", "mysql2": "3.15.3", "postgres": "3.4.7" }, "peerDependencies": { "better-sqlite3": ">=9.0.0", "typescript": ">=5.4.0" }, "optionalPeers": ["better-sqlite3", "typescript"], "bin": { "prisma": "build/index.js" } }, "sha512-o0ornyJOWgygVAzGCpr8PdXV8EJLHyVGDDUr/voBQt8Azzw8cYTByzzPGcA/m4tCkPcnJA8raEOv2CslsKhPEw=="],
|
||||
|
||||
"prisma-json-types-generator": ["prisma-json-types-generator@5.1.1", "", { "dependencies": { "@prisma/generator-helper": "^7.8.0", "semver": "^7.8.5", "try": "^1.0.3", "tslib": "^2.8.1", "typescript": "npm:typescript@^6.0.3" }, "peerDependencies": { "@prisma/client": "^7.8.0", "prisma": "^7.8.0" }, "bin": { "prisma-json-types-generator": "index.js" } }, "sha512-vtj68YkCla7bEnblC54xzKU3w+H/1uoKxtLU3F2eC9LL8j1Dz8GadVagKhQqQd03KGfHmO7o83Tf3yXxCKQwzQ=="],
|
||||
@@ -2742,7 +2768,7 @@
|
||||
|
||||
"siginfo": ["siginfo@2.0.0", "", {}, "sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g=="],
|
||||
|
||||
"signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="],
|
||||
"signal-exit": ["signal-exit@4.1.0", "", {}, "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw=="],
|
||||
|
||||
"skin-tone": ["skin-tone@2.0.0", "", { "dependencies": { "unicode-emoji-modifier-base": "^1.0.0" } }, "sha512-kUMbT1oBJCpgrnKoSr0o6wPtvRWT9W9UKvGLwfJYO2WuahZRHOpEyL1ckyMGgMWh0UdpmaoFqKKD29WTomNEGA=="],
|
||||
|
||||
@@ -2798,6 +2824,8 @@
|
||||
|
||||
"strip-ansi-cjs": ["strip-ansi@6.0.1", "", { "dependencies": { "ansi-regex": "^5.0.1" } }, "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A=="],
|
||||
|
||||
"strip-final-newline": ["strip-final-newline@4.0.0", "", {}, "sha512-aulFJcD6YK8V1G7iRB5tigAP4TsHBZZrOV8pjV++zdUwmeV8uzbY7yn6h9MswN62adStNZFuCIx4haBnRuMDaw=="],
|
||||
|
||||
"strip-json-comments": ["strip-json-comments@5.0.3", "", {}, "sha512-1tB5mhVo7U+ETBKNf92xT4hrQa3pm0MZ0PQvuDnWgAAGHDsfp4lPSpiS6psrSiet87wyGPh9ft6wmhOMQ0hDiw=="],
|
||||
|
||||
"style-to-js": ["style-to-js@1.1.21", "", { "dependencies": { "style-to-object": "1.0.14" } }, "sha512-RjQetxJrrUJLQPHbLku6U/ocGtzyjbJMP9lCNK7Ag0CNh690nSH8woqWH9u16nMjYBAok+i7JO1NP2pOy8IsPQ=="],
|
||||
@@ -2896,6 +2924,8 @@
|
||||
|
||||
"unicode-trie": ["unicode-trie@2.0.0", "", { "dependencies": { "pako": "^0.2.5", "tiny-inflate": "^1.0.0" } }, "sha512-x7bc76x0bm4prf1VLg79uhAzKw8DVboClSN5VxJuQ+LKDOVEW9CdH+VY7SP+vX7xCYQqzzgQpFqz15zeLvAtZQ=="],
|
||||
|
||||
"unicorn-magic": ["unicorn-magic@0.3.0", "", {}, "sha512-+QBBXBCvifc56fsbuxZQ6Sic3wqqc3WWaqxs58gvJrcOuN83HGTCwz3oS5phzU9LthRNE9VrJCFCLUgHeeFnfA=="],
|
||||
|
||||
"unified": ["unified@11.0.5", "", { "dependencies": { "@types/unist": "^3.0.0", "bail": "^2.0.0", "devlop": "^1.0.0", "extend": "^3.0.0", "is-plain-obj": "^4.0.0", "trough": "^2.0.0", "vfile": "^6.0.0" } }, "sha512-xKvGhPWw3k84Qjh8bI3ZeJjqnyadK+GEFtazSfZv/rKeTkTjOJho6mFqh2SM96iIcZokxiOpg78GazTSg8+KHA=="],
|
||||
|
||||
"unist-util-is": ["unist-util-is@6.0.1", "", { "dependencies": { "@types/unist": "^3.0.0" } }, "sha512-LsiILbtBETkDz8I9p1dQ0uyRUWuaQzd/cuEeS1hoRSyW5E5XGmTzlwY1OrNzzakGowI9Dr/I8HVaw4hTtnxy8g=="],
|
||||
@@ -2958,6 +2988,8 @@
|
||||
|
||||
"which-collection": ["which-collection@1.0.2", "", { "dependencies": { "is-map": "^2.0.3", "is-set": "^2.0.3", "is-weakmap": "^2.0.2", "is-weakset": "^2.0.3" } }, "sha512-K4jVyjnBdgvc86Y6BkaLZEN933SwYOuBFkdmBu9ZfkcAbdVbpITnDmjvZ/aQjRXQrv5EPkTnD1s39GiiqbngCw=="],
|
||||
|
||||
"which-command": ["which-command@0.1.0", "", { "bin": { "which-command": "cli.js" } }, "sha512-XZyoF5/5hZtXitIwzrU4NKK+Wtbb9aB9CezUEw2Q0wlYK8NUYQxC1rRXgNueYLtBAJwXIb+/tFVk4dozciNJMA=="],
|
||||
|
||||
"which-typed-array": ["which-typed-array@1.1.22", "", { "dependencies": { "available-typed-arrays": "^1.0.7", "call-bind": "^1.0.9", "call-bound": "^1.0.4", "for-each": "^0.3.5", "get-proto": "^1.0.1", "gopd": "^1.2.0", "has-tostringtag": "^1.0.2" } }, "sha512-fvO4ExWMFsqyhG3AiPAObMuY1lxaqgYcxbc49CNdWDDECOJNgQyvsOWVwbZc+qf3rzRtxojBK+CMEv0Ld5CYpw=="],
|
||||
|
||||
"why-is-node-running": ["why-is-node-running@2.3.0", "", { "dependencies": { "siginfo": "^2.0.0", "stackback": "0.0.2" }, "bin": { "why-is-node-running": "cli.js" } }, "sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w=="],
|
||||
@@ -2972,7 +3004,7 @@
|
||||
|
||||
"wrappy": ["wrappy@1.0.2", "", {}, "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ=="],
|
||||
|
||||
"ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
|
||||
"ws": ["ws@8.22.2", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-/rEZ2ejown2N5gxVTHUu9bVVC7ttu0/Tgzf5hbTOixR3Fs7mbbXgn2DbdbcKgJOmpQMtrIUWivtYuVMnp/W/yA=="],
|
||||
|
||||
"xtend": ["xtend@4.0.2", "", {}, "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ=="],
|
||||
|
||||
@@ -2986,12 +3018,14 @@
|
||||
|
||||
"yargs-parser": ["yargs-parser@21.1.1", "", {}, "sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw=="],
|
||||
|
||||
"yauzl": ["yauzl@2.10.0", "", { "dependencies": { "buffer-crc32": "~0.2.3", "fd-slicer": "~1.1.0" } }, "sha512-p4a9I6X6nu6IhoGmBqAcbJy1mlC4j27vEPZX9F4L4/vZT3Lyq1VkFHw/V/PUcB9Buo+DG3iHkT0x3Qya58zc3g=="],
|
||||
"yauzl": ["yauzl@3.4.0", "", { "dependencies": { "pend": "~1.2.0" } }, "sha512-jIH9yLR9wqr0wOS0TpBvo/g/2UgZH5qePVbjgRliiF0BYvOZyaBknKsF+x9Iht0O6sqgnB93rCICdOZFecJuDw=="],
|
||||
|
||||
"yjs": ["yjs@13.6.32", "", { "dependencies": { "lib0": "^0.2.99" } }, "sha512-lfiJIIC4Xayt5ItynE407ehlE03pCjeOc4hkR4yxxvvNJ4kuiN25B0g+Qp8XagYz361LLL7DCzR5bvFJ81QKtQ=="],
|
||||
|
||||
"yocto-queue": ["yocto-queue@0.1.0", "", {}, "sha512-rVksvsnNCdJ/ohGc6xgPwyN8eheCxsiLM8mxuE/t/mOVqJewPuO1miLpTHQiRgTKCLexL4MeAFVagts7HmNZ2Q=="],
|
||||
|
||||
"yoctocolors": ["yoctocolors@2.2.0", "", {}, "sha512-xYqdZFUK/VYazNl/oCDYN+3WloWQwMfZxBoiNt6qNyk+xfOdi598muWE42rNZFp1kNOiqW936q5RhUdnpqElSg=="],
|
||||
|
||||
"zeptomatch": ["zeptomatch@2.1.0", "", { "dependencies": { "grammex": "^3.1.11", "graphmatch": "^1.1.0" } }, "sha512-KiGErG2J0G82LSpniV0CtIzjlJ10E04j02VOudJsPyPwNZgGnRKQy7I1R7GMyg/QswnE4l7ohSGrQbQbjXPPDA=="],
|
||||
|
||||
"zip-stream": ["zip-stream@6.0.1", "", { "dependencies": { "archiver-utils": "^5.0.0", "compress-commons": "^6.0.2", "readable-stream": "^4.0.0" } }, "sha512-zK7YHHz4ZXpW89AHXUPbQVGKI7uvkd3hzusTdotCg1UxyaVtg0zFJSTfW/Dq5f7OBBVnq6cZIaC8Ti4hb6dtCA=="],
|
||||
@@ -3024,10 +3058,16 @@
|
||||
|
||||
"@dbos-inc/dbos-sdk/commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="],
|
||||
|
||||
"@dbos-inc/dbos-sdk/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
|
||||
|
||||
"@earendil-works/chord/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
|
||||
|
||||
"@earendil-works/pi-ai/@google/genai": ["@google/genai@2.21.0", "", { "dependencies": { "google-auth-library": "^10.3.0", "p-retry": "^4.6.2", "protobufjs": "^7.5.4", "ws": "^8.18.0" }, "peerDependencies": { "@modelcontextprotocol/sdk": "^1.25.2" }, "optionalPeers": ["@modelcontextprotocol/sdk"], "hasInstallScript": true }, "sha512-+PDtco2/Z0ONdzCGekCoCT+O1VJS9xJQNN4XzQpXG/t3El/SWWMkCWlFRO1KmivOHPa4Q0VjUYu1HBKCZ/v33Q=="],
|
||||
|
||||
"@eslint-community/eslint-utils/eslint-visitor-keys": ["eslint-visitor-keys@3.4.3", "", {}, "sha512-wpc+LXeiyiisxPlEkUzU6svyS1frIO3Mgxj1fdy7Pm8Ygzguax2N3Fa/D/ag1WqbOprdI+uY6wMUl8/a2G+iag=="],
|
||||
|
||||
"@google/genai/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
|
||||
|
||||
"@grpc/grpc-js/@grpc/proto-loader": ["@grpc/proto-loader@0.8.1", "", { "dependencies": { "lodash.camelcase": "^4.3.0", "long": "^5.0.0", "protobufjs": "^7.5.5", "yargs": "^17.7.2" }, "bin": { "proto-loader-gen-types": "build/bin/proto-loader-gen-types.js" } }, "sha512-wtF6h+DY6M3YaDBPAmvuuA6jV8Sif9MjtOI5euKFWRgCDl5PeDpPsHR9u2l6St5ceY8AZgoNDww5+HvEsXFsGg=="],
|
||||
|
||||
"@inlang/paraglide-js/commander": ["commander@11.1.0", "", {}, "sha512-yPVavfyCcRhmorC7rWlkHn15b4wDVgVmBA7kV4QVBsF7kv/9TKJAbAXVTxvTnwP8HHKjRCJDClKbciiYS7p0DQ=="],
|
||||
@@ -3054,6 +3094,8 @@
|
||||
|
||||
"@reduxjs/toolkit/reselect": ["reselect@5.3.0", "", {}, "sha512-XGoLeRAVzUTcJ1qkxPQhDJyIZ5d6zzZD9nT7AEZOaaU9UbWclhycElmhO+VD5bFeLuzhPBaOV2oXC8uG35ZSpg=="],
|
||||
|
||||
"@shumai/cli/adm-zip": ["adm-zip@0.6.1", "", {}, "sha512-Xwrja8nx9e5o2N1my4DsKCeKpdrnACyr1wtbPxBDgGzKzKyE9kRtBFA8mWldI+RVlD7CBZNWY/wQ2+ydwOR6kQ=="],
|
||||
|
||||
"@shumai/webui/@types/react-dom": ["@types/react-dom@19.2.5", "", { "peerDependencies": { "@types/react": "^19.2.0" } }, "sha512-fMPwH9v7r/pp43yUd2/Mbiex5KouJwwR3dzHkhLREUC6764VyDsqxhAxv6OFEYR1RhjOyD1naqba8ECDBe7ZQg=="],
|
||||
|
||||
"@smithy/node-http-handler/@smithy/types": ["@smithy/types@4.19.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-r7jh49VJxGerfAcTQA6gXcKc+98zOp/tqRwzYjgOE+iSQsP6cEU1hq2QzbuipmP68QtYdY9wKEhiCQZIzHgZ4Q=="],
|
||||
@@ -3200,6 +3242,8 @@
|
||||
|
||||
"@visx/vendor/d3-format": ["d3-format@3.1.0", "", {}, "sha512-YyUI6AEuY/Wpt8KWLgZHsIU86atmikuoOmCfommt0LYHiQSPjvX2AcFc38PX0CBpr2RCyZhjex+NS/LPOv6YqA=="],
|
||||
|
||||
"archiver-utils/is-stream": ["is-stream@2.0.1", "", {}, "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg=="],
|
||||
|
||||
"bcrypt-pbkdf/tweetnacl": ["tweetnacl@0.14.5", "", {}, "sha512-KXXFFdAbFXY4geFIwoyNK+f5Z1b7swfXABfL7HXCmoIWMKU3dmS26672A4EeQtDzLKy7SXmfBu51JolvEKwtGA=="],
|
||||
|
||||
"better-call/@better-fetch/fetch": ["@better-fetch/fetch@1.3.1", "", {}, "sha512-ABkD1WhyfPZprKRQI3bhATjeiFuNWC9PXhfGWqL+sg/gKrM977oFrYkdb4msM3hgUGonr7KlOsOFT5TU2rht9g=="],
|
||||
@@ -3208,13 +3252,19 @@
|
||||
|
||||
"bl/readable-stream": ["readable-stream@3.6.2", "", { "dependencies": { "inherits": "^2.0.3", "string_decoder": "^1.1.1", "util-deprecate": "^1.0.1" } }, "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA=="],
|
||||
|
||||
"compress-commons/is-stream": ["is-stream@2.0.1", "", {}, "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg=="],
|
||||
|
||||
"docker-modem/readable-stream": ["readable-stream@3.6.2", "", { "dependencies": { "inherits": "^2.0.3", "string_decoder": "^1.1.1", "util-deprecate": "^1.0.1" } }, "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA=="],
|
||||
|
||||
"dockerode/tar-fs": ["tar-fs@2.1.5", "", { "dependencies": { "chownr": "^1.1.1", "mkdirp-classic": "^0.5.2", "pump": "^3.0.0", "tar-stream": "^2.1.4" } }, "sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw=="],
|
||||
|
||||
"eslint-plugin-react/minimatch": ["minimatch@3.1.5", "", { "dependencies": { "brace-expansion": "^1.1.7" } }, "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w=="],
|
||||
|
||||
"foreground-child/signal-exit": ["signal-exit@4.1.0", "", {}, "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw=="],
|
||||
"extract-zip/@types/yauzl": ["@types/yauzl@2.10.3", "", { "dependencies": { "@types/node": "*" } }, "sha512-oJoftv0LSuaDZE3Le4DbKX+KS9G36NzOeSap90UIK0yMA/NhKJhqlSGtNDORNRaIbQfzjXDrQa0ytJ6mNRGz/Q=="],
|
||||
|
||||
"extract-zip/get-stream": ["get-stream@5.2.0", "", { "dependencies": { "pump": "^3.0.0" } }, "sha512-nBF+F1rAZVCu/p7rjzgA+Yb4lfYXrpl7a6VmJrU8wF9I1CKvP/QwPNZHnOlwbTkY6dvtFIzFMSyQXbLoTQPRpA=="],
|
||||
|
||||
"extract-zip/yauzl": ["yauzl@2.10.0", "", { "dependencies": { "buffer-crc32": "~0.2.3", "fd-slicer": "~1.1.0" } }, "sha512-p4a9I6X6nu6IhoGmBqAcbJy1mlC4j27vEPZX9F4L4/vZT3Lyq1VkFHw/V/PUcB9Buo+DG3iHkT0x3Qya58zc3g=="],
|
||||
|
||||
"gaxios/https-proxy-agent": ["https-proxy-agent@7.0.6", "", { "dependencies": { "agent-base": "^7.1.2", "debug": "4" } }, "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw=="],
|
||||
|
||||
@@ -3222,6 +3272,10 @@
|
||||
|
||||
"glob/minimatch": ["minimatch@9.0.9", "", { "dependencies": { "brace-expansion": "^2.0.2" } }, "sha512-OBwBN9AL4dqmETlpS2zasx+vTeWclWzkblfZk7KTA5j3jeOONz/tRCnZomUyvNg83wL5Zv9Ss6HMJXAgL8R2Yg=="],
|
||||
|
||||
"happy-dom/@types/ws": ["@types/ws@8.18.1", "", { "dependencies": { "@types/node": "*" } }, "sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg=="],
|
||||
|
||||
"happy-dom/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
|
||||
|
||||
"its-fine/@types/react-reconciler": ["@types/react-reconciler@0.28.9", "", { "peerDependencies": { "@types/react": "*" } }, "sha512-HHM3nxyUZ3zAylX8ZEyrDNd2XZOnQ0D5XfunJF5FLQnZbHHYq4UWvW1QfelQNXv1ICNkwYhfxjwfnqivYB6bFg=="],
|
||||
|
||||
"lazystream/readable-stream": ["readable-stream@2.3.8", "", { "dependencies": { "core-util-is": "~1.0.0", "inherits": "~2.0.3", "isarray": "~1.0.0", "process-nextick-args": "~2.0.0", "safe-buffer": "~5.1.1", "string_decoder": "~1.1.1", "util-deprecate": "~1.0.1" } }, "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA=="],
|
||||
@@ -3232,6 +3286,8 @@
|
||||
|
||||
"lit-element/@lit/reactive-element": ["@lit/reactive-element@1.6.3", "", { "dependencies": { "@lit-labs/ssr-dom-shim": "^1.0.0" } }, "sha512-QuTgnG52Poic7uM1AN5yJ09QMe0O28e10XzSvWDz02TJiiKee4stsiownEIadWm8nYzyDAyT+gKzUoZmiWQtsQ=="],
|
||||
|
||||
"npm-run-path/path-key": ["path-key@4.0.0", "", {}, "sha512-haREypq7xkM7ErfgIyA0z+Bj4AGKlMSdlQE2jvJo6huWD1EdkKYV+G/T4nq0YEF2vgTT8kqMFKo1uHn950r4SQ=="],
|
||||
|
||||
"p-retry/retry": ["retry@0.13.1", "", {}, "sha512-XQBQ3I8W1Cge0Seh+6gjj03LbmRFWuoszgK9ooCpwYIrhhoO80pfq4cUkU5DkknwfOfFteRwlZ56PYOGYyFWdg=="],
|
||||
|
||||
"parse-entities/@types/unist": ["@types/unist@2.0.11", "", {}, "sha512-CmBKiL6NNo/OqgmMn95Fk9Whlp2mtvIv+KNpQKN2F4SjvrEesubTRWGYSg+BnWZOnlCaSTU1sMpsBOzgbYhnsA=="],
|
||||
@@ -3250,6 +3306,8 @@
|
||||
|
||||
"prop-types/react-is": ["react-is@16.13.1", "", {}, "sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ=="],
|
||||
|
||||
"proper-lockfile/signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="],
|
||||
|
||||
"readdir-glob/minimatch": ["minimatch@5.1.9", "", { "dependencies": { "brace-expansion": "^2.0.1" } }, "sha512-7o1wEA2RyMP7Iu7GNba9vc0RWWGACJOCZBJX2GJWip0ikV+wcOsgVuY9uE8CPiyQhkGFSlhuSkZPavN7u1c2Fw=="],
|
||||
|
||||
"sharp/semver": ["semver@7.8.5", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA=="],
|
||||
@@ -3262,8 +3320,6 @@
|
||||
|
||||
"wrap-ansi-cjs/ansi-styles": ["ansi-styles@4.3.0", "", { "dependencies": { "color-convert": "^2.0.1" } }, "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg=="],
|
||||
|
||||
"yauzl/buffer-crc32": ["buffer-crc32@0.2.13", "", {}, "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ=="],
|
||||
|
||||
"@aws-sdk/client-bedrock-runtime/@aws-sdk/credential-provider-node/@aws-sdk/credential-provider-env": ["@aws-sdk/credential-provider-env@3.972.80", "", { "dependencies": { "@aws-sdk/core": "^3.979.1", "@aws-sdk/types": "^3.975.0", "@smithy/core": "^3.38.0", "@smithy/types": "^4.19.0", "tslib": "^2.6.2" } }, "sha512-uO5TiSvb6j1w9RTe5z28OaOmsrOxVQUPaC5+wZl50RfOXcrYD/KgnS8I9lfyGXc7sGbkCopK6VnhNIAoLqfewQ=="],
|
||||
|
||||
"@aws-sdk/client-bedrock-runtime/@aws-sdk/credential-provider-node/@aws-sdk/credential-provider-http": ["@aws-sdk/credential-provider-http@3.972.82", "", { "dependencies": { "@aws-sdk/core": "^3.979.1", "@aws-sdk/types": "^3.975.0", "@smithy/core": "^3.38.0", "@smithy/fetch-http-handler": "^5.8.0", "@smithy/node-http-handler": "^4.14.0", "@smithy/types": "^4.19.0", "tslib": "^2.6.2" } }, "sha512-vQzlDHlC24Yz42p+rT4NBoP/5VdetkBoujbOLq9h6QYWC8THuZSPYnXIR9OP3XcOrdMgPBBipOrdssfxnIiWUg=="],
|
||||
@@ -3282,6 +3338,8 @@
|
||||
|
||||
"@aws-sdk/client-bedrock-runtime/@aws-sdk/credential-provider-node/@smithy/types": ["@smithy/types@4.19.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-r7jh49VJxGerfAcTQA6gXcKc+98zOp/tqRwzYjgOE+iSQsP6cEU1hq2QzbuipmP68QtYdY9wKEhiCQZIzHgZ4Q=="],
|
||||
|
||||
"@earendil-works/pi-ai/@google/genai/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
|
||||
|
||||
"@isaacs/cliui/string-width/emoji-regex": ["emoji-regex@9.2.2", "", {}, "sha512-L18DaJsXSUk2+42pv8mLs5jJT2hqFkFE4j21wOmgbUqsZ2hL72NsUU785g9RXgo3s0ZNgVl42TiHp3ZtOv/Vyg=="],
|
||||
|
||||
"@isaacs/cliui/strip-ansi/ansi-regex": ["ansi-regex@6.3.0", "", {}, "sha512-WpDfL7NO6j7tH88IDBNVdUJxDh9nmCteAVW9dsep846XdwF4naCBK+/tGLX3KJgcpgMRXCFlTM2hKGoK9FsdrQ=="],
|
||||
@@ -3384,6 +3442,8 @@
|
||||
|
||||
"eslint-plugin-react/minimatch/brace-expansion": ["brace-expansion@1.1.18", "", { "dependencies": { "balanced-match": "^1.0.0", "concat-map": "0.0.1" } }, "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw=="],
|
||||
|
||||
"extract-zip/yauzl/buffer-crc32": ["buffer-crc32@0.2.13", "", {}, "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ=="],
|
||||
|
||||
"gaxios/https-proxy-agent/agent-base": ["agent-base@7.1.4", "", {}, "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ=="],
|
||||
|
||||
"glob/minimatch/brace-expansion": ["brace-expansion@2.1.4", "", { "dependencies": { "balanced-match": "^1.0.0" } }, "sha512-hGfVzPxthbf3+2yjg/RBs60cB0FhqBS/zvdV/4wn4/BmN0bNMMHPc4V/BbFieqf1TKAGGAHnY4eSjajCl0f2Xg=="],
|
||||
|
||||
@@ -21,11 +21,6 @@ services:
|
||||
|
||||
shumai:
|
||||
image: crpi-000nkafzu6xvpwsc.cn-hongkong.personal.cr.aliyuncs.com/shumai-one/shumai:latest
|
||||
cap_add:
|
||||
- SYS_ADMIN
|
||||
security_opt:
|
||||
- apparmor=unconfined
|
||||
- seccomp=unconfined
|
||||
container_name: shumai_app
|
||||
ports:
|
||||
- '3000:3000'
|
||||
@@ -51,9 +46,6 @@ services:
|
||||
BETTER_AUTH_SECRET: qSxs9DxzHDZBbeeTNPEwBuspYwipBqz5Gk5XdBjNhWw=
|
||||
BETTER_AUTH_URL: http://localhost:3000
|
||||
|
||||
# Sandbox Configuration
|
||||
ENABLE_WEAKER_NESTED_SANDBOX: 'true'
|
||||
|
||||
# Embedding Configuration
|
||||
EMBEDDING_PROVIDER: gemma-embedding-2-openai
|
||||
GEMMA_EMBEDDING_OPENAI_BASE_URL: http://embedding-server:9379/v1
|
||||
|
||||
@@ -31,11 +31,6 @@ services:
|
||||
|
||||
shumai:
|
||||
image: crpi-000nkafzu6xvpwsc.cn-hongkong.personal.cr.aliyuncs.com/shumai-one/shumai:latest
|
||||
cap_add:
|
||||
- SYS_ADMIN
|
||||
security_opt:
|
||||
- apparmor=unconfined
|
||||
- seccomp=unconfined
|
||||
container_name: shumai_app
|
||||
ports:
|
||||
- '3000:3000'
|
||||
@@ -61,9 +56,6 @@ services:
|
||||
BETTER_AUTH_SECRET: qSxs9DxzHDZBbeeTNPEwBuspYwipBqz5Gk5XdBjNhWw=
|
||||
BETTER_AUTH_URL: http://localhost:3000
|
||||
|
||||
# Sandbox Configuration
|
||||
ENABLE_WEAKER_NESTED_SANDBOX: 'true'
|
||||
|
||||
# Embedding Configuration
|
||||
EMBEDDING_PROVIDER: gemma-embedding-2-openai
|
||||
GEMMA_EMBEDDING_OPENAI_BASE_URL: http://embedding-server:9379/v1
|
||||
|
||||
@@ -16,11 +16,6 @@ services:
|
||||
|
||||
shumai:
|
||||
image: crpi-000nkafzu6xvpwsc.cn-hongkong.personal.cr.aliyuncs.com/shumai-one/shumai:latest
|
||||
cap_add:
|
||||
- SYS_ADMIN
|
||||
security_opt:
|
||||
- apparmor=unconfined
|
||||
- seccomp=unconfined
|
||||
container_name: shumai_app
|
||||
ports:
|
||||
- '3000:3000'
|
||||
@@ -46,9 +41,6 @@ services:
|
||||
BETTER_AUTH_SECRET: qSxs9DxzHDZBbeeTNPEwBuspYwipBqz5Gk5XdBjNhWw=
|
||||
BETTER_AUTH_URL: http://localhost:3000
|
||||
|
||||
# Sandbox Configuration
|
||||
ENABLE_WEAKER_NESTED_SANDBOX: 'true'
|
||||
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
|
||||
@@ -21,11 +21,6 @@ services:
|
||||
|
||||
shumai:
|
||||
image: shumaione/shumai:latest
|
||||
cap_add:
|
||||
- SYS_ADMIN
|
||||
security_opt:
|
||||
- apparmor=unconfined
|
||||
- seccomp=unconfined
|
||||
container_name: shumai_app
|
||||
ports:
|
||||
- '3000:3000'
|
||||
@@ -51,9 +46,6 @@ services:
|
||||
BETTER_AUTH_SECRET: qSxs9DxzHDZBbeeTNPEwBuspYwipBqz5Gk5XdBjNhWw=
|
||||
BETTER_AUTH_URL: http://localhost:3000
|
||||
|
||||
# Sandbox Configuration
|
||||
ENABLE_WEAKER_NESTED_SANDBOX: 'true'
|
||||
|
||||
# Embedding Configuration
|
||||
EMBEDDING_PROVIDER: gemma-embedding-2-openai
|
||||
GEMMA_EMBEDDING_OPENAI_BASE_URL: http://embedding-server:9379/v1
|
||||
|
||||
@@ -31,11 +31,6 @@ services:
|
||||
|
||||
shumai:
|
||||
image: shumaione/shumai:latest
|
||||
cap_add:
|
||||
- SYS_ADMIN
|
||||
security_opt:
|
||||
- apparmor=unconfined
|
||||
- seccomp=unconfined
|
||||
container_name: shumai_app
|
||||
ports:
|
||||
- '3000:3000'
|
||||
@@ -61,9 +56,6 @@ services:
|
||||
BETTER_AUTH_SECRET: qSxs9DxzHDZBbeeTNPEwBuspYwipBqz5Gk5XdBjNhWw=
|
||||
BETTER_AUTH_URL: http://localhost:3000
|
||||
|
||||
# Sandbox Configuration
|
||||
ENABLE_WEAKER_NESTED_SANDBOX: 'true'
|
||||
|
||||
# Embedding Configuration
|
||||
EMBEDDING_PROVIDER: gemma-embedding-2-openai
|
||||
GEMMA_EMBEDDING_OPENAI_BASE_URL: http://embedding-server:9379/v1
|
||||
|
||||
@@ -16,11 +16,6 @@ services:
|
||||
|
||||
shumai:
|
||||
image: shumaione/shumai:latest
|
||||
cap_add:
|
||||
- SYS_ADMIN
|
||||
security_opt:
|
||||
- apparmor=unconfined
|
||||
- seccomp=unconfined
|
||||
container_name: shumai_app
|
||||
ports:
|
||||
- '3000:3000'
|
||||
@@ -46,9 +41,6 @@ services:
|
||||
BETTER_AUTH_SECRET: qSxs9DxzHDZBbeeTNPEwBuspYwipBqz5Gk5XdBjNhWw=
|
||||
BETTER_AUTH_URL: http://localhost:3000
|
||||
|
||||
# Sandbox Configuration
|
||||
ENABLE_WEAKER_NESTED_SANDBOX: 'true'
|
||||
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
|
||||
@@ -3,6 +3,23 @@ title: "Changelog"
|
||||
description: "Recent updates, new features, and improvements in Shumai"
|
||||
---
|
||||
|
||||
<Update label="v0.7.0" description="November 18, 2026">
|
||||
|
||||
### Added
|
||||
|
||||
- **i18n**: Add Korean as a display language, selectable on the sign-up page and in team settings.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **cli**: Fix release publication failing to find staged standalone downloads.
|
||||
|
||||
### Changed
|
||||
|
||||
- **agent**: ⚠️ **Breaking Change**: Replaced the built-in server-side Bash sandbox with external Runners and removed the team-level sandbox domain allowlist settings. The Bash tool is now disabled by default. If your Agent or Skills require Bash execution, you'll need to set up and connect an external Runner using shumai-cli. The Shumai CLI includes built-in sandbox support, allowing you to configure filesystem access and network domain allowlists for Bash execution directly on the Runner. See [Agent Runners](https://docs.shumai.one/using-shumai/agent-sandbox) for setup instructions and configuration details. The Docker Compose configuration has also been updated to improve security following the removal of the server-side sandbox. We strongly recommend upgrading to the latest Shumai image and updating your Docker Compose file to apply these security improvements.
|
||||
- **cli**: Show copyable commands and shell configuration instructions when the standalone CLI installation directory is missing from PATH.
|
||||
|
||||
</Update>
|
||||
|
||||
<Update label="v0.6.6" description="November 13, 2026">
|
||||
|
||||
### Added
|
||||
|
||||
@@ -72,6 +72,8 @@ When running multiple replicas of the transcode workers or main servers across m
|
||||
- **Custom Worker ID**: Set `SHUMAI_WORKERS_ID` (or `SHUMAI_WORKER_ID`) if you wish to assign an explicit identifier.
|
||||
- **Kubernetes**: In Kubernetes Pods, set `POD_NAME` (via Downward API `fieldRef: { fieldPath: metadata.name }`) to provide stable, unique per-pod identity across pod restarts.
|
||||
|
||||
Agent runners require a single main server per database. With runners configured, stop the old main server before starting its replacement. For Kubernetes, use a single main server replica with the Recreate deployment strategy. Transcode workers can still scale independently.
|
||||
|
||||
---
|
||||
|
||||
## Scalable Deployment on Kubernetes & AWS EKS
|
||||
|
||||
@@ -1,45 +1,101 @@
|
||||
---
|
||||
title: Agent Sandbox
|
||||
description: Secure execution environment and network domain restrictions for AI agent scripts.
|
||||
title: Agent Runners
|
||||
description: Connect an optional command runner and configure sandboxing on its machine.
|
||||
---
|
||||
|
||||
Running automated scripts poses security risks. Shumai resolves this by executing all agent skills inside a secure, isolated **Sandbox** environment.
|
||||
Agents use Shumai's built-in tools without a runner. To execute shell commands or download assets into a shell workspace, connect a **runner** on a machine you control. The runner opens an authenticated outbound WebSocket connection; no inbound port is needed.
|
||||
|
||||
---
|
||||
Agent runners require one main Shumai server per database. Multiple main servers can share a database while it has no configured runners. Stop the other main servers before creating a runner. Transcode workers can run separately.
|
||||
|
||||
## Sandbox Security
|
||||
## Connect a runner
|
||||
|
||||
The sandbox restricts the execution process from accessing the host filesystem, system processes, and internal networks:
|
||||
1. In **Settings → Agent Runners**, create a runner and copy its one-time key. Only team owners can manage runners and rotate their keys.
|
||||
2. Install [Shumai CLI](https://github.com/shumaiOne/shumai/tree/main/apps/cli#installation) on macOS, Linux, or Windows.
|
||||
3. Start it in a dedicated, existing workspace:
|
||||
|
||||
- **Isolation**: Scripts run in a virtualized workspace container that prevents them from modifying critical system files or reading other project data.
|
||||
- **Network Bridging**: External network calls from inside the sandbox are strictly monitored.
|
||||
```bash
|
||||
export SHUMAI_API_SERVER="https://shumai.example.com"
|
||||
export SHUMAI_RUNNER_KEY="your-runner-key"
|
||||
shumai-cli runner start --cwd /path/to/workspace
|
||||
```
|
||||
|
||||
### File System Restrictions
|
||||
On Windows, set the variables in PowerShell:
|
||||
|
||||
To protect your workspace environments and sensitive data, the sandbox enforces strict file-level access controls:
|
||||
```powershell
|
||||
$env:SHUMAI_API_SERVER = "https://shumai.example.com"
|
||||
$env:SHUMAI_RUNNER_KEY = "your-runner-key"
|
||||
shumai-cli runner start --cwd C:\workspace
|
||||
```
|
||||
|
||||
- **Read Access**: The sandbox is allowed to read files within the workspace, with a strict exception for sensitive environment and certificate files. It cannot read: `['.env', '.env.*', '*.pem', '*.key']`.
|
||||
- **Write Access**: The sandbox is heavily restricted and may only write to the `.pi` folder located in the current working directory (cwd). Even within this allowed directory, the sandbox is explicitly blocked from writing or creating sensitive files matching: `['.env', '.env.*', '*.pem', '*.key']`.
|
||||
Use a runner key, not a personal API token. Prefer the environment variable over `--key` to keep the key out of shell history. The runner stays in the foreground; Ctrl+C stops it and its active commands. Temporary connection losses trigger reconnection. Revoked or replaced connections stop instead of reconnecting.
|
||||
|
||||
---
|
||||
The first runner becomes the team default. Each agent can use that default or a dedicated runner selected in its settings. Without a configured runner, `bash` and `download_asset` are hidden. With one configured, authorized editors and reviewers can run commands immediately without reading a skill first. Disabled-tool policies still apply. An offline runner produces an execution error.
|
||||
|
||||
## Domain Whitelisting
|
||||
The `bash` tool uses `/bin/sh` on macOS and Linux. On Windows, the runner looks for Git Bash in the standard installation directories, then for `bash.exe` on PATH, and falls back to `cmd.exe`. The runner prints its selected shell at startup and reports it to the agent so it can use compatible syntax. For Bash with SRT on Windows, use Git Bash in place of the legacy WSL Bash launcher.
|
||||
|
||||
By default, Network Sandbox is **disabled** (allowing all outbound network requests). Team Owners can toggle Network Sandbox **ON** in Sandbox Settings to enforce strict domain whitelisting:
|
||||
Commands start in the workspace directory. Skills, downloaded assets, generated files, and output logs live under `.pi`. Commands inherit baseline OS settings and explicit skill variables, rather than the runner's full environment, including its runner key and personal API token.
|
||||
|
||||
### Allowed Domains
|
||||
Everyone with shell access to the same runner can read files left in its workspace, including assets and logs from other projects. Shumai project permissions apply when assets are downloaded. Share a runner only among users who may read all files in that workspace. For separate trust groups, use separate runner accounts or virtual machines and restrict access to the agents bound to each runner.
|
||||
|
||||
A list of domains that sandboxed scripts are permitted to access.
|
||||
## Optional SRT sandbox
|
||||
|
||||
- Default whitelisted domains include standard package registries (like npm or python package hosts) and repository hosts (like `github.com`).
|
||||
- If a script needs to call an external API (e.g. `api.slack.com`), the administrator must add it to the allowed list.
|
||||
Without `--srt`, commands have the runner operating-system user's permissions. Connect only to a trusted Shumai server; use a dedicated container, VM, or account when appropriate.
|
||||
|
||||
### Pending Domains
|
||||
[Anthropic Sandbox Runtime](https://github.com/anthropics/sandbox-runtime) is bundled as a library in Shumai CLI. Install its platform prerequisites on the runner machine (Linux: bubblewrap, socat, and ripgrep; macOS: ripgrep), then start with its JSON settings:
|
||||
|
||||
When an agent script attempts to reach a domain that is not currently authorized:
|
||||
```bash
|
||||
shumai-cli runner start --cwd /path/to/workspace --srt sandbox.json
|
||||
```
|
||||
|
||||
1. The network connection is blocked immediately.
|
||||
2. The target domain is logged and added to the **Pending Domains** list.
|
||||
3. Admins can review this list and click **Approve** to authorize the domain, or **Reject** to keep it blocked.
|
||||
Windows uses SRT's alpha Windows support. Run the one-time setup before starting a sandboxed runner; it requests administrator permission:
|
||||
|
||||

|
||||
```powershell
|
||||
npx @anthropic-ai/sandbox-runtime windows-install
|
||||
```
|
||||
|
||||
SRT verifies the setup at startup and reports an error if it is missing. A runner without `--srt` does not need this setup.
|
||||
|
||||
The JSON uses upstream SRT settings directly. For example:
|
||||
|
||||
```json
|
||||
{
|
||||
"filesystem": {
|
||||
"denyRead": ["~/.ssh", "~/.aws", ".env", ".env.*"],
|
||||
"allowWrite": [".pi"],
|
||||
"denyWrite": []
|
||||
},
|
||||
"network": {
|
||||
"allowedDomains": [],
|
||||
"deniedDomains": []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This example allows writes only in `.pi`, denies reads of the listed credential locations, and blocks command network access. Review read permissions and network rules for your environment using SRT's documentation. Relative filesystem paths use the runner workspace. Settings are snapshotted at startup; restart after changing them. Missing or invalid settings, missing SRT prerequisites, and sandbox initialization failures stop startup. Commands never fall back to unrestricted execution.
|
||||
|
||||
SRT wraps shell commands and their child processes. One shared SRT runtime handles concurrent commands and is reset when the runner stops. The runner's WebSocket and server-directed file transfers operate outside SRT. **The Shumai server address is not automatically added to allowed domains.** If a script itself needs network access, explicitly allow its destination in the JSON.
|
||||
|
||||
Shumai has no server-side sandbox or domain settings. SRT is initialized only on runners that opt into it. SRT in Docker may require namespace permissions on that dedicated runner container; these permissions are not needed by the Shumai server.
|
||||
|
||||
## Runner in Docker
|
||||
|
||||
An ordinary runner can run in its own container without extra capabilities or an exposed port:
|
||||
|
||||
```bash
|
||||
docker run --rm --init --name shumai-runner \
|
||||
--cap-drop ALL --security-opt no-new-privileges \
|
||||
-e SHUMAI_API_SERVER -e SHUMAI_RUNNER_KEY \
|
||||
-v shumai-runner-workspace:/workspace -w /workspace \
|
||||
node:22-bookworm sh -c \
|
||||
'npm install -g @shumai-one/shumai-cli && exec shumai-cli runner start'
|
||||
```
|
||||
|
||||
For development, HTTP is accepted for `localhost`, `127.0.0.1`, `[::1]`, and Docker's `host.docker.internal`. Remote servers require HTTPS. Avoid mounting sensitive host directories or the Docker socket into the runner workspace.
|
||||
|
||||
## Limits and lifecycle
|
||||
|
||||
Commands default to 120 seconds and accept a maximum timeout of one hour. Timeout and cancellation attempt to terminate the command and its process tree. Each command saves up to 64 MiB of output in `.pi/logs`; manage retained logs as part of workspace maintenance. Transfers and extracted archives are limited to 256 MiB. Protocol file operations reject paths outside `.pi`, existing symlinks, and unsafe archive entries. V1 assumes commands do not deliberately change workspace paths while transfers are running.
|
||||
|
||||
Temporary reconnects reconcile active and recently completed commands without deliberately executing them again. Restarting the runner process does not resume commands from the previous process.
|
||||
|
||||
Up to eight commands and file transfers can run concurrently in the shared workspace. Additional requests return a busy error. On Windows without SRT, background processes may survive after the command's shell exits; remaining output pipes are closed after one second. Use foreground commands for predictable timeout and cancellation. SRT manages the sandboxed Windows process tree.
|
||||
|
||||
@@ -53,6 +53,8 @@ Worker 根据可配置的并发上限从 PostgreSQL 队列中拉取任务:
|
||||
- **自定义 Worker ID**:若需显式指定标识符,可设置 `SHUMAI_WORKERS_ID`(或 `SHUMAI_WORKER_ID`)。
|
||||
- **Kubernetes 环境**:在 Kubernetes Pod 中,可通过 Downward API(`fieldRef: { fieldPath: metadata.name }`)设置 `POD_NAME`,以在 Pod 重启期间保持稳定且唯一的单 Pod 身份标识。
|
||||
|
||||
使用智能体运行器时,每个数据库只允许运行一个主服务器。已配置运行器时,请先停止旧主服务器,再启动替换实例。在 Kubernetes 中,主服务器应使用单副本和 Recreate 部署策略。转码 Worker 仍可独立扩展。
|
||||
|
||||
---
|
||||
|
||||
## 基于 Kubernetes 与 AWS EKS 的弹性扩展部署
|
||||
|
||||
@@ -1,45 +1,99 @@
|
||||
---
|
||||
title: 智能体沙箱
|
||||
description: AI 智能体脚本的安全执行环境与网络域名限制规范。
|
||||
title: 智能体运行器
|
||||
description: 连接可选的命令运行器,并在运行器机器上配置沙箱。
|
||||
---
|
||||
|
||||
运行自动化脚本存在潜在的安全风险。Shumai 通过在安全且隔离的**沙箱(Sandbox)**环境中执行所有智能体技能来化解这一隐患。
|
||||
智能体无需运行器即可使用内置工具。如需执行命令或将素材下载到命令工作区,请在自己管理的机器上连接**运行器**。运行器主动通过经过身份验证的 WebSocket 连接 Shumai,无需开放入站端口。
|
||||
|
||||
---
|
||||
每个数据库只支持一台主 Shumai 服务器使用智能体运行器。未配置运行器时,多台主服务器可以共享数据库。创建运行器前,请停止其他主服务器副本。转码 Worker 可以独立运行。
|
||||
|
||||
## 沙箱安全机制
|
||||
## 连接运行器
|
||||
|
||||
沙箱严格限制执行进程访问宿主机文件系统、系统进程及内部网络:
|
||||
1. 在 **设置 → 智能体运行器** 中创建运行器,复制仅显示一次的密钥。只有团队所有者可以管理运行器和轮换密钥。
|
||||
2. 在 macOS、Linux 或 Windows 机器上安装 [Shumai CLI](https://github.com/shumaiOne/shumai/tree/main/apps/cli#installation)。
|
||||
3. 在已有的专用工作目录中启动:
|
||||
|
||||
- **环境隔离**:脚本在虚拟化的工作区容器中运行,无法篡改系统核心文件或读取其他项目数据。
|
||||
- **网络桥接管控**:来自沙箱内部的外部网络请求会受到严格监控。
|
||||
```bash
|
||||
export SHUMAI_API_SERVER="https://shumai.example.com"
|
||||
export SHUMAI_RUNNER_KEY="your-runner-key"
|
||||
shumai-cli runner start --cwd /path/to/workspace
|
||||
```
|
||||
|
||||
### 文件系统访问限制
|
||||
Windows 用户可在 PowerShell 中设置变量:
|
||||
|
||||
为保障工作区环境与敏感数据安全,沙箱实施了严密的文件级访问控制策略:
|
||||
```powershell
|
||||
$env:SHUMAI_API_SERVER = "https://shumai.example.com"
|
||||
$env:SHUMAI_RUNNER_KEY = "your-runner-key"
|
||||
shumai-cli runner start --cwd C:\workspace
|
||||
```
|
||||
|
||||
- **读取权限**:沙箱允许读取工作区内的普通文件,但严格禁止访问敏感环境配置与证书密钥文件。禁止读取的文件模式包括:`['.env', '.env.*', '*.pem', '*.key']`。
|
||||
- **写入权限**:沙箱的写入权限受到极为严苛的限制,仅允许写入当前工作目录(cwd)下的 `.pi` 文件夹。即便在该受信任的目录内部,沙箱也严禁写入或创建匹配以下模式的敏感文件:`['.env', '.env.*', '*.pem', '*.key']`。
|
||||
使用运行器密钥,而非个人 API 令牌。建议通过环境变量传入,避免 `--key` 将密钥留在命令历史中。运行器在前台执行,Ctrl+C 会停止运行器及其活动命令。临时断线后自动重连;密钥撤销或连接被替换时退出。
|
||||
|
||||
---
|
||||
第一个运行器是团队默认运行器,也可在智能体设置中指定专用运行器。未配置运行器时隐藏 `bash` 和 `download_asset`。配置后,具有智能体使用权限的编辑者和审阅者可立即使用 Bash,无需先读取技能;禁用工具设置仍然生效。运行器离线时执行命令会返回错误。
|
||||
|
||||
## 域名白名单
|
||||
命令从工作目录启动,技能、下载素材、生成文件和日志位于 `.pi` 中。子进程只继承基础系统环境及明确配置的技能变量,不自动继承运行器密钥、个人 API 令牌或其他任意环境变量。
|
||||
|
||||
默认情况下,网络沙箱处于**关闭**状态(允许所有出站网络请求)。团队所有者可在沙箱设置中开启网络沙箱,以强制推行严格的域名白名单管控:
|
||||
所有有权在同一运行器执行命令的用户都可以读取工作区中保留的文件,包括其他项目的素材和日志。Shumai 下载素材时会检查项目权限。请仅在有权读取工作区内所有文件的用户之间共享运行器。不同信任范围应使用独立的运行器账号或虚拟机,并限制对应智能体的访问权限。
|
||||
|
||||
### 允许的域名 (Allowed Domains)
|
||||
## 可选 SRT 沙箱
|
||||
|
||||
允许沙箱内脚本正常访问的域名清单。
|
||||
未使用 `--srt` 时,命令具有运行器系统用户的权限。只连接可信 Shumai 服务器,可按需使用独立容器、虚拟机或账号。
|
||||
|
||||
- 默认白名单包含标准软件包注册表(如 npm、Python 官方源等)以及代码托管平台(如 `github.com`)。
|
||||
- 若脚本需要调用外部 API(如 `api.slack.com`),管理员须预先将其添加至白名单中。
|
||||
[Anthropic Sandbox Runtime](https://github.com/anthropics/sandbox-runtime) 已作为库内置于 Shumai CLI。安装平台依赖(Linux:bubblewrap、socat 和 ripgrep;macOS:ripgrep)后传入 JSON 配置:
|
||||
|
||||
### 待审批域名 (Pending Domains)
|
||||
```bash
|
||||
shumai-cli runner start --cwd /path/to/workspace --srt sandbox.json
|
||||
```
|
||||
|
||||
当智能体脚本尝试访问未经授权的域名时:
|
||||
Windows 使用 SRT 的实验性 Windows 支持。启用沙箱前执行一次初始化;此命令会请求管理员权限:
|
||||
|
||||
1. 网络连接将被即刻拦截。
|
||||
2. 系统会记录目标域名并将其加入**待审批域名**列表。
|
||||
3. 管理员可审阅该列表,点击**批准**将其加白,或点击**拒绝**保持拦截。
|
||||
```powershell
|
||||
npx @anthropic-ai/sandbox-runtime windows-install
|
||||
```
|
||||
|
||||

|
||||
SRT 会在启动时检查安装状态,缺少初始化时返回错误。未使用 `--srt` 的运行器不需要此步骤。`bash` 工具在 macOS 和 Linux 上使用 `/bin/sh`。在 Windows 上,运行器先在标准安装目录查找 Git Bash,再在 PATH 中查找 `bash.exe`,找不到时使用 `cmd.exe`。运行器在启动时显示所选 Shell,并将其报告给智能体,让智能体使用匹配的命令语法。在 Windows 上搭配 SRT 使用 Bash 时,请用 Git Bash 代替旧版 WSL Bash 启动器。
|
||||
|
||||
直接使用上游 SRT JSON 格式,例如:
|
||||
|
||||
```json
|
||||
{
|
||||
"filesystem": {
|
||||
"denyRead": ["~/.ssh", "~/.aws", ".env", ".env.*"],
|
||||
"allowWrite": [".pi"],
|
||||
"denyWrite": []
|
||||
},
|
||||
"network": {
|
||||
"allowedDomains": [],
|
||||
"deniedDomains": []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
此例只允许写入 `.pi`,禁止读取所列凭证位置,并禁止命令访问网络。请参考 SRT 文档审查适合自己环境的读取权限和域名规则。相对路径以运行器工作目录为基准。配置在启动时读取并保存快照,修改后需要重启。配置缺失或无效、SRT 依赖缺失、沙箱初始化失败都会阻止启动,不会回退到无沙箱执行。
|
||||
|
||||
SRT 只包裹 Shell 命令及其子进程。并发命令共享同一个 SRT 运行时,运行器停止时统一清理。运行器 WebSocket 和服务器发起的文件传输在 SRT 外运行。**不会自动将 Shumai 服务器地址添加到允许域名中。** 如果脚本自身需要网络,请在 JSON 中明确允许目标域名。
|
||||
|
||||
Shumai 服务器不管理沙箱和域名规则。只有启用沙箱的运行器才初始化 SRT。在 Docker 中使用 SRT 可能需要为专用运行器容器增加命名空间权限,Shumai 服务器不需要这些权限。
|
||||
|
||||
## Docker 运行器
|
||||
|
||||
普通运行器可在独立容器中执行,无需额外权限或暴露端口:
|
||||
|
||||
```bash
|
||||
docker run --rm --init --name shumai-runner \
|
||||
--cap-drop ALL --security-opt no-new-privileges \
|
||||
-e SHUMAI_API_SERVER -e SHUMAI_RUNNER_KEY \
|
||||
-v shumai-runner-workspace:/workspace -w /workspace \
|
||||
node:22-bookworm sh -c \
|
||||
'npm install -g @shumai-one/shumai-cli && exec shumai-cli runner start'
|
||||
```
|
||||
|
||||
本地开发允许 `localhost`、`127.0.0.1`、`[::1]`、Docker 的 `host.docker.internal` HTTP 地址;远程连接必须使用 HTTPS。不要将敏感宿主机目录或 Docker socket 挂载到运行器工作区。
|
||||
|
||||
## 限制与生命周期
|
||||
|
||||
默认超时 120 秒,最长一小时。超时或取消会尝试终止命令及其子进程。单条命令最多在 `.pi/logs` 保存 64 MiB 输出,请按工作区维护需求清理日志。文件传输和技能解压限制为 256 MiB。协议文件操作拒绝 `.pi` 外路径、已有符号链接及不安全的压缩包条目。V1 假定命令不会在文件传输期间故意修改工作区路径。
|
||||
|
||||
临时重连会同步执行中及近期完成的命令,避免主动重复执行。重启运行器进程不会恢复旧进程中的命令。
|
||||
|
||||
共享工作区最多同时执行八个命令或文件传输,超出时返回繁忙错误。Windows 未启用 SRT 时,后台进程可能在命令 Shell 退出后继续运行,剩余输出管道会在一秒后关闭。请使用前台命令,以确保超时和取消可预测。启用 SRT 时由 SRT 管理 Windows 沙箱进程树。
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "shumai",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"repository": {
|
||||
|
||||
@@ -1,12 +1,11 @@
|
||||
{
|
||||
"name": "@shumai/agent",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "./src/index.ts",
|
||||
"types": "./src/index.ts",
|
||||
"dependencies": {
|
||||
"@anthropic-ai/sandbox-runtime": "0.0.74",
|
||||
"@earendil-works/chord": "1.5.0",
|
||||
"@earendil-works/pi-ai": "1.5.0",
|
||||
"@earendil-works/pi-durable": "1.5.0",
|
||||
|
||||
@@ -66,7 +66,7 @@ export interface AgentExecutionContext {
|
||||
teamSkills: Skill[]
|
||||
mcpServers: Prisma.McpServerGetPayload<Record<string, never>>[]
|
||||
enabledSkillIds: string[]
|
||||
allowedDomains: string[]
|
||||
allowedDomains?: string[]
|
||||
}
|
||||
|
||||
export function formatProjectContextPrompt(
|
||||
@@ -97,7 +97,7 @@ async function executeAgentPrompt(params: {
|
||||
context: AgentExecutionContext
|
||||
attachedAssets?: Array<{ id: string; name: string; type: string }>
|
||||
}): Promise<{ text: string; usage: Usage; sessionId: string }> {
|
||||
const { agent, dbProviders, teamSkills, enabledSkillIds, allowedDomains } = params.context
|
||||
const { agent, dbProviders, teamSkills, enabledSkillIds } = params.context
|
||||
|
||||
if (!agent.provider) {
|
||||
throw ApplicationFailure.create({
|
||||
@@ -242,7 +242,6 @@ User messages may contain a <context> block detailing the user, active asset loc
|
||||
description: s.description,
|
||||
})),
|
||||
enabledSkillIds,
|
||||
allowedDomains,
|
||||
sessionId: persistedSessionId,
|
||||
userId: params.userId,
|
||||
projectId: params.projectId,
|
||||
@@ -793,18 +792,12 @@ export const getAgentChatContextActivity = defineStep(
|
||||
})
|
||||
const mcpServers = assignments.map((a) => a.mcpServer)
|
||||
|
||||
const sandbox = await prisma.sandbox.findUnique({
|
||||
where: { teamId: params.teamId },
|
||||
})
|
||||
const allowedDomains = sandbox?.networkSandboxEnabled ? sandbox.allowedDomains : ['*']
|
||||
|
||||
return {
|
||||
agent,
|
||||
dbProviders,
|
||||
teamSkills: permittedSkills,
|
||||
mcpServers,
|
||||
enabledSkillIds,
|
||||
allowedDomains,
|
||||
}
|
||||
},
|
||||
)
|
||||
@@ -882,18 +875,12 @@ export const getAgentAutofillContextActivity = defineStep(
|
||||
// Autofill agents currently do not use MCP tools.
|
||||
const mcpServers: Prisma.McpServerGetPayload<Record<string, never>>[] = []
|
||||
|
||||
const sandbox = await prisma.sandbox.findUnique({
|
||||
where: { teamId: params.teamId },
|
||||
})
|
||||
const allowedDomains = sandbox?.networkSandboxEnabled ? sandbox.allowedDomains : ['*']
|
||||
|
||||
return {
|
||||
agent: agentWithDetails,
|
||||
dbProviders,
|
||||
teamSkills: enabledSkills,
|
||||
mcpServers,
|
||||
enabledSkillIds,
|
||||
allowedDomains,
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
@@ -0,0 +1,278 @@
|
||||
import { invokeTool } from '@shumai/core/src/agent/tool-test-utils'
|
||||
import { hashRunnerKey, runnerService, type RunnerSocket } from '@shumai/core/src/runner'
|
||||
import { prisma } from '@shumai/db'
|
||||
import { setupTestDbHooks } from '@shumai/db/test'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { createAgentSession, type DbProviderInfo } from './index'
|
||||
import { closeAgentSessions } from './runtime'
|
||||
import { createRunnerBashTool } from './tools/runner-bash'
|
||||
|
||||
const mockProviders: DbProviderInfo[] = [
|
||||
{
|
||||
name: 'google',
|
||||
config: { api: 'google-generative-ai', apiKey: 'GOOGLE_API_KEY' },
|
||||
models: [
|
||||
{
|
||||
modelId: 'gemini',
|
||||
name: 'Gemini',
|
||||
config: {
|
||||
api: 'google-generative-ai',
|
||||
reasoning: false,
|
||||
input: ['text'],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: 8192,
|
||||
maxTokens: 4096,
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
function createMockWs() {
|
||||
const sent: string[] = []
|
||||
const ws: RunnerSocket = {
|
||||
send: vi.fn((data: string) => {
|
||||
sent.push(data)
|
||||
}),
|
||||
close: vi.fn(),
|
||||
}
|
||||
return {
|
||||
ws,
|
||||
sent,
|
||||
lastSentJson: () => (sent.length > 0 ? JSON.parse(sent[sent.length - 1]) : null),
|
||||
}
|
||||
}
|
||||
|
||||
function reconcile(runnerId: string, ws: RunnerSocket) {
|
||||
runnerService.handleMessage(
|
||||
runnerId,
|
||||
ws,
|
||||
JSON.stringify({
|
||||
type: 'reconcile',
|
||||
environment: { platform: 'linux', shell: '/bin/sh' },
|
||||
activeExecutions: [],
|
||||
completedExecutions: [],
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
describe('Concurrent Runner Sessions', () => {
|
||||
setupTestDbHooks()
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
await closeAgentSessions()
|
||||
})
|
||||
|
||||
it('executes concurrent bash commands on the same runner with isolated executionIds', async () => {
|
||||
const team = await prisma.team.create({
|
||||
data: { name: 'Concurrent Runner Team' },
|
||||
})
|
||||
const { runner, key } = await runnerService.createRunner(team.id, { name: 'Shared Runner' })
|
||||
const conn = createMockWs()
|
||||
await runnerService.registerConnection(
|
||||
{ id: runner.id, teamId: team.id, name: runner.name, keyHash: hashRunnerKey(key) },
|
||||
conn.ws,
|
||||
)
|
||||
reconcile(runner.id, conn.ws)
|
||||
|
||||
await prisma.user.create({
|
||||
data: {
|
||||
id: 'agent-1',
|
||||
name: 'Agent User',
|
||||
email: 'agent-user@shumai.ai',
|
||||
type: 'agent',
|
||||
},
|
||||
})
|
||||
await prisma.agent.create({
|
||||
data: {
|
||||
id: 'agent-1',
|
||||
teamId: team.id,
|
||||
type: 'chat',
|
||||
config: { provider: 'google', model: 'gemini' },
|
||||
},
|
||||
})
|
||||
|
||||
await createAgentSession({
|
||||
teamId: team.id,
|
||||
agentId: 'agent-1',
|
||||
providerName: 'google',
|
||||
modelId: 'gemini',
|
||||
systemPrompt: 'Session A prompt',
|
||||
teamSkills: [],
|
||||
providers: mockProviders,
|
||||
})
|
||||
|
||||
const tool = createRunnerBashTool({}, { teamId: team.id })
|
||||
|
||||
// Note: registerConnection sends a 'welcome' message first (conn.sent[0])
|
||||
const execA = invokeTool(tool, 'call-a', {
|
||||
command: 'echo session-a',
|
||||
})
|
||||
await vi.waitFor(() => expect(conn.sent.length).toBe(2))
|
||||
const msgA = JSON.parse(conn.sent[1])
|
||||
|
||||
const execB = invokeTool(tool, 'call-b', {
|
||||
command: 'echo session-b',
|
||||
})
|
||||
await vi.waitFor(() => expect(conn.sent.length).toBe(3))
|
||||
const msgB = JSON.parse(conn.sent[2])
|
||||
|
||||
expect(msgA.executionId).not.toBe(msgB.executionId)
|
||||
|
||||
// Interleave output and completion for A and B
|
||||
runnerService.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.output',
|
||||
executionId: msgB.executionId,
|
||||
stream: 'stdout',
|
||||
offset: 0,
|
||||
data: 'out-b',
|
||||
}),
|
||||
)
|
||||
runnerService.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.output',
|
||||
executionId: msgA.executionId,
|
||||
stream: 'stdout',
|
||||
offset: 0,
|
||||
data: 'out-a',
|
||||
}),
|
||||
)
|
||||
runnerService.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.result',
|
||||
executionId: msgA.executionId,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
}),
|
||||
)
|
||||
runnerService.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.result',
|
||||
executionId: msgB.executionId,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
}),
|
||||
)
|
||||
|
||||
const [resA, resB] = await Promise.all([execA, execB])
|
||||
expect(resA.content).toContainEqual(
|
||||
expect.objectContaining({ text: expect.stringContaining('out-a') }),
|
||||
)
|
||||
expect(resB.content).toContainEqual(
|
||||
expect.objectContaining({ text: expect.stringContaining('out-b') }),
|
||||
)
|
||||
|
||||
runnerService.handleDisconnect(runner.id, conn.ws)
|
||||
})
|
||||
|
||||
it('routes agent commands to their bound runner or team default runner independently', async () => {
|
||||
const team = await prisma.team.create({
|
||||
data: { name: 'Multi Runner Team' },
|
||||
})
|
||||
const { runner: defaultRunner, key: defaultKey } = await runnerService.createRunner(team.id, {
|
||||
name: 'Default Runner',
|
||||
isDefault: true,
|
||||
})
|
||||
const { runner: gpuRunner, key: gpuKey } = await runnerService.createRunner(team.id, {
|
||||
name: 'GPU Runner',
|
||||
isDefault: false,
|
||||
})
|
||||
|
||||
const defaultConn = createMockWs()
|
||||
const gpuConn = createMockWs()
|
||||
await runnerService.registerConnection(
|
||||
{
|
||||
id: defaultRunner.id,
|
||||
teamId: team.id,
|
||||
name: defaultRunner.name,
|
||||
keyHash: hashRunnerKey(defaultKey),
|
||||
},
|
||||
defaultConn.ws,
|
||||
)
|
||||
await runnerService.registerConnection(
|
||||
{
|
||||
id: gpuRunner.id,
|
||||
teamId: team.id,
|
||||
name: gpuRunner.name,
|
||||
keyHash: hashRunnerKey(gpuKey),
|
||||
},
|
||||
gpuConn.ws,
|
||||
)
|
||||
reconcile(defaultRunner.id, defaultConn.ws)
|
||||
reconcile(gpuRunner.id, gpuConn.ws)
|
||||
|
||||
const defaultTool = createRunnerBashTool(
|
||||
{},
|
||||
{
|
||||
teamId: team.id,
|
||||
agentRunnerId: null,
|
||||
},
|
||||
)
|
||||
const gpuTool = createRunnerBashTool(
|
||||
{},
|
||||
{
|
||||
teamId: team.id,
|
||||
agentRunnerId: gpuRunner.id,
|
||||
},
|
||||
)
|
||||
|
||||
const pDefault = invokeTool(defaultTool, '1', { command: 'uname -a' })
|
||||
const pGpu = invokeTool(gpuTool, '2', { command: 'nvidia-smi' })
|
||||
|
||||
await vi.waitFor(() => {
|
||||
expect(defaultConn.lastSentJson()?.type).toBe('exec.start')
|
||||
expect(gpuConn.lastSentJson()?.type).toBe('exec.start')
|
||||
})
|
||||
|
||||
const defaultMsg = defaultConn.lastSentJson()
|
||||
const gpuMsg = gpuConn.lastSentJson()
|
||||
expect(defaultMsg.command).toBe('uname -a')
|
||||
expect(gpuMsg.command).toBe('nvidia-smi')
|
||||
|
||||
runnerService.handleMessage(
|
||||
defaultRunner.id,
|
||||
defaultConn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.result',
|
||||
executionId: defaultMsg.executionId,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
}),
|
||||
)
|
||||
runnerService.handleMessage(
|
||||
gpuRunner.id,
|
||||
gpuConn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.result',
|
||||
executionId: gpuMsg.executionId,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
}),
|
||||
)
|
||||
|
||||
await Promise.all([pDefault, pGpu])
|
||||
runnerService.handleDisconnect(defaultRunner.id, defaultConn.ws)
|
||||
runnerService.handleDisconnect(gpuRunner.id, gpuConn.ws)
|
||||
})
|
||||
})
|
||||
@@ -1,328 +0,0 @@
|
||||
import type { SandboxConfig, SandboxProvider, WrapCommandOptions } from '@shumai/core'
|
||||
import { sandboxService } from '@shumai/core'
|
||||
import { invokeTool } from '@shumai/core/src/agent/tool-test-utils'
|
||||
import { prisma } from '@shumai/db'
|
||||
import { setupTestDbHooks } from '@shumai/db/test'
|
||||
import { spawn } from 'node:child_process'
|
||||
import { EventEmitter } from 'node:events'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { createAgentSession, type DbProviderInfo } from './index'
|
||||
import { closeAgentSessions } from './runtime'
|
||||
import { createSandboxedBashTool } from './tools/sandboxed-bash'
|
||||
|
||||
vi.mock('node:child_process')
|
||||
|
||||
class TestSandboxProvider implements SandboxProvider {
|
||||
initialized = false
|
||||
config: SandboxConfig | null = null
|
||||
initializeCalls = 0
|
||||
updateConfigCalls = 0
|
||||
resetCalls = 0
|
||||
wrapCommandCalls: Array<{ command: string; options?: WrapCommandOptions }> = []
|
||||
violationsByCommandId = new Map<string, string>()
|
||||
|
||||
async initialize(config: SandboxConfig): Promise<void> {
|
||||
this.initialized = true
|
||||
this.config = config
|
||||
this.initializeCalls++
|
||||
}
|
||||
|
||||
async wrapCommand(command: string, options?: WrapCommandOptions): Promise<string> {
|
||||
this.wrapCommandCalls.push({ command, options })
|
||||
return `sandboxed-${command}`
|
||||
}
|
||||
|
||||
async updateConfig(config: Partial<SandboxConfig>): Promise<void> {
|
||||
this.updateConfigCalls++
|
||||
if (this.config && config.network) {
|
||||
this.config.network = { ...this.config.network, ...config.network }
|
||||
}
|
||||
}
|
||||
|
||||
async reset(): Promise<void> {
|
||||
this.resetCalls++
|
||||
this.initialized = false
|
||||
this.config = null
|
||||
}
|
||||
|
||||
isInitialized(): boolean {
|
||||
return this.initialized
|
||||
}
|
||||
|
||||
getBlockedHostForCommand(commandId: string): string | null {
|
||||
return this.violationsByCommandId.get(commandId) ?? null
|
||||
}
|
||||
}
|
||||
|
||||
const mockProviders: DbProviderInfo[] = [
|
||||
{
|
||||
name: 'google',
|
||||
config: { api: 'google-generative-ai', apiKey: 'GOOGLE_API_KEY' },
|
||||
models: [
|
||||
{
|
||||
modelId: 'gemini',
|
||||
name: 'Gemini',
|
||||
config: {
|
||||
api: 'google-generative-ai',
|
||||
reasoning: false,
|
||||
input: ['text'],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: 8192,
|
||||
maxTokens: 4096,
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
describe('Concurrent Sandbox Sessions', () => {
|
||||
setupTestDbHooks()
|
||||
|
||||
let testProvider: TestSandboxProvider
|
||||
|
||||
beforeEach(async () => {
|
||||
vi.clearAllMocks()
|
||||
testProvider = new TestSandboxProvider()
|
||||
sandboxService.setProvider(testProvider)
|
||||
})
|
||||
|
||||
it('does not reset sandbox when new sessions start while a command is running', async () => {
|
||||
const team = await prisma.team.create({
|
||||
data: { name: 'Concurrent Test Team' },
|
||||
})
|
||||
await prisma.sandbox.create({
|
||||
data: {
|
||||
teamId: team.id,
|
||||
networkSandboxEnabled: true,
|
||||
allowedDomains: ['api.github.com'],
|
||||
},
|
||||
})
|
||||
await prisma.user.create({
|
||||
data: {
|
||||
id: 'agent-1',
|
||||
name: 'Agent User',
|
||||
email: 'agent-user@shumai.ai',
|
||||
type: 'agent',
|
||||
},
|
||||
})
|
||||
await prisma.agent.create({
|
||||
data: {
|
||||
id: 'agent-1',
|
||||
teamId: team.id,
|
||||
type: 'chat',
|
||||
config: { provider: 'google', model: 'gemini' },
|
||||
},
|
||||
})
|
||||
|
||||
// 1. Session A starts and syncs domains
|
||||
await createAgentSession({
|
||||
teamId: team.id,
|
||||
agentId: 'agent-1',
|
||||
providerName: 'google',
|
||||
modelId: 'gemini',
|
||||
systemPrompt: 'Session A prompt',
|
||||
teamSkills: [],
|
||||
allowedDomains: ['api.github.com'],
|
||||
providers: mockProviders,
|
||||
})
|
||||
|
||||
expect(testProvider.initializeCalls).toBe(1)
|
||||
expect(testProvider.resetCalls).toBe(0)
|
||||
|
||||
// 2. Session A runs a bash command that takes time
|
||||
const tool = createSandboxedBashTool('/mock/cwd')
|
||||
|
||||
const mockStdout = new EventEmitter()
|
||||
const mockStderr = new EventEmitter()
|
||||
const mockChild = Object.assign(new EventEmitter(), {
|
||||
stdout: mockStdout,
|
||||
stderr: mockStderr,
|
||||
pid: 999,
|
||||
kill: vi.fn(),
|
||||
})
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mock spawn return value
|
||||
;(spawn as any).mockReturnValue(mockChild)
|
||||
|
||||
const sessionExecutionA = invokeTool(tool, 'call-session-a-1', {
|
||||
command: 'curl https://api.github.com/repos',
|
||||
source: 'skill',
|
||||
})
|
||||
|
||||
// 3. Concurrently, Session B starts with additional allowed domains
|
||||
await createAgentSession({
|
||||
teamId: team.id,
|
||||
agentId: 'agent-1',
|
||||
providerName: 'google',
|
||||
modelId: 'gemini',
|
||||
systemPrompt: 'Session B prompt',
|
||||
teamSkills: [],
|
||||
allowedDomains: ['api.github.com', 'pypi.org'],
|
||||
providers: mockProviders,
|
||||
})
|
||||
|
||||
// Assert sandbox was hot-reloaded without any reset!
|
||||
expect(testProvider.resetCalls).toBe(0)
|
||||
expect(testProvider.updateConfigCalls).toBe(1)
|
||||
expect(sandboxService.getCurrentAllowedDomains()).toEqual(['api.github.com', 'pypi.org'])
|
||||
|
||||
// 4. Concurrently, Session C starts with the same domains (should be no-op, no reset, no updateConfig)
|
||||
await createAgentSession({
|
||||
teamId: team.id,
|
||||
agentId: 'agent-1',
|
||||
providerName: 'google',
|
||||
modelId: 'gemini',
|
||||
systemPrompt: 'Session C prompt',
|
||||
teamSkills: [],
|
||||
allowedDomains: ['pypi.org', 'api.github.com'],
|
||||
providers: mockProviders,
|
||||
})
|
||||
|
||||
expect(testProvider.resetCalls).toBe(0)
|
||||
expect(testProvider.updateConfigCalls).toBe(1) // Still 1, no duplicate update
|
||||
|
||||
// 5. Session A's command completes successfully
|
||||
mockStdout.emit('data', Buffer.from('{"status":"ok"}'))
|
||||
mockChild.emit('close', 0)
|
||||
|
||||
const result = await sessionExecutionA
|
||||
expect(result.details).toEqual(
|
||||
expect.objectContaining({
|
||||
exitCode: 0,
|
||||
}),
|
||||
)
|
||||
|
||||
// Verify command was wrapped with options containing commandId: 'call-session-a-1'
|
||||
expect(testProvider.wrapCommandCalls).toContainEqual({
|
||||
command: 'curl https://api.github.com/repos',
|
||||
options: { commandId: 'call-session-a-1' },
|
||||
})
|
||||
})
|
||||
|
||||
it('isolates blocked host violations per commandId during concurrent execution', async () => {
|
||||
await sandboxService.ensureInitialized({ allowedDomains: ['api.github.com'] })
|
||||
|
||||
// Command 1 and Command 2 run concurrently
|
||||
await sandboxService.wrapCommand('curl http://blocked1.com', { commandId: 'cmd-1' })
|
||||
await sandboxService.wrapCommand('curl http://blocked2.com', { commandId: 'cmd-2' })
|
||||
|
||||
testProvider.violationsByCommandId.set('cmd-1', 'blocked1.com')
|
||||
testProvider.violationsByCommandId.set('cmd-2', 'blocked2.com')
|
||||
|
||||
// Blocked hosts should be accurately isolated
|
||||
expect(sandboxService.getBlockedHost('cmd-1')).toBe('blocked1.com')
|
||||
expect(sandboxService.getBlockedHost('cmd-2')).toBe('blocked2.com')
|
||||
expect(sandboxService.getBlockedHost('cmd-unknown')).toBe('')
|
||||
})
|
||||
|
||||
it('does not reject concurrent allowed command when another command is blocked', async () => {
|
||||
await sandboxService.ensureInitialized({ allowedDomains: ['api.github.com'] })
|
||||
|
||||
const tool = createSandboxedBashTool('/mock/cwd', undefined, {
|
||||
getBlockedHost: (commandId?: string) => sandboxService.getBlockedHost(commandId),
|
||||
clearBlockedHost: () => sandboxService.clearBlockedHost(),
|
||||
})
|
||||
|
||||
const childBlocked = Object.assign(new EventEmitter(), {
|
||||
stdout: new EventEmitter(),
|
||||
stderr: new EventEmitter(),
|
||||
pid: 101,
|
||||
kill: vi.fn(),
|
||||
})
|
||||
const childAllowed = Object.assign(new EventEmitter(), {
|
||||
stdout: new EventEmitter(),
|
||||
stderr: new EventEmitter(),
|
||||
pid: 102,
|
||||
kill: vi.fn(),
|
||||
})
|
||||
|
||||
let spawnIndex = 0
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mock spawn implementation
|
||||
;(spawn as any).mockImplementation(() => {
|
||||
spawnIndex++
|
||||
return spawnIndex === 1 ? childBlocked : childAllowed
|
||||
})
|
||||
|
||||
// Record violation for the blocked command only
|
||||
testProvider.violationsByCommandId.set('call-blocked', 'blocked-target.com')
|
||||
|
||||
const blockedExecution = invokeTool(tool, 'call-blocked', {
|
||||
command: 'curl http://blocked-target.com',
|
||||
source: 'skill',
|
||||
})
|
||||
const allowedExecution = invokeTool(tool, 'call-allowed', {
|
||||
command: 'curl https://api.github.com/data',
|
||||
source: 'skill',
|
||||
})
|
||||
|
||||
await vi.waitFor(() => expect(spawn).toHaveBeenCalledTimes(2))
|
||||
|
||||
childBlocked.stdout.emit('data', Buffer.from('blocked output'))
|
||||
childBlocked.emit('close', 0)
|
||||
|
||||
childAllowed.stdout.emit('data', Buffer.from('allowed output'))
|
||||
childAllowed.emit('close', 0)
|
||||
|
||||
await expect(blockedExecution).rejects.toThrow(
|
||||
'Network request to blocked-target.com is blocked',
|
||||
)
|
||||
const allowedResult = await allowedExecution
|
||||
expect(allowedResult.details).toEqual(expect.objectContaining({ exitCode: 0 }))
|
||||
})
|
||||
|
||||
it('does not synchronize or overwrite sandbox allowlist when allowedDomains is omitted (e.g. naming sessions)', async () => {
|
||||
const team = await prisma.team.create({
|
||||
data: { name: 'Omitted Domains Team' },
|
||||
})
|
||||
await prisma.user.create({
|
||||
data: {
|
||||
id: 'agent-omitted',
|
||||
name: 'Agent User',
|
||||
email: 'agent-omitted@shumai.ai',
|
||||
type: 'agent',
|
||||
},
|
||||
})
|
||||
await prisma.agent.create({
|
||||
data: {
|
||||
id: 'agent-omitted',
|
||||
teamId: team.id,
|
||||
type: 'chat',
|
||||
config: { provider: 'google', model: 'gemini' },
|
||||
},
|
||||
})
|
||||
|
||||
// 1. First session sets allowedDomains to ['*.github.com']
|
||||
await createAgentSession({
|
||||
teamId: team.id,
|
||||
agentId: 'agent-omitted',
|
||||
providerName: 'google',
|
||||
modelId: 'gemini',
|
||||
systemPrompt: 'Main prompt',
|
||||
teamSkills: [],
|
||||
allowedDomains: ['*.github.com'],
|
||||
providers: mockProviders,
|
||||
})
|
||||
|
||||
expect(sandboxService.getCurrentAllowedDomains()).toEqual(['*.github.com'])
|
||||
const updateCallsBefore = testProvider.updateConfigCalls
|
||||
|
||||
// 2. Utility / naming session runs WITHOUT allowedDomains
|
||||
await createAgentSession({
|
||||
teamId: team.id,
|
||||
agentId: 'agent-omitted',
|
||||
providerName: 'google',
|
||||
modelId: 'gemini',
|
||||
systemPrompt: 'Title generator',
|
||||
teamSkills: [],
|
||||
// allowedDomains omitted
|
||||
disableTools: true,
|
||||
providers: mockProviders,
|
||||
})
|
||||
|
||||
// Assert sandbox was NOT touched or overwritten
|
||||
expect(testProvider.updateConfigCalls).toBe(updateCallsBefore)
|
||||
expect(sandboxService.getCurrentAllowedDomains()).toEqual(['*.github.com'])
|
||||
})
|
||||
})
|
||||
|
||||
afterEach(closeAgentSessions)
|
||||
+245
-883
File diff suppressed because it is too large
Load Diff
@@ -1,11 +1,11 @@
|
||||
import { invokeTool } from '@shumai/core/src/agent/tool-test-utils'
|
||||
import { authzService } from '@shumai/core/src/authz/authz'
|
||||
import { getOperation } from '@shumai/core/src/operations'
|
||||
import { runnerService } from '@shumai/core/src/runner'
|
||||
import { s3Service } from '@shumai/core/src/s3/s3'
|
||||
import { prisma, type Asset, type StorageKey } from '@shumai/db'
|
||||
import * as fs from 'fs'
|
||||
import * as path from 'path'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { toAgentTool } from '../operation-tools'
|
||||
import './download-asset'
|
||||
|
||||
@@ -32,18 +32,28 @@ vi.mock('@shumai/core/src/s3/s3', async (importOriginal) => {
|
||||
}
|
||||
})
|
||||
|
||||
vi.mock('@shumai/core/src/runner', () => ({
|
||||
runnerService: {
|
||||
requireOnlineRunner: vi.fn(),
|
||||
isRunnerConnected: vi.fn(),
|
||||
downloadToRunner: vi.fn(),
|
||||
},
|
||||
}))
|
||||
|
||||
vi.mock('@shumai/core/src/authz/authz', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('@shumai/core/src/authz/authz')>()
|
||||
return { ...actual, authzService: { hasPermission: vi.fn() } }
|
||||
})
|
||||
|
||||
type Head = Awaited<ReturnType<typeof s3Service.headObject>>
|
||||
type Saved = { filePath: string; absolutePath: string; contentType: string; size: number }
|
||||
|
||||
const piDir = path.join(process.cwd(), '.pi')
|
||||
type Saved = { filePath: string; contentType: string; size: number }
|
||||
|
||||
function downloadTool(userId = 'user-1') {
|
||||
return toAgentTool(getOperation('download_asset')!, { userId })
|
||||
return toAgentTool(getOperation('download_asset')!, {
|
||||
userId,
|
||||
teamId: 'team-1',
|
||||
runnerId: null,
|
||||
})
|
||||
}
|
||||
|
||||
function textOf(result: { content: unknown[] }): string {
|
||||
@@ -57,10 +67,14 @@ function mockAssets(...assets: Array<Record<string, unknown>>) {
|
||||
)
|
||||
}
|
||||
|
||||
function mockStorage(contentType: string, write = 'bytes') {
|
||||
vi.mocked(s3Service.headObject).mockResolvedValue({ size: 1234, contentType } as Head)
|
||||
vi.mocked(s3Service.downloadToFile).mockImplementation(async (_bucket, _key, dest) => {
|
||||
await fs.promises.writeFile(dest, write)
|
||||
function mockStorage(contentType: string, size = 11) {
|
||||
vi.mocked(s3Service.headObject).mockResolvedValue({ size, contentType } as Head)
|
||||
vi.mocked(runnerService.downloadToRunner).mockResolvedValue({
|
||||
type: 'file.download.result',
|
||||
executionId: 'exec-dl-1',
|
||||
ok: true,
|
||||
size,
|
||||
mimeType: contentType,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -74,15 +88,19 @@ const photo = {
|
||||
}
|
||||
|
||||
describe('download_asset agent tool', () => {
|
||||
const created: string[] = []
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
vi.mocked(authzService.hasPermission).mockResolvedValue()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
for (const file of created.splice(0)) fs.rmSync(file, { force: true })
|
||||
vi.mocked(runnerService.requireOnlineRunner).mockResolvedValue({
|
||||
id: 'runner-1',
|
||||
teamId: 'team-1',
|
||||
name: 'Local Mac Mini',
|
||||
keyPrefix: 'shm_rn_12345678',
|
||||
isDefault: true,
|
||||
status: 'connected',
|
||||
createdAt: '2026-11-14T00:00:00.000Z',
|
||||
updatedAt: '2026-11-14T00:00:00.000Z',
|
||||
})
|
||||
})
|
||||
|
||||
it('offers assetIds, proxy, and the agent-only key parameter', () => {
|
||||
@@ -93,35 +111,32 @@ describe('download_asset agent tool', () => {
|
||||
expect(tool.description).toContain('.pi directory')
|
||||
})
|
||||
|
||||
it('saves the web preview into .pi by default, named after the asset', async () => {
|
||||
it('saves the web preview into .pi on the runner by default, named after the asset', async () => {
|
||||
mockAssets(photo)
|
||||
mockStorage('image/webp', 'proxy-bytes')
|
||||
mockStorage('image/webp', 11)
|
||||
|
||||
const result = await invokeTool(downloadTool(), 'call-1', { assetIds: ['img-1'] })
|
||||
|
||||
const expectedPath = path.join(piDir, 'img-1_sample_photo.webp')
|
||||
created.push(expectedPath)
|
||||
expect(authzService.hasPermission).toHaveBeenCalledWith({
|
||||
user: { id: 'user-1' },
|
||||
permission: 'Read',
|
||||
type: 'asset',
|
||||
id: 'img-1',
|
||||
})
|
||||
expect(s3Service.downloadToFile).toHaveBeenCalledWith(
|
||||
'shumai',
|
||||
'files/img-1/proxy.webp',
|
||||
expectedPath,
|
||||
)
|
||||
expect(fs.readFileSync(expectedPath, 'utf-8')).toBe('proxy-bytes')
|
||||
expect(runnerService.requireOnlineRunner).toHaveBeenCalledWith('team-1', null)
|
||||
expect(runnerService.downloadToRunner).toHaveBeenCalledWith('runner-1', {
|
||||
url: 'https://signed.example/file',
|
||||
destPath: path.posix.join('.pi', 'img-1_sample_photo.webp'),
|
||||
})
|
||||
expect(textOf(result)).toBe(
|
||||
[
|
||||
`Downloaded "sample_photo.webp" (ID: img-1, proxy, image/webp, 11 B) to "${path.join('.pi', 'img-1_sample_photo.webp')}".`,
|
||||
`Downloaded "sample_photo.webp" (ID: img-1, proxy, image/webp, 11 B) to "${path.posix.join('.pi', 'img-1_sample_photo.webp')}".`,
|
||||
'Remember to delete these temporary files when finished.',
|
||||
].join('\n'),
|
||||
)
|
||||
const details = result.details as { files: Saved[] }
|
||||
expect(details.files[0]).toMatchObject({
|
||||
filePath: path.join('.pi', 'img-1_sample_photo.webp'),
|
||||
filePath: path.posix.join('.pi', 'img-1_sample_photo.webp'),
|
||||
contentType: 'image/webp',
|
||||
size: 11,
|
||||
})
|
||||
@@ -129,17 +144,23 @@ describe('download_asset agent tool', () => {
|
||||
|
||||
it('saves the original with proxy false', async () => {
|
||||
mockAssets(photo)
|
||||
mockStorage('image/jpeg')
|
||||
mockStorage('image/jpeg', 1234)
|
||||
|
||||
await invokeTool(downloadTool(), 'call-1', { assetIds: ['img-1'], proxy: false })
|
||||
|
||||
const expectedPath = path.join(piDir, 'img-1_sample_photo.jpg')
|
||||
created.push(expectedPath)
|
||||
expect(s3Service.downloadToFile).toHaveBeenCalledWith(
|
||||
expect(s3Service.presign).toHaveBeenCalledWith(
|
||||
'shumai',
|
||||
'files/up-1/sample_photo.jpg',
|
||||
expectedPath,
|
||||
'GET',
|
||||
false,
|
||||
undefined,
|
||||
false,
|
||||
600,
|
||||
)
|
||||
expect(runnerService.downloadToRunner).toHaveBeenCalledWith('runner-1', {
|
||||
url: 'https://signed.example/file',
|
||||
destPath: path.posix.join('.pi', 'img-1_sample_photo.jpg'),
|
||||
})
|
||||
})
|
||||
|
||||
it('downloads several files and the latest version of a version stack', async () => {
|
||||
@@ -159,16 +180,12 @@ describe('download_asset agent tool', () => {
|
||||
key: null,
|
||||
})
|
||||
|
||||
created.push(
|
||||
path.join(piDir, 'img-1_sample_photo.webp'),
|
||||
path.join(piDir, 'file-v2_banana_pig.webp'),
|
||||
)
|
||||
expect(prisma.asset.findFirst).toHaveBeenCalledWith({
|
||||
where: { parentId: 'stack-1', isDeleted: false },
|
||||
orderBy: { sortIndex: 'asc' },
|
||||
include: { storageKey: true },
|
||||
})
|
||||
expect(s3Service.downloadToFile).toHaveBeenCalledTimes(2)
|
||||
expect(runnerService.downloadToRunner).toHaveBeenCalledTimes(2)
|
||||
expect(textOf(result)).toContain('Downloaded "banana_pig.webp" (ID: file-v2, proxy')
|
||||
})
|
||||
|
||||
@@ -177,7 +194,7 @@ describe('download_asset agent tool', () => {
|
||||
await expect(invokeTool(downloadTool(), 'call-1', { assetIds: ['folder-1'] })).rejects.toThrow(
|
||||
'Asset folder-1 ("Renders") is a folder. Download the files inside it instead.',
|
||||
)
|
||||
expect(s3Service.downloadToFile).not.toHaveBeenCalled()
|
||||
expect(runnerService.downloadToRunner).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('checks Read on every asset before downloading anything', async () => {
|
||||
@@ -188,7 +205,7 @@ describe('download_asset agent tool', () => {
|
||||
await expect(
|
||||
invokeTool(downloadTool(), 'call-1', { assetIds: ['img-1', 'secret'] }),
|
||||
).rejects.toThrow('Forbidden')
|
||||
expect(s3Service.downloadToFile).not.toHaveBeenCalled()
|
||||
expect(runnerService.downloadToRunner).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('requires assetIds or key', async () => {
|
||||
@@ -240,27 +257,28 @@ describe('download_asset agent tool', () => {
|
||||
type: 'asset',
|
||||
id: 'other-project-asset',
|
||||
})
|
||||
expect(s3Service.downloadToFile).not.toHaveBeenCalled()
|
||||
expect(runnerService.downloadToRunner).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('downloads a read_asset artifact into .pi', async () => {
|
||||
it('downloads a read_asset artifact into .pi on the runner', async () => {
|
||||
mockAssets({ id: 'ast-123' })
|
||||
mockStorage('image/webp', 'frame-bytes')
|
||||
mockStorage('image/webp', 11)
|
||||
|
||||
const result = await invokeTool(downloadTool(), 'call-1', {
|
||||
key: 'files/ast-123/screenshots/shot_5.0s.webp',
|
||||
})
|
||||
|
||||
const expectedPath = path.join(piDir, 'shot_5.0s.webp')
|
||||
created.push(expectedPath)
|
||||
expect(fs.readFileSync(expectedPath, 'utf-8')).toBe('frame-bytes')
|
||||
expect(runnerService.downloadToRunner).toHaveBeenCalledWith('runner-1', {
|
||||
url: 'https://signed.example/file',
|
||||
destPath: path.posix.join('.pi', 'shot_5.0s.webp'),
|
||||
})
|
||||
expect(textOf(result)).toBe(
|
||||
`Downloaded storage key "files/ast-123/screenshots/shot_5.0s.webp" to "${path.join('.pi', 'shot_5.0s.webp')}". Remember to delete this temporary file when finished.`,
|
||||
`Downloaded storage key "files/ast-123/screenshots/shot_5.0s.webp" to "${path.posix.join('.pi', 'shot_5.0s.webp')}". Remember to delete this temporary file when finished.`,
|
||||
)
|
||||
expect(result.details).toMatchObject({
|
||||
key: 'files/ast-123/screenshots/shot_5.0s.webp',
|
||||
assetId: 'ast-123',
|
||||
size: Buffer.from('frame-bytes').length,
|
||||
size: 11,
|
||||
})
|
||||
})
|
||||
|
||||
@@ -271,24 +289,25 @@ describe('download_asset agent tool', () => {
|
||||
key: 'files/upload-ulid-456/photo.png',
|
||||
assets: [{ id: 'asset-photo-1' }],
|
||||
} as unknown as StorageKey & { assets: Asset[] })
|
||||
const pngHeader = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a])
|
||||
vi.mocked(s3Service.headObject).mockResolvedValue({
|
||||
size: 8,
|
||||
contentType: 'application/octet-stream',
|
||||
} as Head)
|
||||
vi.mocked(s3Service.downloadToFile).mockImplementation(async (_b, _k, dest) => {
|
||||
await fs.promises.writeFile(dest, pngHeader)
|
||||
vi.mocked(runnerService.downloadToRunner).mockResolvedValue({
|
||||
type: 'file.download.result',
|
||||
executionId: 'exec-dl-2',
|
||||
ok: true,
|
||||
size: 8,
|
||||
mimeType: 'image/png',
|
||||
})
|
||||
|
||||
const result = await invokeTool(downloadTool(), 'call-1', {
|
||||
key: 'files/upload-ulid-456/photo.png',
|
||||
})
|
||||
|
||||
created.push(path.join(piDir, 'photo.png'))
|
||||
expect(authzService.hasPermission).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ id: 'asset-photo-1' }),
|
||||
)
|
||||
// Falls back to detecting the type from the bytes
|
||||
expect(result.details).toMatchObject({ assetId: 'asset-photo-1', contentType: 'image/png' })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -2,52 +2,66 @@ import type { ToolExecutionResult as AgentToolResult } from '@earendil-works/pi-
|
||||
import { storageBucket } from '@shumai/core/src/asset-read'
|
||||
import { assertStorageKeyNotArchived } from '@shumai/core/src/asset/asset-archive'
|
||||
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
|
||||
import { runnerService } from '@shumai/core/src/runner'
|
||||
import { s3Service } from '@shumai/core/src/s3/s3'
|
||||
import { readFileMimeType } from '@shumai/core/src/utils/file-mime'
|
||||
import { getFileMimeType } from '@shumai/core/src/utils/file-mime'
|
||||
import { sanitizeFilename } from '@shumai/core/src/utils/filename'
|
||||
import type { User } from '@shumai/db'
|
||||
import { downloadAsset, MAX_DOWNLOAD_ASSETS, type DownloadAssetOutput } from '@shumai/operations'
|
||||
import * as fs from 'fs'
|
||||
import * as path from 'path'
|
||||
import { z } from 'zod'
|
||||
import { registerAgentToolExtension, type AgentToolExtension } from '../operation-tools'
|
||||
import {
|
||||
registerAgentToolExtension,
|
||||
type AgentOperationContext,
|
||||
type AgentToolExtension,
|
||||
} from '../operation-tools'
|
||||
import { resolveAssetIdFromKey } from '../tools/download-asset'
|
||||
|
||||
const CLEANUP_REMINDER = 'Remember to delete these temporary files when finished.'
|
||||
|
||||
/** The agent's sandbox, where downloaded files are saved for local scripts and bash. */
|
||||
function sandboxDir(): string {
|
||||
const dir = path.join(process.cwd(), '.pi')
|
||||
fs.mkdirSync(dir, { recursive: true })
|
||||
return dir
|
||||
}
|
||||
|
||||
function formatSize(bytes: number): string {
|
||||
if (bytes < 1024) return `${bytes} B`
|
||||
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`
|
||||
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
|
||||
}
|
||||
|
||||
/** Saves a storage object into the sandbox; returns the paths and detected content type. */
|
||||
async function saveToSandbox(key: string, filename: string) {
|
||||
const dir = sandboxDir()
|
||||
const absolutePath = path.join(dir, filename)
|
||||
async function resolveTargetRunnerId(ctx: AgentOperationContext): Promise<string> {
|
||||
if (ctx.runnerId && !ctx.teamId) {
|
||||
if (!runnerService.isRunnerConnected(ctx.runnerId)) {
|
||||
throw new Error(
|
||||
'Runner is currently offline. Please start your runner with "shumai-cli runner start" and try again.',
|
||||
)
|
||||
}
|
||||
return ctx.runnerId
|
||||
}
|
||||
const runner = await runnerService.requireOnlineRunner(ctx.teamId || '', ctx.runnerId)
|
||||
return runner.id
|
||||
}
|
||||
|
||||
/** Saves a storage object into the runner's .pi directory; returns the path, size, and content type. */
|
||||
async function saveToRunner(key: string, filename: string, ctx: AgentOperationContext) {
|
||||
const runnerId = await resolveTargetRunnerId(ctx)
|
||||
const bucket = storageBucket()
|
||||
const info = await s3Service.headObject(bucket, key).catch(() => null)
|
||||
await s3Service.downloadToFile(bucket, key, absolutePath)
|
||||
const size = await fs.promises
|
||||
.stat(absolutePath)
|
||||
.then((s) => s.size)
|
||||
.catch(() => info?.size ?? 0)
|
||||
const url = await s3Service.presign(bucket, key, 'GET', false, undefined, false, 600)
|
||||
const filePath = path.posix.join('.pi', filename)
|
||||
const res = await runnerService.downloadToRunner(runnerId, { url, destPath: filePath })
|
||||
if (!res.ok) {
|
||||
throw new Error(res.error || `Failed to download "${filename}" to runner`)
|
||||
}
|
||||
const size = res.size ?? info?.size ?? 0
|
||||
const contentType =
|
||||
info?.contentType && info.contentType !== 'application/octet-stream'
|
||||
? info.contentType
|
||||
: readFileMimeType(absolutePath)
|
||||
return { absolutePath, filePath: path.join('.pi', filename), size, contentType }
|
||||
res.mimeType && res.mimeType !== 'application/octet-stream'
|
||||
? res.mimeType
|
||||
: info?.contentType && info.contentType !== 'application/octet-stream'
|
||||
? info.contentType
|
||||
: getFileMimeType(null, filename)
|
||||
return { filePath, size, contentType }
|
||||
}
|
||||
|
||||
/** Downloads one storage key (e.g. a frame from read_asset) after checking Read on its asset. */
|
||||
async function downloadKey(key: string, userId: string): Promise<AgentToolResult> {
|
||||
async function downloadKey(key: string, ctx: AgentOperationContext): Promise<AgentToolResult> {
|
||||
const userId = ctx.userId || ''
|
||||
const owningAssetId = await resolveAssetIdFromKey(key)
|
||||
await authzService.hasPermission({
|
||||
user: { id: userId } as User,
|
||||
@@ -57,7 +71,7 @@ async function downloadKey(key: string, userId: string): Promise<AgentToolResult
|
||||
})
|
||||
await assertStorageKeyNotArchived(key)
|
||||
const filename = sanitizeFilename(path.basename(key)) || 'downloaded_file'
|
||||
const saved = await saveToSandbox(key, filename)
|
||||
const saved = await saveToRunner(key, filename, ctx)
|
||||
return {
|
||||
output: [
|
||||
{
|
||||
@@ -69,15 +83,19 @@ async function downloadKey(key: string, userId: string): Promise<AgentToolResult
|
||||
}
|
||||
}
|
||||
|
||||
/** Saves each file of a download_asset result into the sandbox. */
|
||||
export async function presentDownloadOutput(output: DownloadAssetOutput): Promise<AgentToolResult> {
|
||||
/** Saves each file of a download_asset result into the runner's .pi directory. */
|
||||
export async function presentDownloadOutput(
|
||||
output: DownloadAssetOutput,
|
||||
ctx: AgentOperationContext,
|
||||
): Promise<AgentToolResult> {
|
||||
const lines: string[] = []
|
||||
const files = []
|
||||
for (const file of output.files) {
|
||||
if (!file.key) throw new Error(`download_asset returned no storage key for ${file.assetId}`)
|
||||
const saved = await saveToSandbox(
|
||||
const saved = await saveToRunner(
|
||||
file.key,
|
||||
`${file.assetId}_${sanitizeFilename(file.name) || 'file'}`,
|
||||
ctx,
|
||||
)
|
||||
lines.push(
|
||||
`Downloaded "${file.name}" (ID: ${file.assetId}, ${file.kind}, ${saved.contentType}, ${formatSize(saved.size)}) to "${saved.filePath}".`,
|
||||
@@ -90,7 +108,7 @@ export async function presentDownloadOutput(output: DownloadAssetOutput): Promis
|
||||
|
||||
const downloadAssetExtension: AgentToolExtension = {
|
||||
description:
|
||||
`Downloads up to ${MAX_DOWNLOAD_ASSETS} workspace files into the local .pi directory, for processing with local scripts or bash commands. ` +
|
||||
`Downloads up to ${MAX_DOWNLOAD_ASSETS} workspace files into the runner's .pi directory, for processing with local scripts or bash commands. ` +
|
||||
'By default it saves the web preview version (proxy) when there is one, e.g. the transcoded video or the web image; set proxy to false for the original uploaded file. ' +
|
||||
'A version stack downloads its latest version; folders are not supported. ' +
|
||||
'To download a specific artifact from read_asset (e.g. a frame or page), pass its storage key as "key" instead of assetIds. ' +
|
||||
@@ -116,9 +134,9 @@ const downloadAssetExtension: AgentToolExtension = {
|
||||
throw new Error('Provide either "assetIds" or "key", not both.')
|
||||
}
|
||||
if (!ctx.userId) throw new Error('User ID is required for authorization.')
|
||||
return downloadKey(extra.key, ctx.userId)
|
||||
return downloadKey(extra.key, ctx)
|
||||
},
|
||||
present: (output) => presentDownloadOutput(output as DownloadAssetOutput),
|
||||
present: (output, _extra, ctx) => presentDownloadOutput(output as DownloadAssetOutput, ctx),
|
||||
}
|
||||
|
||||
registerAgentToolExtension('download_asset', downloadAssetExtension)
|
||||
|
||||
@@ -142,7 +142,7 @@ export function createAutofillReadAssetTool(access: {
|
||||
description: readAsset.description,
|
||||
parameters: toToolParameters(op, readAssetExtension),
|
||||
execute: async (params) => {
|
||||
const { input, extra } = splitAgentParams(op, readAssetExtension, params)
|
||||
const { input, extra } = await splitAgentParams(op, readAssetExtension, params)
|
||||
const parsed = readAsset.input.safeParse(input)
|
||||
if (!parsed.success) {
|
||||
throw new Error(`Invalid input for read_asset: ${z.prettifyError(parsed.error)}`)
|
||||
|
||||
@@ -5,18 +5,22 @@ import { auditLogService } from '@shumai/core/src/auditLog/auditLog'
|
||||
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
|
||||
import { metadataService } from '@shumai/core/src/metadata/metadata'
|
||||
import { getOperation } from '@shumai/core/src/operations'
|
||||
import { runnerService } from '@shumai/core/src/runner'
|
||||
import { s3Service } from '@shumai/core/src/s3/s3'
|
||||
import { AuditAction, type AssetInfo } from '@shumai/dtos'
|
||||
import * as fs from 'fs'
|
||||
import * as os from 'os'
|
||||
import * as path from 'path'
|
||||
import { Value } from 'typebox/value'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { fieldsToTypeBoxSchema } from '../index'
|
||||
import { toAgentTool, type AgentToolOptions } from '../operation-tools'
|
||||
import './upload'
|
||||
|
||||
const ctx = { userId: 'user-1', agentId: 'agent-1', teamId: 'team-1', projectId: 'proj-1' }
|
||||
const ctx = {
|
||||
userId: 'user-1',
|
||||
agentId: 'agent-1',
|
||||
teamId: 'team-1',
|
||||
projectId: 'proj-1',
|
||||
runnerId: null,
|
||||
}
|
||||
const bucket = process.env.S3_BUCKET || 'shumai'
|
||||
|
||||
function tool(name: 'create_file' | 'create_version', options: AgentToolOptions = {}) {
|
||||
@@ -55,14 +59,6 @@ const promptField = {
|
||||
}
|
||||
|
||||
describe('create_file and create_version agent tools', () => {
|
||||
const tempFiles: string[] = []
|
||||
const tempFile = (content: string, filename: string): string => {
|
||||
const filePath = path.join(os.tmpdir(), filename)
|
||||
fs.writeFileSync(filePath, content)
|
||||
tempFiles.push(filePath)
|
||||
return filePath
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.spyOn(authzService, 'hasPermission').mockResolvedValue(undefined)
|
||||
vi.spyOn(auditLogService, 'logAction').mockResolvedValue(null)
|
||||
@@ -83,7 +79,6 @@ describe('create_file and create_version agent tools', () => {
|
||||
vi.spyOn(assetService, 'createVersion').mockResolvedValue(assetInfo({ name: 'v2' }))
|
||||
vi.spyOn(metadataService, 'listProjectFields').mockResolvedValue([promptField] as never)
|
||||
vi.spyOn(s3Service, 'presign').mockResolvedValue('https://storage.example/upload')
|
||||
// Read file streams to the end, as real storage would.
|
||||
vi.spyOn(s3Service, 'putObject').mockImplementation(async (_bucket, _key, body) => {
|
||||
if (body && typeof body === 'object' && Symbol.asyncIterator in body) {
|
||||
for await (const chunk of body as AsyncIterable<unknown>) void chunk
|
||||
@@ -96,20 +91,46 @@ describe('create_file and create_version agent tools', () => {
|
||||
contentType: 'text/markdown',
|
||||
eTag: '',
|
||||
}))
|
||||
vi.spyOn(s3Service, 'getObject').mockResolvedValue({
|
||||
buffer: Buffer.from('# Generated content'),
|
||||
contentType: 'text/markdown',
|
||||
})
|
||||
vi.spyOn(runnerService, 'requireOnlineRunner').mockResolvedValue({
|
||||
id: 'runner-1',
|
||||
teamId: 'team-1',
|
||||
name: 'Local Mac Mini',
|
||||
keyPrefix: 'shm_rn_12345678',
|
||||
isDefault: true,
|
||||
status: 'connected',
|
||||
createdAt: '2026-11-14T00:00:00.000Z',
|
||||
updatedAt: '2026-11-14T00:00:00.000Z',
|
||||
})
|
||||
vi.spyOn(runnerService, 'statOnRunner').mockResolvedValue({
|
||||
type: 'file.stat.result',
|
||||
executionId: 'exec-stat-1',
|
||||
ok: true,
|
||||
exists: true,
|
||||
isFile: true,
|
||||
size: 17,
|
||||
mimeType: 'text/markdown',
|
||||
})
|
||||
vi.spyOn(runnerService, 'uploadFromRunner').mockResolvedValue({
|
||||
type: 'file.upload.result',
|
||||
executionId: 'exec-ul-1',
|
||||
ok: true,
|
||||
size: 17,
|
||||
})
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
for (const f of tempFiles.splice(0)) fs.rmSync(f, { force: true })
|
||||
})
|
||||
|
||||
describe('create_file', () => {
|
||||
it('uploads a local file to storage and creates the file as the agent', async () => {
|
||||
const filePath = tempFile('# Hello from disk', 'agent-create-file.md')
|
||||
|
||||
it('uploads a file from the runner to storage and creates the file as the agent', async () => {
|
||||
const result = await invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
path: filePath,
|
||||
path: '.pi/agent-create-file.md',
|
||||
} as never)
|
||||
|
||||
expect(authzService.hasPermission).toHaveBeenCalledWith({
|
||||
@@ -118,16 +139,21 @@ describe('create_file and create_version agent tools', () => {
|
||||
type: ResourceType.Asset,
|
||||
id: 'folder-1',
|
||||
})
|
||||
const [putBucket, key, , size, contentType] = vi.mocked(s3Service.putObject).mock.calls[0]
|
||||
expect(putBucket).toBe(bucket)
|
||||
expect(key).toMatch(/^files\/.+\/agent-create-file\.md$/)
|
||||
expect(size).toBe(Buffer.byteLength('# Hello from disk'))
|
||||
expect(contentType).toBe('text/markdown')
|
||||
expect(runnerService.requireOnlineRunner).toHaveBeenCalledWith('team-1', null)
|
||||
expect(runnerService.statOnRunner).toHaveBeenCalledWith(
|
||||
'runner-1',
|
||||
'.pi/agent-create-file.md',
|
||||
)
|
||||
expect(runnerService.uploadFromRunner).toHaveBeenCalledWith('runner-1', {
|
||||
srcPath: '.pi/agent-create-file.md',
|
||||
url: 'https://storage.example/upload',
|
||||
contentType: 'text/markdown',
|
||||
})
|
||||
|
||||
expect(assetService.createFile).toHaveBeenCalledWith({
|
||||
parentId: 'folder-1',
|
||||
name: 'agent-create-file.md',
|
||||
key,
|
||||
key: expect.stringMatching(/^files\/.+\/agent-create-file\.md$/),
|
||||
sizeByte: 17,
|
||||
contentType: 'text/markdown',
|
||||
creatorId: 'user-1',
|
||||
@@ -155,6 +181,90 @@ describe('create_file and create_version agent tools', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('falls back to temporary S3 storage when no runner is configured for .pi/generated/', async () => {
|
||||
vi.mocked(runnerService.requireOnlineRunner).mockRejectedValueOnce(
|
||||
new Error('No Runner is configured for this team.'),
|
||||
)
|
||||
|
||||
await invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
path: '.pi/generated/gen123/poster.png',
|
||||
} as never)
|
||||
|
||||
expect(runnerService.statOnRunner).not.toHaveBeenCalled()
|
||||
expect(s3Service.getObject).toHaveBeenCalledWith(bucket, 'temp/generated/gen123/poster.png')
|
||||
expect(s3Service.putObject).toHaveBeenCalled()
|
||||
expect(assetService.createFile).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
parentId: 'folder-1',
|
||||
name: 'poster.png',
|
||||
}),
|
||||
)
|
||||
})
|
||||
|
||||
it('falls back to temporary S3 storage when .pi/generated/ file is not staged on the online runner', async () => {
|
||||
vi.mocked(runnerService.statOnRunner).mockResolvedValueOnce({
|
||||
type: 'file.stat.result',
|
||||
executionId: 'exec-stat-missing',
|
||||
ok: true,
|
||||
exists: false,
|
||||
})
|
||||
|
||||
await invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
path: '.pi/generated/gen123/unstaged.png',
|
||||
} as never)
|
||||
|
||||
expect(runnerService.statOnRunner).toHaveBeenCalledWith(
|
||||
'runner-1',
|
||||
'.pi/generated/gen123/unstaged.png',
|
||||
)
|
||||
expect(s3Service.getObject).toHaveBeenCalledWith(bucket, 'temp/generated/gen123/unstaged.png')
|
||||
expect(runnerService.uploadFromRunner).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('prefers uploading edited or newly created .pi/generated/ files from the runner over S3', async () => {
|
||||
vi.mocked(s3Service.headObject).mockImplementation(async (_bucket, key) => {
|
||||
if (key === 'temp/generated/gen123/edited.png') {
|
||||
throw new Error('NotFound')
|
||||
}
|
||||
return {
|
||||
key,
|
||||
size: 17,
|
||||
lastModified: new Date(),
|
||||
contentType: 'text/markdown',
|
||||
eTag: '',
|
||||
}
|
||||
})
|
||||
|
||||
await invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
path: '.pi/generated/gen123/edited.png',
|
||||
} as never)
|
||||
|
||||
expect(runnerService.statOnRunner).toHaveBeenCalledWith(
|
||||
'runner-1',
|
||||
'.pi/generated/gen123/edited.png',
|
||||
)
|
||||
expect(runnerService.uploadFromRunner).toHaveBeenCalledWith('runner-1', {
|
||||
srcPath: '.pi/generated/gen123/edited.png',
|
||||
url: 'https://storage.example/upload',
|
||||
contentType: 'text/markdown',
|
||||
})
|
||||
expect(s3Service.getObject).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('reads explicit temp/generated/ paths directly from S3 without touching the runner', async () => {
|
||||
await invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
path: 'temp/generated/gen123/direct.png',
|
||||
} as never)
|
||||
|
||||
expect(runnerService.requireOnlineRunner).not.toHaveBeenCalled()
|
||||
expect(runnerService.statOnRunner).not.toHaveBeenCalled()
|
||||
expect(s3Service.getObject).toHaveBeenCalledWith(bucket, 'temp/generated/gen123/direct.png')
|
||||
})
|
||||
|
||||
it('creates a text file from name and content', async () => {
|
||||
await invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
@@ -199,11 +309,9 @@ describe('create_file and create_version agent tools', () => {
|
||||
})
|
||||
|
||||
it('forwards Creation Context metadata, including new select options', async () => {
|
||||
const filePath = tempFile('# Hello', 'agent-create-file-meta.md')
|
||||
|
||||
await invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
path: filePath,
|
||||
path: '.pi/agent-create-file-meta.md',
|
||||
metadata: { prompt: { newOption: { value: 'Kling' } } },
|
||||
} as never)
|
||||
|
||||
@@ -213,11 +321,9 @@ describe('create_file and create_version agent tools', () => {
|
||||
})
|
||||
|
||||
it('treats null metadata as omitted', async () => {
|
||||
const filePath = tempFile('# Hello', 'agent-create-file-null.md')
|
||||
|
||||
await invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
path: filePath,
|
||||
path: '.pi/agent-create-file-null.md',
|
||||
metadata: null,
|
||||
} as never)
|
||||
|
||||
@@ -225,17 +331,15 @@ describe('create_file and create_version agent tools', () => {
|
||||
})
|
||||
|
||||
it('rejects metadata keys that are not Creation Context fields before uploading', async () => {
|
||||
const filePath = tempFile('# Hello', 'agent-create-file-bad-meta.md')
|
||||
|
||||
await expect(
|
||||
invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
path: filePath,
|
||||
path: '.pi/agent-create-file-bad-meta.md',
|
||||
metadata: { status: 'approved' },
|
||||
} as never),
|
||||
).rejects.toThrow(/not allowed: status/)
|
||||
|
||||
expect(s3Service.putObject).not.toHaveBeenCalled()
|
||||
expect(runnerService.uploadFromRunner).not.toHaveBeenCalled()
|
||||
expect(assetService.createFile).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
@@ -260,14 +364,20 @@ describe('create_file and create_version agent tools', () => {
|
||||
).rejects.toThrow('"name" is required')
|
||||
})
|
||||
|
||||
it('fails when the local file does not exist, without touching storage', async () => {
|
||||
it('fails when the file does not exist on the runner, without touching storage', async () => {
|
||||
vi.mocked(runnerService.statOnRunner).mockResolvedValue({
|
||||
type: 'file.stat.result',
|
||||
executionId: 'exec-stat-2',
|
||||
ok: true,
|
||||
exists: false,
|
||||
})
|
||||
await expect(
|
||||
invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
path: '/nonexistent/file.md',
|
||||
path: '.pi/nonexistent.md',
|
||||
} as never),
|
||||
).rejects.toThrow('Local file not found at path: /nonexistent/file.md')
|
||||
expect(s3Service.putObject).not.toHaveBeenCalled()
|
||||
).rejects.toThrow('Local file not found at path: .pi/nonexistent.md')
|
||||
expect(runnerService.uploadFromRunner).not.toHaveBeenCalled()
|
||||
expect(assetService.createFile).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
@@ -275,10 +385,12 @@ describe('create_file and create_version agent tools', () => {
|
||||
vi.mocked(authzService.hasPermission).mockRejectedValue(
|
||||
new Error('User is not a member of the team'),
|
||||
)
|
||||
const filePath = tempFile('# Not allowed', 'agent-create-file-denied.md')
|
||||
|
||||
await expect(
|
||||
invokeTool(tool('create_file'), 'call-1', { parent: 'folder-1', path: filePath } as never),
|
||||
invokeTool(tool('create_file'), 'call-1', {
|
||||
parent: 'folder-1',
|
||||
path: '.pi/agent-create-file-denied.md',
|
||||
} as never),
|
||||
).rejects.toThrow('User is not a member of the team')
|
||||
await expect(
|
||||
invokeTool(tool('create_file'), 'call-1', {
|
||||
@@ -290,6 +402,7 @@ describe('create_file and create_version agent tools', () => {
|
||||
|
||||
expect(s3Service.putObject).not.toHaveBeenCalled()
|
||||
expect(s3Service.presign).not.toHaveBeenCalled()
|
||||
expect(runnerService.uploadFromRunner).not.toHaveBeenCalled()
|
||||
expect(assetService.createFile).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
@@ -322,19 +435,21 @@ describe('create_file and create_version agent tools', () => {
|
||||
})
|
||||
|
||||
describe('create_version', () => {
|
||||
it('uploads a local file and creates a version as the agent', async () => {
|
||||
const filePath = tempFile('# Hello from disk', 'agent-create-version.md')
|
||||
|
||||
it('uploads a file from the runner and creates a version as the agent', async () => {
|
||||
const result = await invokeTool(tool('create_version'), 'call-1', {
|
||||
parent: 'file-1',
|
||||
path: filePath,
|
||||
path: '.pi/agent-create-version.md',
|
||||
} as never)
|
||||
|
||||
const key = vi.mocked(s3Service.putObject).mock.calls[0][1]
|
||||
expect(runnerService.uploadFromRunner).toHaveBeenCalledWith('runner-1', {
|
||||
srcPath: '.pi/agent-create-version.md',
|
||||
url: 'https://storage.example/upload',
|
||||
contentType: 'text/markdown',
|
||||
})
|
||||
expect(assetService.createVersion).toHaveBeenCalledWith({
|
||||
parentId: 'file-1',
|
||||
name: 'agent-create-version.md',
|
||||
key,
|
||||
key: expect.stringMatching(/^files\/.+\/agent-create-version\.md$/),
|
||||
sizeByte: 17,
|
||||
contentType: 'text/markdown',
|
||||
creatorId: 'user-1',
|
||||
@@ -348,11 +463,9 @@ describe('create_file and create_version agent tools', () => {
|
||||
})
|
||||
|
||||
it('forwards Creation Context metadata', async () => {
|
||||
const filePath = tempFile('# Hello', 'agent-create-version-meta.md')
|
||||
|
||||
await invokeTool(tool('create_version'), 'call-1', {
|
||||
parent: 'file-1',
|
||||
path: filePath,
|
||||
path: '.pi/agent-create-version-meta.md',
|
||||
metadata: { prompt: 'Generated using gemini' },
|
||||
} as never)
|
||||
|
||||
@@ -365,23 +478,31 @@ describe('create_file and create_version agent tools', () => {
|
||||
vi.mocked(authzService.hasPermission).mockRejectedValueOnce(
|
||||
new Error('User is not a member of the team'),
|
||||
)
|
||||
const filePath = tempFile('# Not allowed', 'agent-create-version-denied.md')
|
||||
|
||||
await expect(
|
||||
invokeTool(tool('create_version'), 'call-1', { parent: 'file-1', path: filePath } as never),
|
||||
invokeTool(tool('create_version'), 'call-1', {
|
||||
parent: 'file-1',
|
||||
path: '.pi/agent-create-version-denied.md',
|
||||
} as never),
|
||||
).rejects.toThrow('User is not a member of the team')
|
||||
expect(s3Service.putObject).not.toHaveBeenCalled()
|
||||
expect(runnerService.uploadFromRunner).not.toHaveBeenCalled()
|
||||
expect(assetService.createVersion).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('fails when the local file does not exist', async () => {
|
||||
vi.mocked(runnerService.statOnRunner).mockResolvedValue({
|
||||
type: 'file.stat.result',
|
||||
executionId: 'exec-stat-3',
|
||||
ok: true,
|
||||
exists: false,
|
||||
})
|
||||
await expect(
|
||||
invokeTool(tool('create_version'), 'call-1', {
|
||||
parent: 'file-1',
|
||||
path: '/nonexistent/file.md',
|
||||
path: '.pi/nonexistent.md',
|
||||
} as never),
|
||||
).rejects.toThrow('Local file not found at path: /nonexistent/file.md')
|
||||
expect(s3Service.putObject).not.toHaveBeenCalled()
|
||||
).rejects.toThrow('Local file not found at path: .pi/nonexistent.md')
|
||||
expect(runnerService.uploadFromRunner).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('shows the agent its own parameters', () => {
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
import type { ToolExecutionResult as AgentToolResult } from '@earendil-works/pi-durable'
|
||||
import { storageBucket } from '@shumai/core/src/asset-read'
|
||||
import { completeUploadOp, runOperation } from '@shumai/core/src/operations'
|
||||
import { runnerService } from '@shumai/core/src/runner'
|
||||
import { s3Service } from '@shumai/core/src/s3/s3'
|
||||
import { getFileMimeType, readFileMimeType } from '@shumai/core/src/utils/file-mime'
|
||||
import { getFileMimeType } from '@shumai/core/src/utils/file-mime'
|
||||
import { createFile, type StartedUploadOutput } from '@shumai/operations'
|
||||
import * as fs from 'fs'
|
||||
import * as path from 'path'
|
||||
import { Type } from 'typebox'
|
||||
import { z } from 'zod'
|
||||
@@ -32,17 +32,126 @@ function setCreationMetadata(schema: ToolParametersSchema, options: AgentToolOpt
|
||||
}
|
||||
}
|
||||
|
||||
function resolveLocalFile(localPath: string): string {
|
||||
const absolutePath = path.resolve(process.cwd(), localPath)
|
||||
if (!fs.existsSync(absolutePath) || !fs.statSync(absolutePath).isFile()) {
|
||||
function resolveGeneratedStorageKey(rawPath: string): {
|
||||
storageKey: string
|
||||
explicitStorageRef: boolean
|
||||
} | null {
|
||||
const cleaned = rawPath.replace(/\\/g, '/').replace(/^\.\//, '')
|
||||
if (cleaned.includes('..')) return null
|
||||
if (cleaned.startsWith('.pi/generated/')) {
|
||||
const rest = cleaned.slice('.pi/generated/'.length)
|
||||
return rest ? { storageKey: `temp/generated/${rest}`, explicitStorageRef: false } : null
|
||||
}
|
||||
if (cleaned.startsWith('temp/generated/')) {
|
||||
const rest = cleaned.slice('temp/generated/'.length)
|
||||
return rest ? { storageKey: cleaned, explicitStorageRef: true } : null
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
function normalizeRunnerPath(rawPath: string): string | null {
|
||||
const cleaned = rawPath.replace(/\\/g, '/').replace(/^\.\//, '')
|
||||
if (!cleaned || cleaned.startsWith('/') || cleaned.includes('..')) {
|
||||
return null
|
||||
}
|
||||
const normalized = path.posix.normalize(cleaned)
|
||||
if (normalized.startsWith('..') || normalized.startsWith('/')) {
|
||||
return null
|
||||
}
|
||||
if (normalized === '.pi' || normalized.startsWith('.pi/')) {
|
||||
return normalized
|
||||
}
|
||||
return `.pi/${normalized}`
|
||||
}
|
||||
|
||||
async function resolveTargetRunnerId(ctx?: AgentOperationContext): Promise<string> {
|
||||
if (ctx?.runnerId && !ctx.teamId) {
|
||||
if (!runnerService.isRunnerConnected(ctx.runnerId)) {
|
||||
throw new Error(
|
||||
'Runner is currently offline. Please start your runner with "shumai-cli runner start" and try again.',
|
||||
)
|
||||
}
|
||||
return ctx.runnerId
|
||||
}
|
||||
const runner = await runnerService.requireOnlineRunner(ctx?.teamId || '', ctx?.runnerId)
|
||||
return runner.id
|
||||
}
|
||||
|
||||
async function resolveStorageGeneratedSource(storageKey: string): Promise<{
|
||||
kind: 'generated'
|
||||
storageKey: string
|
||||
name: string
|
||||
contentType: string
|
||||
} | null> {
|
||||
const head = await s3Service.headObject(storageBucket(), storageKey).catch(() => null)
|
||||
if (!head) {
|
||||
return null
|
||||
}
|
||||
const name = path.posix.basename(storageKey)
|
||||
const contentType =
|
||||
head.contentType && head.contentType !== 'application/octet-stream'
|
||||
? head.contentType
|
||||
: getFileMimeType(null, name)
|
||||
return { kind: 'generated', storageKey, name, contentType }
|
||||
}
|
||||
|
||||
async function resolvePathSource(
|
||||
localPath: string,
|
||||
ctx?: AgentOperationContext,
|
||||
): Promise<
|
||||
| { kind: 'generated'; storageKey: string; name: string; contentType: string }
|
||||
| { kind: 'runner'; runnerId: string; runnerPath: string; name: string; contentType: string }
|
||||
> {
|
||||
const generatedRef = resolveGeneratedStorageKey(localPath)
|
||||
if (generatedRef?.explicitStorageRef) {
|
||||
const source = await resolveStorageGeneratedSource(generatedRef.storageKey)
|
||||
if (!source) {
|
||||
throw new Error(`Local file not found at path: ${localPath}`)
|
||||
}
|
||||
return source
|
||||
}
|
||||
|
||||
const runnerPath = normalizeRunnerPath(localPath)
|
||||
if (!runnerPath) {
|
||||
throw new Error(`Local file not found at path: ${localPath}`)
|
||||
}
|
||||
return absolutePath
|
||||
|
||||
let runnerError: unknown
|
||||
try {
|
||||
const runnerId = await resolveTargetRunnerId(ctx)
|
||||
const stat = await runnerService.statOnRunner(runnerId, runnerPath)
|
||||
if (stat.ok && stat.exists && stat.isFile !== false) {
|
||||
const name = path.posix.basename(runnerPath)
|
||||
const contentType =
|
||||
stat.mimeType && stat.mimeType !== 'application/octet-stream'
|
||||
? stat.mimeType
|
||||
: getFileMimeType(null, name)
|
||||
return { kind: 'runner', runnerId, runnerPath, name, contentType }
|
||||
}
|
||||
} catch (err) {
|
||||
if (!generatedRef) {
|
||||
throw err
|
||||
}
|
||||
runnerError = err
|
||||
}
|
||||
|
||||
// Fallback for unstaged generated files stored in temporary S3
|
||||
if (generatedRef) {
|
||||
const source = await resolveStorageGeneratedSource(generatedRef.storageKey)
|
||||
if (source) {
|
||||
return source
|
||||
}
|
||||
if (runnerError) {
|
||||
throw runnerError
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error(`Local file not found at path: ${localPath}`)
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes the file straight to the upload's storage key (the agent runs next to storage, so
|
||||
* it doesn't need the upload URL), then completes the upload as the agent.
|
||||
* Transfers the file from either temporary generated storage or the remote runner workspace
|
||||
* into the upload's storage key, then completes the upload as the agent.
|
||||
*/
|
||||
async function uploadAndComplete(
|
||||
raw: unknown,
|
||||
@@ -53,14 +162,29 @@ async function uploadAndComplete(
|
||||
if (!started.key) throw new Error('The upload did not return a storage key')
|
||||
|
||||
if (typeof extra.path === 'string') {
|
||||
const absolutePath = resolveLocalFile(extra.path)
|
||||
await s3Service.putObject(
|
||||
storageBucket(),
|
||||
started.key,
|
||||
fs.createReadStream(absolutePath),
|
||||
fs.statSync(absolutePath).size,
|
||||
readFileMimeType(absolutePath),
|
||||
)
|
||||
const source =
|
||||
(extra.__resolvedSource as Awaited<ReturnType<typeof resolvePathSource>> | undefined) ??
|
||||
(await resolvePathSource(extra.path, ctx))
|
||||
|
||||
if (source.kind === 'generated') {
|
||||
const obj = await s3Service.getObject(storageBucket(), source.storageKey)
|
||||
await s3Service.putObject(
|
||||
storageBucket(),
|
||||
started.key,
|
||||
obj.buffer,
|
||||
obj.buffer.length,
|
||||
source.contentType,
|
||||
)
|
||||
} else {
|
||||
const uploadRes = await runnerService.uploadFromRunner(source.runnerId, {
|
||||
srcPath: source.runnerPath,
|
||||
url: started.uploadUrl,
|
||||
contentType: source.contentType,
|
||||
})
|
||||
if (!uploadRes.ok) {
|
||||
throw new Error(uploadRes.error || `Failed to upload file "${extra.path}" from runner`)
|
||||
}
|
||||
}
|
||||
} else {
|
||||
const content = Buffer.from(typeof extra.content === 'string' ? extra.content : '', 'utf-8')
|
||||
// The key ends with the file name, which gives the same type as the upload's.
|
||||
@@ -85,7 +209,7 @@ async function uploadAndComplete(
|
||||
const createFileExtension: AgentToolExtension = {
|
||||
description:
|
||||
'Create a new file in a folder. Choose exactly ONE source: ' +
|
||||
'(1) "path" to upload a local file (e.g. one you made in .pi), OR ' +
|
||||
'(1) "path" to upload a file from .pi (e.g. a generated file in .pi/generated/... or a file created in .pi on the runner), OR ' +
|
||||
'(2) "name" and "content" to create a text file with that content. ' +
|
||||
METADATA_HINT,
|
||||
parameterOverrides: z.object({
|
||||
@@ -100,7 +224,9 @@ const createFileExtension: AgentToolExtension = {
|
||||
.string()
|
||||
.min(1)
|
||||
.optional()
|
||||
.describe('Absolute or relative path of a local file to upload. Omit when using "content".'),
|
||||
.describe(
|
||||
'Path of a file in .pi (e.g. ".pi/generated/..." or ".pi/output.mp4") to upload. Omit when using "content".',
|
||||
),
|
||||
content: z
|
||||
.string()
|
||||
.optional()
|
||||
@@ -108,18 +234,19 @@ const createFileExtension: AgentToolExtension = {
|
||||
}),
|
||||
hiddenParameters: ['contentType'],
|
||||
adjustParameters: setCreationMetadata,
|
||||
toInput(input, extra) {
|
||||
async toInput(input, extra, ctx) {
|
||||
const hasPath = typeof extra.path === 'string'
|
||||
const hasContent = typeof extra.content === 'string'
|
||||
if (hasPath === hasContent) {
|
||||
throw new Error('Provide exactly one of "path" (a local file) or "content" (with "name").')
|
||||
}
|
||||
if (hasPath) {
|
||||
const absolutePath = resolveLocalFile(extra.path as string)
|
||||
const source = await resolvePathSource(extra.path as string, ctx)
|
||||
extra.__resolvedSource = source
|
||||
return {
|
||||
...input,
|
||||
name: input.name ?? path.basename(absolutePath),
|
||||
contentType: readFileMimeType(absolutePath),
|
||||
name: input.name ?? source.name,
|
||||
contentType: source.contentType,
|
||||
}
|
||||
}
|
||||
if (typeof input.name !== 'string' || !input.name.trim()) {
|
||||
@@ -132,23 +259,26 @@ const createFileExtension: AgentToolExtension = {
|
||||
|
||||
const createVersionExtension: AgentToolExtension = {
|
||||
description:
|
||||
'Create a new version of an existing file (or version stack) from a local file path. ' +
|
||||
'Create a new version of an existing file (or version stack) from a file path in .pi. ' +
|
||||
'The new version becomes the current one. ' +
|
||||
METADATA_HINT,
|
||||
extraParameters: z.object({
|
||||
path: z
|
||||
.string()
|
||||
.min(1)
|
||||
.describe('Absolute or relative path of the local file to upload as the new version.'),
|
||||
.describe(
|
||||
'Path of the file in .pi (e.g. ".pi/generated/..." or ".pi/output.mp4") to upload as the new version.',
|
||||
),
|
||||
}),
|
||||
hiddenParameters: ['name', 'contentType'],
|
||||
adjustParameters: setCreationMetadata,
|
||||
toInput(input, extra) {
|
||||
const absolutePath = resolveLocalFile(extra.path as string)
|
||||
async toInput(input, extra, ctx) {
|
||||
const source = await resolvePathSource(extra.path as string, ctx)
|
||||
extra.__resolvedSource = source
|
||||
return {
|
||||
...input,
|
||||
name: path.basename(absolutePath),
|
||||
contentType: readFileMimeType(absolutePath),
|
||||
name: source.name,
|
||||
contentType: source.contentType,
|
||||
}
|
||||
},
|
||||
present: uploadAndComplete,
|
||||
|
||||
@@ -14,7 +14,9 @@ import { Compile } from 'typebox/compile'
|
||||
import { z } from 'zod'
|
||||
|
||||
/** Context for agent-invoked Operations; `via` is always `agent`. */
|
||||
export type AgentOperationContext = Omit<OperationContext, 'via'>
|
||||
export type AgentOperationContext = Omit<OperationContext, 'via'> & {
|
||||
runnerId?: string | null
|
||||
}
|
||||
|
||||
/** Per-session settings for building tools, e.g. schemas that depend on the project. */
|
||||
export interface AgentToolOptions {
|
||||
@@ -23,6 +25,10 @@ export interface AgentToolOptions {
|
||||
* parameter of tools that create files. Undefined when the project has none.
|
||||
*/
|
||||
creationMetadataSchema?: TSchema
|
||||
/**
|
||||
* Operation names to exclude from the built tool list (e.g. `download_asset` when no runner is configured).
|
||||
*/
|
||||
excludeOperations?: string[]
|
||||
}
|
||||
|
||||
/** The JSON Schema of a tool's parameters (an object schema). */
|
||||
@@ -54,7 +60,11 @@ export interface AgentToolExtension {
|
||||
/** Adjust the parameters' JSON Schema for this session (e.g. a project-specific field). */
|
||||
adjustParameters?(schema: ToolParametersSchema, options: AgentToolOptions): void
|
||||
/** Adjust the Operation input using the extra parameters. */
|
||||
toInput?(input: Record<string, unknown>, extra: Record<string, unknown>): Record<string, unknown>
|
||||
toInput?(
|
||||
input: Record<string, unknown>,
|
||||
extra: Record<string, unknown>,
|
||||
ctx?: AgentOperationContext,
|
||||
): Promise<Record<string, unknown>> | Record<string, unknown>
|
||||
/**
|
||||
* Handle an agent-only mode without running the Operation (e.g. a storage key instead of
|
||||
* an asset). Return undefined to run the Operation.
|
||||
@@ -110,11 +120,12 @@ export function toToolParameters(
|
||||
}
|
||||
|
||||
/** Split agent-only parameters from the Operation input. */
|
||||
export function splitAgentParams(
|
||||
export async function splitAgentParams(
|
||||
op: Operation,
|
||||
extension: AgentToolExtension | undefined,
|
||||
params: unknown,
|
||||
): { input: unknown; extra: Record<string, unknown> } {
|
||||
ctx?: AgentOperationContext,
|
||||
): Promise<{ input: unknown; extra: Record<string, unknown> }> {
|
||||
const cleaned = dropNullOptionals(toolInputSchema(op, extension), params)
|
||||
if (!extension?.extraParameters || !cleaned || typeof cleaned !== 'object') {
|
||||
return { input: cleaned, extra: {} }
|
||||
@@ -125,7 +136,10 @@ export function splitAgentParams(
|
||||
for (const [key, value] of Object.entries(cleaned)) {
|
||||
;(extraKeys.has(key) ? extra : input)[key] = value
|
||||
}
|
||||
return { input: extension.toInput ? extension.toInput(input, extra) : input, extra }
|
||||
return {
|
||||
input: extension.toInput ? await extension.toInput(input, extra, ctx) : input,
|
||||
extra,
|
||||
}
|
||||
}
|
||||
|
||||
/** Expose an Operation as a native agent tool that runs through `runOperation`. */
|
||||
@@ -141,7 +155,7 @@ export function toAgentTool(
|
||||
description: extension?.description ?? definition.description,
|
||||
parameters: toToolParameters(op, extension, options),
|
||||
execute: async (params) => {
|
||||
const { input, extra } = splitAgentParams(op, extension, params)
|
||||
const { input, extra } = await splitAgentParams(op, extension, params, ctx)
|
||||
if (extension?.intercept) {
|
||||
const handled = await extension.intercept(input as Record<string, unknown>, extra, ctx)
|
||||
if (handled) return handled
|
||||
@@ -163,5 +177,8 @@ export function buildOperationTools(
|
||||
ctx: AgentOperationContext,
|
||||
options: AgentToolOptions = {},
|
||||
): AgentTool[] {
|
||||
return getOperationsForSurface('agent').map((op) => toAgentTool(op, ctx, options))
|
||||
const excluded = new Set(options.excludeOperations ?? [])
|
||||
return getOperationsForSurface('agent')
|
||||
.filter((op) => !excluded.has(op.definition.name))
|
||||
.map((op) => toAgentTool(op, ctx, options))
|
||||
}
|
||||
|
||||
@@ -25,7 +25,6 @@ import {
|
||||
hook,
|
||||
type ToolRegistration as AgentTool,
|
||||
} from '@earendil-works/pi-durable'
|
||||
import { sandboxService } from '@shumai/core'
|
||||
import { agentService } from '@shumai/core/src/agent/agent'
|
||||
import { hydrateRequest } from '@shumai/core/src/agent/session-content'
|
||||
import { agentSessionService } from '@shumai/core/src/agent/session-service'
|
||||
@@ -34,9 +33,9 @@ import { logger } from '@shumai/core/src/logger'
|
||||
import { mediaGenerationService } from '@shumai/core/src/media-generation/media-generation'
|
||||
import { metadataService } from '@shumai/core/src/metadata/metadata'
|
||||
import { QuotaExceededError, quotaService } from '@shumai/core/src/quota/quota-service'
|
||||
import { runnerService } from '@shumai/core/src/runner'
|
||||
import { prisma } from '@shumai/db'
|
||||
import { DEFAULT_DENIED_TOOLS } from '@shumai/dtos'
|
||||
import * as fs from 'fs'
|
||||
import * as path from 'path'
|
||||
import { Type, type TSchema } from 'typebox'
|
||||
import { createAutofillReadAssetTool } from './operation-extensions'
|
||||
@@ -45,7 +44,7 @@ import { createGenerateImageTool } from './tools/generate-image'
|
||||
import { createGenerateVideoTool } from './tools/generate-video'
|
||||
import { createReadSkillTool } from './tools/read-skill'
|
||||
import { createReadThreadTool } from './tools/read-thread'
|
||||
import { createSandboxedBashTool } from './tools/sandboxed-bash'
|
||||
import { createRunnerBashTool, type LoadedSkillRef } from './tools/runner-bash'
|
||||
|
||||
const activeHarnesses = new Set<Harness>()
|
||||
|
||||
@@ -153,21 +152,6 @@ export interface CreateAgentSessionParams {
|
||||
baseDelayMs?: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Security instructions appended to the system prompt when the requesting user is not a team
|
||||
* owner (i.e. editor/reviewer). These complement the hard enforcement in the bash tool itself
|
||||
* (source="user" is blocked) and the delayed bash tool injection after a skill is loaded.
|
||||
*/
|
||||
export function buildRestrictedUserInstructions(): string {
|
||||
return [
|
||||
'# Restricted User Context',
|
||||
'The current user is NOT a team owner. You must follow these security rules:',
|
||||
'1. NEVER execute a bash command that the user asks you to run directly. Direct user bash requests are not permitted for this user role. Politely decline and explain that they need a team owner or an approved skill.',
|
||||
'2. You may ONLY use the `bash` tool to execute commands that are explicitly required by a skill you have loaded via the `read_skill` tool.',
|
||||
'3. When calling `bash`, you MUST set the `source` parameter to "skill". Calls with source="user" will be rejected by the system.',
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
export async function createAgentSession(params: CreateAgentSessionParams) {
|
||||
const {
|
||||
teamId,
|
||||
@@ -177,7 +161,6 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
|
||||
systemPrompt,
|
||||
teamSkills,
|
||||
enabledSkillIds,
|
||||
allowedDomains,
|
||||
sessionId,
|
||||
userId,
|
||||
projectId,
|
||||
@@ -202,17 +185,9 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
|
||||
})
|
||||
const storage = opened.storage
|
||||
try {
|
||||
const agentDir = path.join(process.cwd(), '.pi', 'agents', agentId)
|
||||
if (!fs.existsSync(agentDir)) fs.mkdirSync(agentDir, { recursive: true })
|
||||
|
||||
getModelFromDb(params.providers, providerName, modelId)
|
||||
|
||||
const piDir = path.join(process.cwd(), '.pi')
|
||||
if (!fs.existsSync(piDir)) fs.mkdirSync(piDir, { recursive: true })
|
||||
|
||||
const role = userId ? await resolveEffectiveRole(teamId, projectId, userId) : undefined
|
||||
const isOwner = !userId || role === 'owner'
|
||||
const restricted = !isOwner
|
||||
|
||||
const wrapToolWithQuota = (tool: AgentTool): AgentTool => {
|
||||
const originalExecute = tool.execute
|
||||
@@ -262,17 +237,14 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
|
||||
}
|
||||
}
|
||||
|
||||
if (allowedDomains !== undefined) {
|
||||
await sandboxService.syncAllowedDomains(allowedDomains, teamId)
|
||||
}
|
||||
|
||||
const skillEnvs: Record<string, string> = {}
|
||||
const onEnvsAdded = (envs: Record<string, string>) => {
|
||||
Object.assign(skillEnvs, envs)
|
||||
}
|
||||
|
||||
// Restore env variables from previously loaded skills in this session
|
||||
let skillLoadedInSession = false
|
||||
const loadedSkills = new Map<string, LoadedSkillRef>()
|
||||
|
||||
// Restore env variables and loaded skill refs from previously loaded skills in this session
|
||||
if (sessionId) {
|
||||
try {
|
||||
const entries = await agentSessionService.getEntries(record.id)
|
||||
@@ -291,8 +263,14 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
|
||||
'skillId' in details &&
|
||||
typeof details.skillId === 'string'
|
||||
) {
|
||||
skillLoadedInSession = true
|
||||
onEnvsAdded(await agentService.getSkillEnvs(details.skillId))
|
||||
const dbSkill = await prisma.skill.findUnique({
|
||||
where: { id: details.skillId },
|
||||
select: { id: true, assetId: true, hash: true },
|
||||
})
|
||||
if (dbSkill) {
|
||||
loadedSkills.set(dbSkill.id, dbSkill)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -308,6 +286,9 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
|
||||
const agentConfig = agent?.config as PrismaJson.AgentConfig | null | undefined
|
||||
const deniedTools = agentConfig?.deniedTools ?? [...DEFAULT_DENIED_TOOLS]
|
||||
|
||||
const resolvedRunner = await runnerService.resolveRunnerForAgent(teamId, agent?.runnerId)
|
||||
const hasRunner = resolvedRunner.hasConfiguredRunner
|
||||
|
||||
const mediaTools: AgentTool[] = []
|
||||
// Agents acting for a user get read_asset / get_asset_render from the Operation registry
|
||||
// (see systemTools below); the autofill system agent has its own team-scoped read_asset.
|
||||
@@ -335,42 +316,33 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
|
||||
}
|
||||
}
|
||||
|
||||
const sandboxedBash = createSandboxedBashTool(process.cwd(), skillEnvs, {
|
||||
const runnerBash = createRunnerBashTool(skillEnvs, {
|
||||
teamId,
|
||||
agentRunnerId: agent?.runnerId,
|
||||
environment: resolvedRunner.runner
|
||||
? runnerService.getRunnerEnvironment(resolvedRunner.runner.id)
|
||||
: undefined,
|
||||
userId,
|
||||
role,
|
||||
getBlockedHost: (commandId?: string) => sandboxService.getBlockedHost(commandId),
|
||||
clearBlockedHost: () => {
|
||||
sandboxService.clearBlockedHost()
|
||||
},
|
||||
// Non-owner users can only run bash commands required by a loaded skill
|
||||
restrictedUser: restricted,
|
||||
getLoadedSkills: () => loadedSkills.values(),
|
||||
})
|
||||
|
||||
// For non-owner users, the bash tool is not injected initially. It becomes available
|
||||
// only after a skill is successfully loaded via the read_skill tool (or immediately when
|
||||
// a skill was already loaded earlier in this session).
|
||||
const registry = createRegistry()
|
||||
const bashIncludedFromStart = isOwner || skillLoadedInSession
|
||||
let bashInjected = bashIncludedFromStart
|
||||
const onSkillLoaded = async () => {
|
||||
if (bashInjected || deniedTools.includes('bash')) return
|
||||
bashInjected = true
|
||||
registry.install(
|
||||
defineExtension({ name: 'shumai.bash', tools: [wrapToolWithQuota(sandboxedBash)] }),
|
||||
)
|
||||
await session?.configure(
|
||||
{
|
||||
tools: [...enabledTools, wrapToolWithQuota(sandboxedBash)],
|
||||
modelTools: { add: [sandboxedBash] },
|
||||
},
|
||||
piContext,
|
||||
)
|
||||
const onSkillLoaded = async (skill: LoadedSkillRef) => {
|
||||
loadedSkills.set(skill.id, skill)
|
||||
if (resolvedRunner.runner && runnerService.isRunnerConnected(resolvedRunner.runner.id)) {
|
||||
await runnerService.ensureSkillOnRunner(resolvedRunner.runner.id, skill).catch((err) => {
|
||||
logger.warn(
|
||||
{ err, skillId: skill.id, runnerId: resolvedRunner.runner?.id },
|
||||
'Failed to eagerly stage skill on runner',
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
const readSkill = createReadSkillTool(
|
||||
userId,
|
||||
onEnvsAdded,
|
||||
restricted ? onSkillLoaded : undefined,
|
||||
onSkillLoaded,
|
||||
enabledSkillIds,
|
||||
projectId,
|
||||
)
|
||||
@@ -381,7 +353,13 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
|
||||
systemTools.push(
|
||||
// Shared Operations (also exposed via MCP and the CLI): browsing, searching, organizing,
|
||||
// reading and downloading assets, creating files and versions, comments and metadata
|
||||
...buildOperationTools({ userId, agentId, teamId, projectId }, { creationMetadataSchema }),
|
||||
...buildOperationTools(
|
||||
{ userId, agentId, teamId, projectId, runnerId: agent?.runnerId },
|
||||
{
|
||||
creationMetadataSchema,
|
||||
excludeOperations: hasRunner ? [] : ['download_asset'],
|
||||
},
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -390,7 +368,7 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
|
||||
...mediaTools,
|
||||
readSkill,
|
||||
readThread,
|
||||
...(bashIncludedFromStart ? [sandboxedBash] : []),
|
||||
...(hasRunner ? [runnerBash] : []),
|
||||
...systemTools,
|
||||
...customTools,
|
||||
]
|
||||
@@ -458,22 +436,18 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
|
||||
return prompt
|
||||
}
|
||||
|
||||
if (restricted) {
|
||||
prompt += `\n\n${buildRestrictedUserInstructions()}`
|
||||
if (hasRunner && !deniedTools.includes('bash')) {
|
||||
prompt +=
|
||||
'\n\n' +
|
||||
[
|
||||
'# Runner Environment & Workspace Rules',
|
||||
'Your shell environment (`bash` tool) runs on a dedicated Runner workspace:',
|
||||
'1. **Workspace Directory**: Read and write files inside the `.pi` folder in the current working directory.',
|
||||
'2. **Generated Media Staging**: Files generated by `generate_image` or `generate_video` are returned at virtual paths `.pi/generated/<id>/<filename>`. You can pass those paths directly to `create_file` or `create_version` without using `bash`. If you need to process a generated file inside `bash`, pass its path in the `inputFiles` array parameter of the `bash` tool so it is staged into the runner workspace before the command runs.',
|
||||
'3. **Temporary File Cleanup**: If you download workspace assets to `.pi` using the `download_asset` tool for local processing or script execution, you MUST delete the temporary downloaded files from `.pi` after finishing your task.',
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
// Sandbox environment restrictions
|
||||
prompt +=
|
||||
'\n\n' +
|
||||
[
|
||||
'# Sandbox Environment Restrictions',
|
||||
'Your shell environment (the `bash` tool) is highly sandboxed to protect the host system:',
|
||||
'1. **Filesystem Isolation**: You only have read and write permissions to the `.pi` folder in the project root directory.',
|
||||
'2. **Read/Write Restrictions**: All reading and writing to directories outside `.pi` (e.g. your home directory `~/`, `/tmp`, `/etc`, or the rest of the workspace) are strictly denied by the sandbox security policy.',
|
||||
"3. **Avoid System Temp Directory Writes**: You must strictly avoid any commands or shell constructs that attempt to write to the system temporary directory `/tmp` or `/var/tmp`. For example, do not use Bash here documents (`<<EOF` or `<<'EOF'`) in your commands, as the bash shell internally implements here documents by writing temporary files to `/tmp`. If you need to create a file or write content, write it directly using file creation tools or write to files located inside the `.pi` directory without utilizing here documents.",
|
||||
'4. **Temporary File Cleanup**: If you download workspace assets to `.pi` using the `download_asset` tool for local processing or script execution, you MUST delete the temporary downloaded files from `.pi` after finishing your task.',
|
||||
].join('\n')
|
||||
|
||||
if (teamSkills.length > 0) {
|
||||
prompt += formatSkillsForPrompt(teamSkills)
|
||||
}
|
||||
@@ -643,13 +617,13 @@ export function formatSkillsForPrompt(
|
||||
const lines = [
|
||||
'\n\nThe following skills provide specialized instructions for specific tasks.',
|
||||
"Use the read_skill tool to load a skill's file when the task matches its description.",
|
||||
'When a skill file references a relative path, resolve it against the skill directory (parent of SKILL.md / dirname of the path) and use that absolute path in tool commands.',
|
||||
'When a skill file references a relative path, resolve it against the skill directory (parent of SKILL.md / dirname of the path) and use that relative path in tool commands.',
|
||||
'',
|
||||
'<available_skills>',
|
||||
]
|
||||
|
||||
for (const skill of skills) {
|
||||
const filePath = path.join(process.cwd(), '.pi', 'skills', skill.id, 'SKILL.md')
|
||||
const filePath = path.posix.join('.pi', 'skills', skill.id, 'SKILL.md')
|
||||
lines.push(' <skill>')
|
||||
lines.push(` <id>${escapeXml(skill.id)}</id>`)
|
||||
lines.push(` <name>${escapeXml(skill.name)}</name>`)
|
||||
|
||||
@@ -1,12 +1,9 @@
|
||||
import { invokeTool } from '@shumai/core/src/agent/tool-test-utils'
|
||||
import { vi } from 'vitest'
|
||||
|
||||
vi.mock('fs')
|
||||
|
||||
import { mediaGenerationService } from '@shumai/core/src/media-generation/media-generation'
|
||||
import { s3Service } from '@shumai/core/src/s3/s3'
|
||||
import { EnabledMediaModel } from '@shumai/dtos'
|
||||
import * as fs from 'fs'
|
||||
import * as path from 'path'
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
import * as downloadAsset from './download-asset'
|
||||
import { createGenerateImageTool } from './generate-image'
|
||||
@@ -45,13 +42,9 @@ describe('generate_image tool', () => {
|
||||
google: { apiKey: 'mock-google-key' },
|
||||
}
|
||||
|
||||
const piDir = path.join(process.cwd(), '.pi')
|
||||
|
||||
beforeEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
vi.spyOn(fs, 'mkdirSync').mockImplementation(() => undefined)
|
||||
vi.spyOn(fs, 'writeFileSync').mockImplementation(() => {})
|
||||
vi.spyOn(s3Service, 'putObject').mockResolvedValue(undefined)
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
@@ -68,7 +61,7 @@ describe('generate_image tool', () => {
|
||||
expect(modelDesc).toContain('azure:dall-e-3')
|
||||
})
|
||||
|
||||
it('generates an image successfully and writes to .pi directory', async () => {
|
||||
it('generates an image successfully and saves to temporary storage with .pi/generated path', async () => {
|
||||
const tool = createGenerateImageTool(enabledModels, providerKeys, 'user-1')
|
||||
|
||||
const mockBuffer = Buffer.from('fake-image-png-content')
|
||||
@@ -76,7 +69,7 @@ describe('generate_image tool', () => {
|
||||
buffer: mockBuffer,
|
||||
mimeType: 'image/png',
|
||||
})
|
||||
const writeSpy = vi.spyOn(fs, 'writeFileSync').mockImplementation(() => {})
|
||||
const putSpy = vi.spyOn(s3Service, 'putObject').mockResolvedValue(undefined)
|
||||
|
||||
const result = await invokeTool(
|
||||
tool,
|
||||
@@ -105,14 +98,19 @@ describe('generate_image tool', () => {
|
||||
}),
|
||||
)
|
||||
|
||||
const expectedPath = path.join(piDir, 'test_city.png')
|
||||
expect(writeSpy).toHaveBeenCalledWith(expectedPath, mockBuffer)
|
||||
expect(putSpy).toHaveBeenCalledWith(
|
||||
expect.any(String),
|
||||
expect.stringMatching(/^temp\/generated\/.+\/test_city\.png$/),
|
||||
mockBuffer,
|
||||
mockBuffer.length,
|
||||
'image/png',
|
||||
)
|
||||
|
||||
const textContent = result.content[0]
|
||||
expect(textContent.type).toBe('text')
|
||||
if (textContent.type === 'text') {
|
||||
expect(textContent.text).toContain('Successfully generated image')
|
||||
expect(textContent.text).toContain('.pi/test_city.png')
|
||||
expect(textContent.text).toMatch(/\.pi\/generated\/.+\/test_city\.png/)
|
||||
}
|
||||
expect(result.details).toMatchObject({
|
||||
fileName: 'test_city.png',
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
import { type ToolRegistration as AgentTool } from '@earendil-works/pi-durable'
|
||||
import { storageBucket } from '@shumai/core/src/asset-read'
|
||||
import { mediaGenerationService } from '@shumai/core/src/media-generation/media-generation'
|
||||
import { s3Service } from '@shumai/core/src/s3/s3'
|
||||
import { sanitizeFilename } from '@shumai/core/src/utils/filename'
|
||||
import { EnabledMediaModel } from '@shumai/dtos'
|
||||
import * as fs from 'fs'
|
||||
import * as path from 'path'
|
||||
import { Type, type Static, type TSchema } from 'typebox'
|
||||
import { ulid } from 'ulid'
|
||||
import { resolveS3MediaBuffer } from './download-asset'
|
||||
@@ -138,27 +138,23 @@ export function createGenerateImageTool(
|
||||
seed: p.seed !== null && p.seed !== undefined ? p.seed : undefined,
|
||||
})
|
||||
|
||||
const piDir = path.join(process.cwd(), '.pi')
|
||||
if (!fs.existsSync(piDir)) {
|
||||
fs.mkdirSync(piDir, { recursive: true })
|
||||
}
|
||||
|
||||
const safeMime = mimeType || 'image/png'
|
||||
let ext = '.png'
|
||||
if (safeMime.includes('jpeg') || safeMime.includes('jpg')) ext = '.jpg'
|
||||
else if (safeMime.includes('webp')) ext = '.webp'
|
||||
|
||||
const genId = ulid()
|
||||
let filename: string
|
||||
if (p.outputFileName) {
|
||||
const safe = sanitizeFilename(p.outputFileName)
|
||||
filename = safe.includes('.') ? safe : `${safe}${ext}`
|
||||
} else {
|
||||
filename = `generated_image_${ulid()}${ext}`
|
||||
filename = `generated_image_${genId}${ext}`
|
||||
}
|
||||
|
||||
const targetPath = path.join(piDir, filename)
|
||||
const relativePath = path.join('.pi', filename)
|
||||
fs.writeFileSync(targetPath, buffer)
|
||||
const storageKey = `temp/generated/${genId}/${filename}`
|
||||
const relativePath = `.pi/generated/${genId}/${filename}`
|
||||
await s3Service.putObject(storageBucket(), storageKey, buffer, buffer.length, safeMime)
|
||||
|
||||
return {
|
||||
output: [
|
||||
@@ -170,7 +166,7 @@ export function createGenerateImageTool(
|
||||
details: {
|
||||
fileName: filename,
|
||||
filePath: relativePath,
|
||||
absolutePath: targetPath,
|
||||
storageKey,
|
||||
size: buffer.length,
|
||||
mimeType,
|
||||
provider: modelConfig.provider,
|
||||
|
||||
@@ -1,12 +1,9 @@
|
||||
import { invokeTool } from '@shumai/core/src/agent/tool-test-utils'
|
||||
import { vi } from 'vitest'
|
||||
|
||||
vi.mock('fs')
|
||||
|
||||
import { mediaGenerationService } from '@shumai/core/src/media-generation/media-generation'
|
||||
import { s3Service } from '@shumai/core/src/s3/s3'
|
||||
import { EnabledMediaModel } from '@shumai/dtos'
|
||||
import * as fs from 'fs'
|
||||
import * as path from 'path'
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
import * as downloadAsset from './download-asset'
|
||||
import { createGenerateVideoTool } from './generate-video'
|
||||
@@ -45,13 +42,9 @@ describe('generate_video tool', () => {
|
||||
google: { apiKey: 'mock-google-key' },
|
||||
}
|
||||
|
||||
const piDir = path.join(process.cwd(), '.pi')
|
||||
|
||||
beforeEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
vi.spyOn(fs, 'mkdirSync').mockImplementation(() => undefined)
|
||||
vi.spyOn(fs, 'writeFileSync').mockImplementation(() => {})
|
||||
vi.spyOn(s3Service, 'putObject').mockResolvedValue(undefined)
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
@@ -76,7 +69,7 @@ describe('generate_video tool', () => {
|
||||
buffer: mockBuffer,
|
||||
mimeType: 'video/mp4',
|
||||
})
|
||||
const writeSpy = vi.spyOn(fs, 'writeFileSync').mockImplementation(() => {})
|
||||
const putSpy = vi.spyOn(s3Service, 'putObject').mockResolvedValue(undefined)
|
||||
|
||||
const result = await invokeTool(
|
||||
tool,
|
||||
@@ -115,14 +108,19 @@ describe('generate_video tool', () => {
|
||||
}),
|
||||
)
|
||||
|
||||
const expectedPath = path.join(piDir, 'test_nature.mp4')
|
||||
expect(writeSpy).toHaveBeenCalledWith(expectedPath, mockBuffer)
|
||||
expect(putSpy).toHaveBeenCalledWith(
|
||||
expect.any(String),
|
||||
expect.stringMatching(/^temp\/generated\/.+\/test_nature\.mp4$/),
|
||||
mockBuffer,
|
||||
mockBuffer.length,
|
||||
'video/mp4',
|
||||
)
|
||||
|
||||
const textContent = result.content[0]
|
||||
expect(textContent.type).toBe('text')
|
||||
if (textContent.type === 'text') {
|
||||
expect(textContent.text).toContain('Successfully generated video')
|
||||
expect(textContent.text).toContain('.pi/test_nature.mp4')
|
||||
expect(textContent.text).toMatch(/\.pi\/generated\/.+\/test_nature\.mp4/)
|
||||
}
|
||||
expect(result.details).toMatchObject({
|
||||
fileName: 'test_nature.mp4',
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
import { type ToolRegistration as AgentTool } from '@earendil-works/pi-durable'
|
||||
import { storageBucket } from '@shumai/core/src/asset-read'
|
||||
import { mediaGenerationService } from '@shumai/core/src/media-generation/media-generation'
|
||||
import { s3Service } from '@shumai/core/src/s3/s3'
|
||||
import { sanitizeFilename } from '@shumai/core/src/utils/filename'
|
||||
import { EnabledMediaModel } from '@shumai/dtos'
|
||||
import * as fs from 'fs'
|
||||
import * as path from 'path'
|
||||
import { Type, type Static, type TSchema } from 'typebox'
|
||||
import { ulid } from 'ulid'
|
||||
import { resolveS3MediaBuffer } from './download-asset'
|
||||
@@ -262,22 +262,19 @@ export function createGenerateVideoTool(
|
||||
seed: p.seed !== null && p.seed !== undefined ? p.seed : undefined,
|
||||
})
|
||||
|
||||
const piDir = path.join(process.cwd(), '.pi')
|
||||
if (!fs.existsSync(piDir)) {
|
||||
fs.mkdirSync(piDir, { recursive: true })
|
||||
}
|
||||
|
||||
const genId = ulid()
|
||||
let filename: string
|
||||
if (p.outputFileName) {
|
||||
const safe = sanitizeFilename(p.outputFileName)
|
||||
filename = safe.endsWith('.mp4') ? safe : `${safe}.mp4`
|
||||
} else {
|
||||
filename = `generated_video_${ulid()}.mp4`
|
||||
filename = `generated_video_${genId}.mp4`
|
||||
}
|
||||
|
||||
const targetPath = path.join(piDir, filename)
|
||||
const relativePath = path.join('.pi', filename)
|
||||
fs.writeFileSync(targetPath, buffer)
|
||||
const safeMime = mimeType || 'video/mp4'
|
||||
const storageKey = `temp/generated/${genId}/${filename}`
|
||||
const relativePath = `.pi/generated/${genId}/${filename}`
|
||||
await s3Service.putObject(storageBucket(), storageKey, buffer, buffer.length, safeMime)
|
||||
|
||||
return {
|
||||
output: [
|
||||
@@ -289,7 +286,7 @@ export function createGenerateVideoTool(
|
||||
details: {
|
||||
fileName: filename,
|
||||
filePath: relativePath,
|
||||
absolutePath: targetPath,
|
||||
storageKey,
|
||||
size: buffer.length,
|
||||
mimeType,
|
||||
provider: modelConfig.provider,
|
||||
|
||||
@@ -2,21 +2,39 @@ import { invokeTool } from '@shumai/core/src/agent/tool-test-utils'
|
||||
import { vi } from 'vitest'
|
||||
|
||||
vi.mock('@shumai/core/src/s3/s3')
|
||||
vi.mock('fs')
|
||||
|
||||
import { s3Service } from '@shumai/core/src/s3/s3'
|
||||
import { prisma } from '@shumai/db'
|
||||
import { setupTestDbHooks } from '@shumai/db/test'
|
||||
import AdmZip from 'adm-zip'
|
||||
import * as fs from 'fs'
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
import { createReadSkillTool } from './read-skill'
|
||||
|
||||
function makeSkillZipBuffer(skillMdContent: string): Buffer {
|
||||
const zip = new AdmZip()
|
||||
zip.addFile('SKILL.md', Buffer.from(skillMdContent, 'utf-8'))
|
||||
return zip.toBuffer()
|
||||
}
|
||||
|
||||
describe('readSkillTool', () => {
|
||||
setupTestDbHooks()
|
||||
|
||||
beforeEach(() => {
|
||||
beforeEach(async () => {
|
||||
vi.clearAllMocks()
|
||||
await prisma.asset.create({
|
||||
data: {
|
||||
id: 'asset1',
|
||||
name: 'skill.zip',
|
||||
storageKey: { create: { key: 'skills/skill.zip' } },
|
||||
type: 'file',
|
||||
status: 'uploaded',
|
||||
},
|
||||
})
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking s3Service.getObject return value
|
||||
;(s3Service.getObject as any).mockResolvedValue({
|
||||
buffer: makeSkillZipBuffer('# Content'),
|
||||
contentType: 'application/zip',
|
||||
})
|
||||
})
|
||||
|
||||
it('should throw error if skill not found', async () => {
|
||||
@@ -26,7 +44,7 @@ describe('readSkillTool', () => {
|
||||
).rejects.toThrow('not found')
|
||||
})
|
||||
|
||||
it('should read skill from cache if hash matches', async () => {
|
||||
it('should read SKILL.md directly from the skill zip in storage', async () => {
|
||||
const team = await prisma.team.create({ data: { name: 'Test Team' } })
|
||||
const skill = await prisma.skill.create({
|
||||
data: {
|
||||
@@ -37,12 +55,10 @@ describe('readSkillTool', () => {
|
||||
},
|
||||
})
|
||||
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs calls
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return 'match-hash'
|
||||
if (path.toString().endsWith('SKILL.md')) return '# Test Skill Content'
|
||||
return ''
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking s3Service return
|
||||
;(s3Service.getObject as any).mockResolvedValue({
|
||||
buffer: makeSkillZipBuffer('# Test Skill Content'),
|
||||
contentType: 'application/zip',
|
||||
})
|
||||
|
||||
const readSkillTool = createReadSkillTool(undefined, () => {})
|
||||
@@ -50,10 +66,10 @@ describe('readSkillTool', () => {
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
expect((result.content[0] as any).text).toBe('# Test Skill Content')
|
||||
expect(s3Service.getObject).not.toHaveBeenCalled()
|
||||
expect(s3Service.getObject).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should download and extract skill if cache missing or hash mismatch', async () => {
|
||||
it('should read nested SKILL.md inside zip archive', async () => {
|
||||
const team = await prisma.team.create({ data: { name: 'Test Team' } })
|
||||
const skill = await prisma.skill.create({
|
||||
data: {
|
||||
@@ -64,31 +80,8 @@ describe('readSkillTool', () => {
|
||||
},
|
||||
})
|
||||
|
||||
await prisma.asset.create({
|
||||
data: {
|
||||
id: 'asset1',
|
||||
name: 'skill.zip',
|
||||
storageKey: { create: { key: 'skills/skill.zip' } },
|
||||
type: 'file',
|
||||
status: 'uploaded',
|
||||
},
|
||||
})
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs calls
|
||||
vi.spyOn(fs, 'existsSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return false
|
||||
if (path.toString().endsWith('SKILL.md')) return true
|
||||
return true
|
||||
})
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs calls
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('SKILL.md')) return '# Extracted Content'
|
||||
return ''
|
||||
})
|
||||
|
||||
const zip = new AdmZip()
|
||||
zip.addFile('SKILL.md', Buffer.from('# Extracted Content'))
|
||||
zip.addFile('nested/SKILL.md', Buffer.from('# Extracted Content'))
|
||||
const zipBuffer = zip.toBuffer()
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking s3Service return
|
||||
@@ -103,7 +96,6 @@ describe('readSkillTool', () => {
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
expect((result.content[0] as any).text).toBe('# Extracted Content')
|
||||
expect(s3Service.getObject).toHaveBeenCalled()
|
||||
expect(fs.writeFileSync).toHaveBeenCalledWith(expect.stringContaining('.hash'), 'new-hash')
|
||||
})
|
||||
|
||||
describe('environment variables', () => {
|
||||
@@ -133,14 +125,6 @@ describe('readSkillTool', () => {
|
||||
|
||||
delete process.env.CONFIGURED_VAR
|
||||
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs readFileSync which has complex overloaded signatures
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return 'hash1'
|
||||
if (path.toString().endsWith('SKILL.md')) return '# Content'
|
||||
return ''
|
||||
})
|
||||
|
||||
let capturedEnvs: Record<string, string> = {}
|
||||
const readSkillTool = createReadSkillTool(undefined, (envs) => {
|
||||
capturedEnvs = { ...capturedEnvs, ...envs }
|
||||
@@ -168,14 +152,6 @@ describe('readSkillTool', () => {
|
||||
|
||||
process.env.OVERRIDDEN_VAR = 'host-val'
|
||||
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs readFileSync which has complex overloaded signatures
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return 'hash2'
|
||||
if (path.toString().endsWith('SKILL.md')) return '# Content'
|
||||
return ''
|
||||
})
|
||||
|
||||
let capturedEnvs: Record<string, string> = {}
|
||||
const readSkillTool = createReadSkillTool(undefined, (envs) => {
|
||||
capturedEnvs = { ...capturedEnvs, ...envs }
|
||||
@@ -203,14 +179,6 @@ describe('readSkillTool', () => {
|
||||
|
||||
process.env.FALLBACK_VAR = 'host-val'
|
||||
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs readFileSync which has complex overloaded signatures
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return 'hash3'
|
||||
if (path.toString().endsWith('SKILL.md')) return '# Content'
|
||||
return ''
|
||||
})
|
||||
|
||||
let capturedEnvs: Record<string, string> = {}
|
||||
const readSkillTool = createReadSkillTool(undefined, (envs) => {
|
||||
capturedEnvs = { ...capturedEnvs, ...envs }
|
||||
@@ -238,14 +206,6 @@ describe('readSkillTool', () => {
|
||||
|
||||
process.env.FALLBACK_VAR_UNDEF = 'host-val2'
|
||||
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs readFileSync which has complex overloaded signatures
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return 'hash4'
|
||||
if (path.toString().endsWith('SKILL.md')) return '# Content'
|
||||
return ''
|
||||
})
|
||||
|
||||
let capturedEnvs: Record<string, string> = {}
|
||||
const readSkillTool = createReadSkillTool(undefined, (envs) => {
|
||||
capturedEnvs = { ...capturedEnvs, ...envs }
|
||||
@@ -285,12 +245,10 @@ describe('readSkillTool', () => {
|
||||
},
|
||||
})
|
||||
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return 'hash-owner'
|
||||
if (path.toString().endsWith('SKILL.md')) return '# Owner Skill'
|
||||
return ''
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking s3Service return
|
||||
;(s3Service.getObject as any).mockResolvedValue({
|
||||
buffer: makeSkillZipBuffer('# Owner Skill'),
|
||||
contentType: 'application/zip',
|
||||
})
|
||||
|
||||
const ownerTool = createReadSkillTool(ownerUser.id, () => {})
|
||||
@@ -356,12 +314,10 @@ describe('readSkillTool', () => {
|
||||
data: { projectId: project.id, teamMemberId: projEditorTm.id, role: 'editor' },
|
||||
})
|
||||
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs calls
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return 'proj-editor-hash'
|
||||
if (path.toString().endsWith('SKILL.md')) return '# Editor Project Skill Content'
|
||||
return ''
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking s3Service return
|
||||
;(s3Service.getObject as any).mockResolvedValue({
|
||||
buffer: makeSkillZipBuffer('# Editor Project Skill Content'),
|
||||
contentType: 'application/zip',
|
||||
})
|
||||
|
||||
const projEditorTool = createReadSkillTool(
|
||||
@@ -395,19 +351,16 @@ describe('readSkillTool', () => {
|
||||
},
|
||||
})
|
||||
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs readFileSync which has complex overloaded signatures
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return 'cb-hash'
|
||||
if (path.toString().endsWith('SKILL.md')) return '# Callback Skill'
|
||||
return ''
|
||||
})
|
||||
|
||||
const onSkillLoaded = vi.fn()
|
||||
const readSkillTool = createReadSkillTool(undefined, () => {}, onSkillLoaded)
|
||||
await invokeTool(readSkillTool, '1', { skillId: skill.id }, undefined, undefined)
|
||||
|
||||
expect(onSkillLoaded).toHaveBeenCalledTimes(1)
|
||||
expect(onSkillLoaded).toHaveBeenCalledWith({
|
||||
id: skill.id,
|
||||
assetId: 'asset1',
|
||||
hash: 'cb-hash',
|
||||
})
|
||||
})
|
||||
|
||||
it('should not invoke onSkillLoaded when loading is denied', async () => {
|
||||
@@ -469,12 +422,10 @@ describe('readSkillTool', () => {
|
||||
},
|
||||
})
|
||||
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs readFileSync which has complex overloaded signatures
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return 'enabled-agent-hash'
|
||||
if (path.toString().endsWith('SKILL.md')) return '# Enabled For Agent Skill'
|
||||
return ''
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking s3Service return
|
||||
;(s3Service.getObject as any).mockResolvedValue({
|
||||
buffer: makeSkillZipBuffer('# Enabled For Agent Skill'),
|
||||
contentType: 'application/zip',
|
||||
})
|
||||
|
||||
const readSkillTool = createReadSkillTool(undefined, () => {}, undefined, [skill.id])
|
||||
@@ -500,12 +451,10 @@ describe('readSkillTool', () => {
|
||||
},
|
||||
})
|
||||
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking node fs readFileSync which has complex overloaded signatures
|
||||
vi.spyOn(fs, 'readFileSync').mockImplementation((path: any) => {
|
||||
if (path.toString().endsWith('.hash')) return 'no-filter-hash'
|
||||
if (path.toString().endsWith('SKILL.md')) return '# No Filter Skill'
|
||||
return ''
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mocking s3Service return
|
||||
;(s3Service.getObject as any).mockResolvedValue({
|
||||
buffer: makeSkillZipBuffer('# No Filter Skill'),
|
||||
contentType: 'application/zip',
|
||||
})
|
||||
|
||||
const readSkillTool = createReadSkillTool(undefined, () => {})
|
||||
|
||||
@@ -17,7 +17,7 @@ const ROLE_HIERARCHY: Record<string, number> = {
|
||||
export const createReadSkillTool = (
|
||||
userId: string | undefined,
|
||||
onEnvsAdded: (envs: Record<string, string>) => void,
|
||||
onSkillLoaded?: () => Promise<void> | void,
|
||||
onSkillLoaded?: (skill: { id: string; assetId: string; hash: string }) => Promise<void> | void,
|
||||
enabledSkillIds?: string[],
|
||||
projectId?: string,
|
||||
): AgentTool<typeof readSkillSchema, { skillId: string }> => ({
|
||||
@@ -67,8 +67,8 @@ export const createReadSkillTool = (
|
||||
// Retrieve skill content using agentService
|
||||
const skillMdContent = await agentService.getSkillContent(params.skillId)
|
||||
|
||||
// Notify that a skill was successfully loaded (e.g. to enable restricted tools)
|
||||
await onSkillLoaded?.()
|
||||
// Notify that a skill was successfully loaded (e.g. to stage the skill on the runner)
|
||||
await onSkillLoaded?.({ id: skill.id, assetId: skill.assetId, hash: skill.hash })
|
||||
|
||||
return {
|
||||
output: [{ type: 'text', text: skillMdContent }],
|
||||
|
||||
@@ -0,0 +1,311 @@
|
||||
import { invokeTool } from '@shumai/core/src/agent/tool-test-utils'
|
||||
import { runnerService } from '@shumai/core/src/runner'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { createRunnerBashTool } from './runner-bash'
|
||||
|
||||
vi.mock('@shumai/core/src/runner', () => ({
|
||||
runnerService: {
|
||||
requireOnlineRunner: vi.fn(),
|
||||
isRunnerConnected: vi.fn(),
|
||||
ensureSkillOnRunner: vi.fn(),
|
||||
ensureGeneratedFileOnRunner: vi.fn(),
|
||||
exec: vi.fn(),
|
||||
},
|
||||
}))
|
||||
|
||||
describe('createRunnerBashTool', () => {
|
||||
const mockTeamId = 'team-1'
|
||||
const mockRunner = {
|
||||
id: 'runner-1',
|
||||
teamId: mockTeamId,
|
||||
name: 'Local Mac Mini',
|
||||
keyPrefix: 'shm_rn_12345678',
|
||||
isDefault: true,
|
||||
status: 'connected' as const,
|
||||
createdAt: '2026-11-14T00:00:00.000Z',
|
||||
updatedAt: '2026-11-14T00:00:00.000Z',
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
vi.mocked(runnerService.requireOnlineRunner).mockResolvedValue(mockRunner)
|
||||
vi.mocked(runnerService.isRunnerConnected).mockReturnValue(true)
|
||||
vi.mocked(runnerService.ensureSkillOnRunner).mockResolvedValue('.pi/skills/skill-1')
|
||||
vi.mocked(runnerService.ensureGeneratedFileOnRunner).mockResolvedValue(
|
||||
'.pi/generated/gen1/input.png',
|
||||
)
|
||||
})
|
||||
|
||||
const createMockSignal = () => new AbortController().signal
|
||||
|
||||
const createMockOnUpdate = () => vi.fn()
|
||||
|
||||
it.each(['C:\\Windows\\System32\\cmd.exe', 'C:\\Program Files\\Git\\bin\\bash.exe'])(
|
||||
'tells the agent to use the runner OS and shell syntax for %s',
|
||||
(shell) => {
|
||||
const tool = createRunnerBashTool({}, { environment: { platform: 'win32', shell } })
|
||||
expect(tool.description).toContain('Runner OS: win32')
|
||||
expect(tool.description).toContain(shell)
|
||||
expect(tool.description).toContain('Use syntax and programs compatible with this environment')
|
||||
},
|
||||
)
|
||||
|
||||
it('should execute a command on the runner with injected env vars', async () => {
|
||||
const tool = createRunnerBashTool(
|
||||
{ TEST_ENV: 'test-value' },
|
||||
{
|
||||
teamId: mockTeamId,
|
||||
agentRunnerId: null,
|
||||
},
|
||||
)
|
||||
|
||||
vi.mocked(runnerService.exec).mockImplementation(async (_runnerId, opts) => {
|
||||
opts.onOutput?.('hello output', 'stdout')
|
||||
return {
|
||||
executionId: 'exec-1',
|
||||
output: 'hello output',
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
}
|
||||
})
|
||||
|
||||
const onUpdate = createMockOnUpdate()
|
||||
const signal = createMockSignal()
|
||||
|
||||
const result = await invokeTool(tool, '1', { command: 'echo hello' }, signal, onUpdate)
|
||||
|
||||
expect(runnerService.requireOnlineRunner).toHaveBeenCalledWith(mockTeamId, null)
|
||||
expect(runnerService.exec).toHaveBeenCalledWith(
|
||||
'runner-1',
|
||||
expect.objectContaining({
|
||||
command: 'echo hello',
|
||||
env: { TEST_ENV: 'test-value' },
|
||||
collectOutput: false,
|
||||
}),
|
||||
)
|
||||
expect(result.content).toContainEqual(
|
||||
expect.objectContaining({ text: expect.stringContaining('hello output') }),
|
||||
)
|
||||
expect(onUpdate).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should stage loaded skills and inputFiles before executing', async () => {
|
||||
const loadedSkills = [{ id: 'skill-1', assetId: 'asset-1', hash: 'hash-1' }]
|
||||
const tool = createRunnerBashTool(
|
||||
{},
|
||||
{
|
||||
teamId: mockTeamId,
|
||||
agentRunnerId: 'runner-pinned',
|
||||
getLoadedSkills: () => loadedSkills,
|
||||
},
|
||||
)
|
||||
|
||||
vi.mocked(runnerService.exec).mockResolvedValue({
|
||||
executionId: 'exec-1',
|
||||
output: '',
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
})
|
||||
|
||||
await invokeTool(
|
||||
tool,
|
||||
'1',
|
||||
{
|
||||
command: 'ffmpeg -i .pi/generated/gen1/input.png out.mp4',
|
||||
inputFiles: ['.pi/generated/gen1/input.png'],
|
||||
},
|
||||
createMockSignal(),
|
||||
)
|
||||
|
||||
expect(runnerService.requireOnlineRunner).toHaveBeenCalledWith(mockTeamId, 'runner-pinned')
|
||||
expect(runnerService.ensureSkillOnRunner).toHaveBeenCalledWith('runner-1', loadedSkills[0])
|
||||
expect(runnerService.ensureGeneratedFileOnRunner).toHaveBeenCalledWith(
|
||||
'runner-1',
|
||||
'.pi/generated/gen1/input.png',
|
||||
)
|
||||
})
|
||||
|
||||
it('allows editor commands without a source classification', async () => {
|
||||
const tool = createRunnerBashTool(
|
||||
{},
|
||||
{
|
||||
teamId: mockTeamId,
|
||||
role: 'editor',
|
||||
},
|
||||
)
|
||||
|
||||
vi.mocked(runnerService.exec).mockImplementation(async (_runnerId, opts) => {
|
||||
opts.onOutput?.('skill output', 'stdout')
|
||||
return {
|
||||
executionId: 'exec-1',
|
||||
output: 'skill output',
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
}
|
||||
})
|
||||
|
||||
const signal = createMockSignal()
|
||||
const result = await invokeTool(tool, '1', { command: 'echo skill' }, signal)
|
||||
expect(result.content).toContainEqual(
|
||||
expect.objectContaining({ text: expect.stringContaining('skill output') }),
|
||||
)
|
||||
})
|
||||
|
||||
it('should handle runner execution errors', async () => {
|
||||
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
|
||||
|
||||
vi.mocked(runnerService.exec).mockRejectedValue(new Error('Runner disconnected'))
|
||||
|
||||
const signal = createMockSignal()
|
||||
await expect(
|
||||
invokeTool(tool, '1', { command: 'fail' }, signal, createMockOnUpdate()),
|
||||
).rejects.toThrow('Runner disconnected')
|
||||
})
|
||||
|
||||
it('should handle non-zero exit codes', async () => {
|
||||
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
|
||||
|
||||
vi.mocked(runnerService.exec).mockResolvedValue({
|
||||
executionId: 'exec-1',
|
||||
output: '',
|
||||
exitCode: 1,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
})
|
||||
|
||||
const signal = createMockSignal()
|
||||
await expect(
|
||||
invokeTool(tool, '1', { command: 'exit 1' }, signal, createMockOnUpdate()),
|
||||
).rejects.toThrow(/exited with code 1/)
|
||||
})
|
||||
|
||||
it('reports signal termination as a command failure', async () => {
|
||||
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
|
||||
vi.mocked(runnerService.exec).mockResolvedValue({
|
||||
executionId: 'signaled',
|
||||
output: '',
|
||||
exitCode: null,
|
||||
signal: 'SIGTERM',
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
})
|
||||
await expect(invokeTool(tool, '1', { command: 'command' }, createMockSignal())).rejects.toThrow(
|
||||
'terminated by SIGTERM',
|
||||
)
|
||||
})
|
||||
|
||||
it('should handle timeouts', async () => {
|
||||
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
|
||||
|
||||
vi.mocked(runnerService.exec).mockImplementation(async (_runnerId, opts) => {
|
||||
opts.onOutput?.('output before timeout\n', 'stdout')
|
||||
return {
|
||||
executionId: 'exec-1',
|
||||
output: 'output before timeout\n',
|
||||
exitCode: null,
|
||||
signal: 'SIGKILL',
|
||||
timedOut: true,
|
||||
aborted: false,
|
||||
}
|
||||
})
|
||||
|
||||
const signal = createMockSignal()
|
||||
const executePromise = invokeTool(
|
||||
tool,
|
||||
'1',
|
||||
{ command: 'sleep 10', timeout: 2 },
|
||||
signal,
|
||||
createMockOnUpdate(),
|
||||
)
|
||||
|
||||
await expect(executePromise).rejects.toThrow(/output before timeout/)
|
||||
await expect(executePromise).rejects.toThrow(/timed out after 2 seconds/)
|
||||
})
|
||||
|
||||
it('should handle abort signals', async () => {
|
||||
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
|
||||
|
||||
vi.mocked(runnerService.exec).mockResolvedValue({
|
||||
executionId: 'exec-1',
|
||||
output: '',
|
||||
exitCode: null,
|
||||
signal: 'SIGKILL',
|
||||
timedOut: false,
|
||||
aborted: true,
|
||||
})
|
||||
|
||||
const signal = createMockSignal()
|
||||
await expect(
|
||||
invokeTool(tool, '1', { command: 'long-run' }, signal, createMockOnUpdate()),
|
||||
).rejects.toThrow(/aborted/)
|
||||
})
|
||||
|
||||
it('should truncate output exceeding max lines (2000 lines) and report runner log file path', async () => {
|
||||
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
|
||||
|
||||
vi.mocked(runnerService.exec).mockImplementation(async (_runnerId, opts) => {
|
||||
const lines: string[] = []
|
||||
for (let i = 1; i <= 3000; i++) {
|
||||
lines.push(`line-${i}`)
|
||||
}
|
||||
const chunk = lines.join('\n') + '\n'
|
||||
opts.onOutput?.(chunk, 'stdout')
|
||||
return {
|
||||
executionId: 'exec1',
|
||||
output: chunk,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
fullOutputPath: '.pi/logs/bash-exec1.log',
|
||||
}
|
||||
})
|
||||
|
||||
const signal = createMockSignal()
|
||||
const result = await invokeTool(tool, '1', { command: 'generate lines' }, signal)
|
||||
const text = result.content.map((block) => (block.type === 'text' ? block.text : '')).join('')
|
||||
expect(text).toContain('line-3000')
|
||||
expect(text).toContain(
|
||||
'[Showing lines 1001-3000 of 3000. Full output: .pi/logs/bash-exec1.log]',
|
||||
)
|
||||
expect(result.details?.truncation).toMatchObject({
|
||||
truncated: true,
|
||||
truncatedBy: 'lines',
|
||||
totalLines: 3000,
|
||||
outputLines: 2000,
|
||||
})
|
||||
expect(result.details?.fullOutputPath).toBe('.pi/logs/bash-exec1.log')
|
||||
})
|
||||
|
||||
it('should truncate output exceeding max bytes (50KB) and report partial line size', async () => {
|
||||
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
|
||||
|
||||
vi.mocked(runnerService.exec).mockImplementation(async (_runnerId, opts) => {
|
||||
const longLine = 'a'.repeat(60000)
|
||||
opts.onOutput?.(longLine, 'stdout')
|
||||
return {
|
||||
executionId: 'exec2',
|
||||
output: longLine,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
fullOutputPath: '.pi/logs/bash-exec2.log',
|
||||
}
|
||||
})
|
||||
|
||||
const signal = createMockSignal()
|
||||
const result = await invokeTool(tool, '1', { command: 'long single line' }, signal)
|
||||
const text = result.content.map((block) => (block.type === 'text' ? block.text : '')).join('')
|
||||
expect(text).toMatch(
|
||||
/Showing last 50\.0KB of line 1 \(line is 58\.6KB\)\. Full output: \.pi\/logs\/bash-exec2\.log/,
|
||||
)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,243 @@
|
||||
import {
|
||||
type ToolRegistration as AgentTool,
|
||||
type ToolExecutionResult as AgentToolResult,
|
||||
} from '@earendil-works/pi-durable'
|
||||
import { logger } from '@shumai/core/src/logger'
|
||||
import { quotaService } from '@shumai/core/src/quota/quota-service'
|
||||
import { runnerService } from '@shumai/core/src/runner'
|
||||
import type { TeamMemberRole } from '@shumai/db'
|
||||
import type { RunnerEnvironment } from '@shumai/dtos'
|
||||
import { Type } from 'typebox'
|
||||
import { OutputAccumulator } from '../utils/output-accumulator'
|
||||
import {
|
||||
DEFAULT_MAX_BYTES,
|
||||
DEFAULT_MAX_LINES,
|
||||
formatSize,
|
||||
type TruncationResult,
|
||||
} from '../utils/truncate'
|
||||
|
||||
const BashParameters = Type.Object({
|
||||
command: Type.String({ description: "The command to execute using the runner's shell." }),
|
||||
timeout: Type.Optional(Type.Number({ description: 'Maximum execution time in seconds.' })),
|
||||
inputFiles: Type.Optional(
|
||||
Type.Union([
|
||||
Type.Array(Type.String(), {
|
||||
description:
|
||||
'Optional list of generated file paths (e.g. ".pi/generated/<id>/<filename>") to stage into the runner workspace before executing the command.',
|
||||
}),
|
||||
Type.Null(),
|
||||
]),
|
||||
),
|
||||
})
|
||||
|
||||
export type BashDetails = {
|
||||
exitCode: number | null
|
||||
truncation?: TruncationResult
|
||||
fullOutputPath?: string
|
||||
}
|
||||
|
||||
export interface LoadedSkillRef {
|
||||
id: string
|
||||
assetId: string
|
||||
hash: string
|
||||
}
|
||||
|
||||
export interface RunnerBashOptions {
|
||||
teamId?: string
|
||||
agentRunnerId?: string | null
|
||||
runnerId?: string
|
||||
userId?: string
|
||||
role?: TeamMemberRole | null
|
||||
environment?: RunnerEnvironment
|
||||
getLoadedSkills?: () => Iterable<LoadedSkillRef>
|
||||
}
|
||||
|
||||
export const createRunnerBashTool = (
|
||||
skillEnvs: Record<string, string> = {},
|
||||
options?: RunnerBashOptions,
|
||||
): AgentTool<typeof BashParameters, BashDetails> => {
|
||||
return {
|
||||
name: 'bash',
|
||||
description: `Execute a shell command on the configured runner. ${options?.environment ? `Runner OS: ${options.environment.platform}. Shell: ${options.environment.shell}. Use syntax and programs compatible with this environment.` : 'The runner reports its OS and shell when connected.'} Returns stdout and stderr. Output is truncated to last ${DEFAULT_MAX_LINES} lines or ${DEFAULT_MAX_BYTES / 1024}KB (whichever is hit first). If truncated, full output is saved to a log file in .pi/logs/ on the runner. Optionally provide a timeout in seconds, or inputFiles to stage generated files (.pi/generated/...) into the runner workspace before running.`,
|
||||
parameters: BashParameters,
|
||||
outputLimits: { maxBytes: DEFAULT_MAX_BYTES, maxLines: DEFAULT_MAX_LINES, retain: 'tail' },
|
||||
execute: async (
|
||||
{ command, timeout, inputFiles },
|
||||
api,
|
||||
context,
|
||||
): Promise<AgentToolResult<BashDetails>> => {
|
||||
const signal = context.abortSignal
|
||||
if (options?.teamId) {
|
||||
await quotaService.checkQuota(
|
||||
{
|
||||
teamId: options.teamId,
|
||||
userId: options.userId,
|
||||
role: options.role,
|
||||
resource: 'agent_bash_call_count',
|
||||
resourceData: { command },
|
||||
},
|
||||
1,
|
||||
)
|
||||
}
|
||||
|
||||
let targetRunnerId: string
|
||||
if (options?.runnerId) {
|
||||
if (!runnerService.isRunnerConnected(options.runnerId)) {
|
||||
throw new Error(
|
||||
'Runner is currently offline. Please start your runner with "shumai-cli runner start" and try again.',
|
||||
)
|
||||
}
|
||||
targetRunnerId = options.runnerId
|
||||
} else if (options?.teamId) {
|
||||
const runner = await runnerService.requireOnlineRunner(
|
||||
options.teamId,
|
||||
options.agentRunnerId,
|
||||
)
|
||||
targetRunnerId = runner.id
|
||||
} else {
|
||||
throw new Error('No runner configured for this session.')
|
||||
}
|
||||
|
||||
// Ensure any loaded skills in this session are extracted on the runner before executing bash
|
||||
if (options?.getLoadedSkills) {
|
||||
for (const skill of options.getLoadedSkills()) {
|
||||
await runnerService.ensureSkillOnRunner(targetRunnerId, skill)
|
||||
}
|
||||
}
|
||||
|
||||
// Stage any explicitly requested generated files (.pi/generated/<id>/<file>) onto the runner
|
||||
if (inputFiles && Array.isArray(inputFiles)) {
|
||||
for (const filePath of inputFiles) {
|
||||
await runnerService.ensureGeneratedFileOnRunner(targetRunnerId, filePath)
|
||||
}
|
||||
}
|
||||
|
||||
const output = new OutputAccumulator({
|
||||
tempFilePrefix: 'shumai-bash',
|
||||
disableTempFile: true,
|
||||
})
|
||||
let acceptingOutput = true
|
||||
await api.details({ exitCode: null }, context)
|
||||
|
||||
const handleData = (data: Buffer) => {
|
||||
if (!acceptingOutput) return
|
||||
output.append(data)
|
||||
api.output(data)
|
||||
}
|
||||
|
||||
const finishOutput = async () => {
|
||||
acceptingOutput = false
|
||||
output.finish()
|
||||
const snapshot = output.snapshot({ persistIfTruncated: false })
|
||||
await output.closeTempFile()
|
||||
return snapshot
|
||||
}
|
||||
|
||||
const formatOutput = (
|
||||
snapshot: Awaited<ReturnType<typeof finishOutput>>,
|
||||
runnerLogPath: string,
|
||||
emptyText = '(no output)',
|
||||
) => {
|
||||
const truncation = snapshot.truncation
|
||||
let text = snapshot.content || emptyText
|
||||
let details: BashDetails | undefined
|
||||
if (truncation.truncated) {
|
||||
details = { exitCode: null, truncation, fullOutputPath: runnerLogPath }
|
||||
const startLine = truncation.totalLines - truncation.outputLines + 1
|
||||
const endLine = truncation.totalLines
|
||||
if (truncation.lastLinePartial) {
|
||||
const lastLineSize = formatSize(output.getLastLineBytes())
|
||||
text += `\n\n[Showing last ${formatSize(truncation.outputBytes)} of line ${endLine} (line is ${lastLineSize}). Full output: ${runnerLogPath}]`
|
||||
} else if (truncation.truncatedBy === 'lines') {
|
||||
text += `\n\n[Showing lines ${startLine}-${endLine} of ${truncation.totalLines}. Full output: ${runnerLogPath}]`
|
||||
} else {
|
||||
text += `\n\n[Showing lines ${startLine}-${endLine} of ${truncation.totalLines} (${formatSize(DEFAULT_MAX_BYTES)} limit). Full output: ${runnerLogPath}]`
|
||||
}
|
||||
}
|
||||
return { text, details }
|
||||
}
|
||||
|
||||
const appendStatus = (text: string, status: string) => `${text ? `${text}\n\n` : ''}${status}`
|
||||
|
||||
const execResult = await runnerService.exec(targetRunnerId, {
|
||||
command,
|
||||
timeoutSec: timeout,
|
||||
env: skillEnvs,
|
||||
signal,
|
||||
collectOutput: false,
|
||||
onOutput: (chunk) => handleData(Buffer.from(chunk, 'utf-8')),
|
||||
})
|
||||
|
||||
const snapshot = await finishOutput()
|
||||
const runnerLogPath =
|
||||
execResult.fullOutputPath || `.pi/logs/bash-${execResult.executionId}.log`
|
||||
const { text: outputText, details: truncationDetails } = formatOutput(snapshot, runnerLogPath)
|
||||
|
||||
if (options?.teamId) {
|
||||
try {
|
||||
await quotaService.consumeQuota(
|
||||
{
|
||||
teamId: options.teamId,
|
||||
userId: options.userId,
|
||||
role: options.role,
|
||||
resource: 'agent_bash_call_count',
|
||||
resourceData: { command },
|
||||
},
|
||||
1,
|
||||
)
|
||||
} catch (err) {
|
||||
logger.error({ err }, 'Failed to record quota usage for bash command')
|
||||
}
|
||||
}
|
||||
|
||||
if (signal?.aborted || execResult.aborted) {
|
||||
logger.error({ command, outputText }, 'Bash command aborted')
|
||||
throw new Error(appendStatus(outputText, 'Command aborted'))
|
||||
}
|
||||
|
||||
if (execResult.timedOut) {
|
||||
logger.error({ timeout, command, outputText }, 'Bash command timed out')
|
||||
throw new Error(appendStatus(outputText, `Bash command timed out after ${timeout} seconds`))
|
||||
}
|
||||
|
||||
if (execResult.error) {
|
||||
logger.error(
|
||||
{ error: execResult.error, command, outputText },
|
||||
'Bash command encountered runner error',
|
||||
)
|
||||
throw new Error(appendStatus(outputText, execResult.error))
|
||||
}
|
||||
|
||||
if (execResult.signal) {
|
||||
logger.error(
|
||||
{ signal: execResult.signal, command, outputText },
|
||||
'Bash command was terminated',
|
||||
)
|
||||
throw new Error(appendStatus(outputText, 'Bash command terminated by ' + execResult.signal))
|
||||
}
|
||||
|
||||
if (execResult.exitCode !== 0 && execResult.exitCode !== null) {
|
||||
logger.error(
|
||||
{ exitCode: execResult.exitCode, command, outputText },
|
||||
'Bash command exited with failure code',
|
||||
)
|
||||
throw new Error(
|
||||
appendStatus(outputText, `Bash command exited with code ${execResult.exitCode}`),
|
||||
)
|
||||
}
|
||||
|
||||
return {
|
||||
output: [
|
||||
{
|
||||
type: 'text',
|
||||
text: outputText,
|
||||
},
|
||||
],
|
||||
details: {
|
||||
exitCode: execResult.exitCode,
|
||||
...(truncationDetails ?? {}),
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -1,376 +0,0 @@
|
||||
import { SandboxManager } from '@anthropic-ai/sandbox-runtime'
|
||||
import { invokeTool } from '@shumai/core/src/agent/tool-test-utils'
|
||||
import { spawn } from 'node:child_process'
|
||||
import { EventEmitter } from 'node:events'
|
||||
import { readFile } from 'node:fs/promises'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { createSandboxedBashTool } from './sandboxed-bash'
|
||||
|
||||
vi.mock('@anthropic-ai/sandbox-runtime')
|
||||
vi.mock('node:child_process')
|
||||
|
||||
describe('createSandboxedBashTool', () => {
|
||||
const mockCwd = '/mock/cwd'
|
||||
const mockSessionManager = {
|
||||
getSkillEnvs: vi.fn(),
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
mockSessionManager.getSkillEnvs.mockReturnValue({ TEST_ENV: 'test-value' })
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- SandboxManager.wrapWithSandbox is a mocked async function from a 3rd party library
|
||||
;(SandboxManager.wrapWithSandbox as any).mockImplementation(
|
||||
async (cmd: string) => `sandboxed-${cmd}`,
|
||||
)
|
||||
})
|
||||
|
||||
const createMockSignal = () => {
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- signal needs to be partially compatible with AbortSignal but we need to emit events for testing
|
||||
const signal = new EventEmitter() as any
|
||||
signal.addEventListener = vi.fn((event: string, cb: (ev: Event) => unknown) => {
|
||||
signal.on(event, cb)
|
||||
})
|
||||
signal.removeEventListener = vi.fn((event: string, cb: (ev: Event) => unknown) => {
|
||||
signal.off(event, cb)
|
||||
})
|
||||
signal.aborted = false
|
||||
return signal
|
||||
}
|
||||
|
||||
const createMockOnUpdate = () => vi.fn()
|
||||
|
||||
it('should execute a command in the sandbox with injected env vars', async () => {
|
||||
const tool = createSandboxedBashTool(mockCwd, mockSessionManager.getSkillEnvs())
|
||||
|
||||
const mockStdout = new EventEmitter()
|
||||
const mockStderr = new EventEmitter()
|
||||
const mockChild = Object.assign(new EventEmitter(), {
|
||||
stdout: mockStdout,
|
||||
stderr: mockStderr,
|
||||
pid: 123,
|
||||
kill: vi.fn(),
|
||||
})
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- spawn is mocked to return a partial ChildProcess
|
||||
;(spawn as any).mockReturnValue(mockChild)
|
||||
|
||||
const onUpdate = createMockOnUpdate()
|
||||
const signal = createMockSignal()
|
||||
|
||||
// pi-coding-agent AgentTool.execute signature: (toolCallId, params, signal?, onUpdate?)
|
||||
const executePromise = invokeTool(
|
||||
tool,
|
||||
'1',
|
||||
{ command: 'echo hello', source: 'user' },
|
||||
signal,
|
||||
onUpdate,
|
||||
)
|
||||
|
||||
await vi.waitFor(() => expect(spawn).toHaveBeenCalled())
|
||||
|
||||
expect(SandboxManager.wrapWithSandbox).toHaveBeenCalledWith(
|
||||
'echo hello',
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
{ commandId: '1' },
|
||||
)
|
||||
expect(spawn).toHaveBeenCalledWith(
|
||||
'bash',
|
||||
['-c', 'sandboxed-echo hello'],
|
||||
expect.objectContaining({
|
||||
env: expect.objectContaining({
|
||||
TEST_ENV: 'test-value',
|
||||
}),
|
||||
}),
|
||||
)
|
||||
|
||||
// Simulate stdout data
|
||||
mockStdout.emit('data', Buffer.from('hello output'))
|
||||
mockChild.emit('close', 0)
|
||||
|
||||
const result = await executePromise
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- ToolResponse content is an array of content blocks
|
||||
expect((result as any).content[0].text).toContain('hello output')
|
||||
})
|
||||
|
||||
it('should block user-requested commands when restrictedUser is enabled', async () => {
|
||||
const tool = createSandboxedBashTool(mockCwd, {}, { restrictedUser: true })
|
||||
|
||||
const signal = createMockSignal()
|
||||
await expect(
|
||||
invokeTool(tool, '1', { command: 'echo hack', source: 'user' }, signal, createMockOnUpdate()),
|
||||
).rejects.toThrow(/not permitted for your role/)
|
||||
|
||||
expect(spawn).not.toHaveBeenCalled()
|
||||
expect(SandboxManager.wrapWithSandbox).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should allow skill-requested commands when restrictedUser is enabled', async () => {
|
||||
const tool = createSandboxedBashTool(mockCwd, {}, { restrictedUser: true })
|
||||
|
||||
const mockStdout = new EventEmitter()
|
||||
const mockStderr = new EventEmitter()
|
||||
const mockChild = Object.assign(new EventEmitter(), {
|
||||
stdout: mockStdout,
|
||||
stderr: mockStderr,
|
||||
pid: 124,
|
||||
kill: vi.fn(),
|
||||
})
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- spawn is mocked to return a partial ChildProcess
|
||||
;(spawn as any).mockReturnValue(mockChild)
|
||||
|
||||
const signal = createMockSignal()
|
||||
const executePromise = invokeTool(tool, '1', { command: 'echo skill', source: 'skill' }, signal)
|
||||
|
||||
await vi.waitFor(() => expect(spawn).toHaveBeenCalled())
|
||||
expect(SandboxManager.wrapWithSandbox).toHaveBeenCalledWith(
|
||||
'echo skill',
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
{ commandId: '1' },
|
||||
)
|
||||
|
||||
mockStdout.emit('data', Buffer.from('skill output'))
|
||||
mockChild.emit('close', 0)
|
||||
|
||||
const result = await executePromise
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- ToolResponse content is an array of content blocks
|
||||
expect((result as any).content[0].text).toContain('skill output')
|
||||
})
|
||||
|
||||
it('should handle process errors', async () => {
|
||||
const tool = createSandboxedBashTool(mockCwd, mockSessionManager.getSkillEnvs())
|
||||
const mockChild = new EventEmitter()
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- spawn returns mocked ChildProcess
|
||||
;(spawn as any).mockReturnValue(mockChild)
|
||||
|
||||
const signal = createMockSignal()
|
||||
const executePromise = invokeTool(
|
||||
tool,
|
||||
'1',
|
||||
{ command: 'fail', source: 'user' },
|
||||
signal,
|
||||
createMockOnUpdate(),
|
||||
)
|
||||
|
||||
await vi.waitFor(() => expect(spawn).toHaveBeenCalled())
|
||||
mockChild.emit('error', new Error('spawn failed'))
|
||||
|
||||
await expect(executePromise).rejects.toThrow('spawn failed')
|
||||
})
|
||||
|
||||
it('should handle non-zero exit codes', async () => {
|
||||
const tool = createSandboxedBashTool(mockCwd, mockSessionManager.getSkillEnvs())
|
||||
const mockChild = Object.assign(new EventEmitter(), {
|
||||
stdout: new EventEmitter(),
|
||||
stderr: new EventEmitter(),
|
||||
pid: 1,
|
||||
})
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- spawn returns mocked ChildProcess
|
||||
;(spawn as any).mockReturnValue(mockChild)
|
||||
|
||||
const signal = createMockSignal()
|
||||
const executePromise = invokeTool(
|
||||
tool,
|
||||
'1',
|
||||
{ command: 'exit 1', source: 'user' },
|
||||
signal,
|
||||
createMockOnUpdate(),
|
||||
)
|
||||
|
||||
await vi.waitFor(() => expect(spawn).toHaveBeenCalled())
|
||||
mockChild.emit('close', 1)
|
||||
|
||||
await expect(executePromise).rejects.toThrow(/exited with code 1/)
|
||||
})
|
||||
|
||||
it('should handle timeouts', async () => {
|
||||
vi.useFakeTimers()
|
||||
|
||||
const tool = createSandboxedBashTool(mockCwd, mockSessionManager.getSkillEnvs())
|
||||
const mockChild = Object.assign(new EventEmitter(), {
|
||||
stdout: new EventEmitter(),
|
||||
stderr: new EventEmitter(),
|
||||
pid: 456,
|
||||
kill: vi.fn(),
|
||||
})
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- spawn returns mocked ChildProcess
|
||||
;(spawn as any).mockReturnValue(mockChild)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- process.kill return type is boolean
|
||||
const killSpy = vi.spyOn(process, 'kill').mockImplementation(() => true as any)
|
||||
|
||||
const signal = createMockSignal()
|
||||
const executePromise = invokeTool(
|
||||
tool,
|
||||
'1',
|
||||
{ command: 'sleep 10', timeout: 2, source: 'user' },
|
||||
signal,
|
||||
createMockOnUpdate(),
|
||||
)
|
||||
|
||||
await vi.waitFor(() => expect(spawn).toHaveBeenCalled())
|
||||
|
||||
await vi.advanceTimersByTimeAsync(2100)
|
||||
|
||||
expect(killSpy).toHaveBeenCalledWith(-456, 'SIGKILL')
|
||||
|
||||
mockChild.emit('close', null)
|
||||
|
||||
await expect(executePromise).rejects.toThrow(/timed out/)
|
||||
vi.useRealTimers()
|
||||
killSpy.mockRestore()
|
||||
})
|
||||
|
||||
it('should handle abort signals', async () => {
|
||||
const tool = createSandboxedBashTool(mockCwd, mockSessionManager.getSkillEnvs())
|
||||
const mockChild = Object.assign(new EventEmitter(), {
|
||||
stdout: new EventEmitter(),
|
||||
stderr: new EventEmitter(),
|
||||
pid: 789,
|
||||
kill: vi.fn(),
|
||||
})
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- spawn returns mocked ChildProcess
|
||||
;(spawn as any).mockReturnValue(mockChild)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- process.kill return type is boolean
|
||||
const killSpy = vi.spyOn(process, 'kill').mockImplementation(() => true as any)
|
||||
|
||||
const signal = createMockSignal()
|
||||
const executePromise = invokeTool(
|
||||
tool,
|
||||
'1',
|
||||
{ command: 'long-run', source: 'user' },
|
||||
signal,
|
||||
createMockOnUpdate(),
|
||||
)
|
||||
|
||||
await vi.waitFor(() => expect(spawn).toHaveBeenCalled())
|
||||
|
||||
signal.aborted = true
|
||||
signal.emit('abort')
|
||||
|
||||
expect(killSpy).toHaveBeenCalledWith(-789, 'SIGKILL')
|
||||
|
||||
mockChild.emit('close', null)
|
||||
|
||||
await expect(executePromise).rejects.toThrow(/aborted/)
|
||||
killSpy.mockRestore()
|
||||
})
|
||||
|
||||
it('should truncate output exceeding max lines (2000 lines) and persist full output to temp file', async () => {
|
||||
const tool = createSandboxedBashTool(mockCwd, mockSessionManager.getSkillEnvs())
|
||||
const mockStdout = new EventEmitter()
|
||||
const mockStderr = new EventEmitter()
|
||||
const mockChild = Object.assign(new EventEmitter(), {
|
||||
stdout: mockStdout,
|
||||
stderr: mockStderr,
|
||||
pid: 101,
|
||||
kill: vi.fn(),
|
||||
})
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- spawn returns mocked ChildProcess
|
||||
;(spawn as any).mockReturnValue(mockChild)
|
||||
|
||||
const signal = createMockSignal()
|
||||
const executePromise = invokeTool(
|
||||
tool,
|
||||
'1',
|
||||
{ command: 'generate lines', source: 'user' },
|
||||
signal,
|
||||
)
|
||||
|
||||
await vi.waitFor(() => expect(spawn).toHaveBeenCalled())
|
||||
|
||||
const lines: string[] = []
|
||||
for (let i = 1; i <= 3000; i++) {
|
||||
lines.push(`line-${i}`)
|
||||
}
|
||||
mockStdout.emit('data', Buffer.from(lines.join('\n') + '\n'))
|
||||
mockChild.emit('close', 0)
|
||||
|
||||
const result = await executePromise
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- content block array
|
||||
const text = (result as any).content[0].text
|
||||
expect(text).toContain('line-3000')
|
||||
expect(text).toContain('[Showing lines 1001-3000 of 3000. Full output:')
|
||||
expect(result.details?.truncation).toMatchObject({
|
||||
truncated: true,
|
||||
truncatedBy: 'lines',
|
||||
totalLines: 3000,
|
||||
outputLines: 2000,
|
||||
})
|
||||
|
||||
expect(result.details?.fullOutputPath).toBeDefined()
|
||||
const fileContent = await readFile(result.details!.fullOutputPath!, 'utf-8')
|
||||
expect(fileContent).toContain('line-1\nline-2')
|
||||
expect(fileContent).toContain('line-2999\nline-3000')
|
||||
})
|
||||
|
||||
it('should truncate output exceeding max bytes (50KB) and report partial line size', async () => {
|
||||
const tool = createSandboxedBashTool(mockCwd, mockSessionManager.getSkillEnvs())
|
||||
const mockStdout = new EventEmitter()
|
||||
const mockChild = Object.assign(new EventEmitter(), {
|
||||
stdout: mockStdout,
|
||||
stderr: new EventEmitter(),
|
||||
pid: 102,
|
||||
kill: vi.fn(),
|
||||
})
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- spawn returns mocked ChildProcess
|
||||
;(spawn as any).mockReturnValue(mockChild)
|
||||
|
||||
const signal = createMockSignal()
|
||||
const executePromise = invokeTool(
|
||||
tool,
|
||||
'1',
|
||||
{ command: 'long single line', source: 'user' },
|
||||
signal,
|
||||
)
|
||||
|
||||
await vi.waitFor(() => expect(spawn).toHaveBeenCalled())
|
||||
|
||||
const longLine = 'a'.repeat(60000)
|
||||
mockStdout.emit('data', Buffer.from(longLine))
|
||||
mockChild.emit('close', 0)
|
||||
|
||||
const result = await executePromise
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- content block array
|
||||
const text = (result as any).content[0].text
|
||||
expect(text).toMatch(/Showing last 50\.0KB of line 1 \(line is 58\.6KB\)\. Full output:/)
|
||||
})
|
||||
|
||||
it('should preserve truncated output when a command times out', async () => {
|
||||
vi.useFakeTimers()
|
||||
const tool = createSandboxedBashTool(mockCwd, mockSessionManager.getSkillEnvs())
|
||||
const mockStdout = new EventEmitter()
|
||||
const mockChild = Object.assign(new EventEmitter(), {
|
||||
stdout: mockStdout,
|
||||
stderr: new EventEmitter(),
|
||||
pid: 103,
|
||||
kill: vi.fn(),
|
||||
})
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- spawn returns mocked ChildProcess
|
||||
;(spawn as any).mockReturnValue(mockChild)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- process.kill return type boolean
|
||||
const killSpy = vi.spyOn(process, 'kill').mockImplementation(() => true as any)
|
||||
|
||||
const signal = createMockSignal()
|
||||
const executePromise = invokeTool(
|
||||
tool,
|
||||
'1',
|
||||
{ command: 'slow', timeout: 1, source: 'user' },
|
||||
signal,
|
||||
)
|
||||
|
||||
await vi.waitFor(() => expect(spawn).toHaveBeenCalled())
|
||||
|
||||
mockStdout.emit('data', Buffer.from('output before timeout\n'))
|
||||
await vi.advanceTimersByTimeAsync(1100)
|
||||
mockChild.emit('close', null)
|
||||
|
||||
await expect(executePromise).rejects.toThrow(/output before timeout/)
|
||||
await expect(executePromise).rejects.toThrow(/timed out after 1 seconds/)
|
||||
|
||||
vi.useRealTimers()
|
||||
killSpy.mockRestore()
|
||||
})
|
||||
})
|
||||
@@ -1,260 +0,0 @@
|
||||
import {
|
||||
type ToolRegistration as AgentTool,
|
||||
type ToolExecutionResult as AgentToolResult,
|
||||
} from '@earendil-works/pi-durable'
|
||||
import { logger } from '@shumai/core/src/logger'
|
||||
import { quotaService } from '@shumai/core/src/quota/quota-service'
|
||||
import { sandboxService } from '@shumai/core/src/sandbox/sandbox-service'
|
||||
import type { TeamMemberRole } from '@shumai/db'
|
||||
import { spawn } from 'node:child_process'
|
||||
import { Type } from 'typebox'
|
||||
import { OutputAccumulator } from '../utils/output-accumulator'
|
||||
import {
|
||||
DEFAULT_MAX_BYTES,
|
||||
DEFAULT_MAX_LINES,
|
||||
formatSize,
|
||||
type TruncationResult,
|
||||
} from '../utils/truncate'
|
||||
|
||||
const BashSource = Type.String({
|
||||
enum: ['user', 'skill'],
|
||||
description:
|
||||
'Indicates who requested this bash command. Set to "user" when the end user directly ' +
|
||||
'requested the command. Set to "skill" when the command is required by a skill you loaded ' +
|
||||
'via the read_skill tool. Users without the owner role can only execute commands with ' +
|
||||
'source="skill"; source="user" will be blocked.',
|
||||
})
|
||||
|
||||
const BashParameters = Type.Object({
|
||||
command: Type.String({ description: 'The bash command to execute.' }),
|
||||
timeout: Type.Optional(Type.Number({ description: 'Maximum execution time in seconds.' })),
|
||||
source: BashSource,
|
||||
})
|
||||
|
||||
export type BashDetails = {
|
||||
exitCode: number | null
|
||||
truncation?: TruncationResult
|
||||
fullOutputPath?: string
|
||||
}
|
||||
|
||||
export interface SandboxedBashOptions {
|
||||
teamId?: string
|
||||
userId?: string
|
||||
role?: TeamMemberRole | null
|
||||
getBlockedHost?: (commandId?: string) => string
|
||||
clearBlockedHost?: () => void
|
||||
/**
|
||||
* When true, the tool only allows commands requested by a loaded skill (source="skill").
|
||||
* Commands requested directly by the end user (source="user") are blocked.
|
||||
*/
|
||||
restrictedUser?: boolean
|
||||
}
|
||||
|
||||
export const createSandboxedBashTool = (
|
||||
cwd: string,
|
||||
skillEnvs: Record<string, string> = {},
|
||||
options?: SandboxedBashOptions,
|
||||
): AgentTool<typeof BashParameters, BashDetails> => {
|
||||
return {
|
||||
name: 'bash',
|
||||
description: `Execute a bash command in a sandboxed environment. Returns stdout and stderr. Output is truncated to last ${DEFAULT_MAX_LINES} lines or ${DEFAULT_MAX_BYTES / 1024}KB (whichever is hit first). If truncated, full output is saved to a temp file. Optionally provide a timeout in seconds. When in a restricted (non-owner) user context, only commands required by a skill loaded via the read_skill tool are allowed, and you must set source="skill".`,
|
||||
parameters: BashParameters,
|
||||
outputLimits: { maxBytes: DEFAULT_MAX_BYTES, maxLines: DEFAULT_MAX_LINES, retain: 'tail' },
|
||||
execute: async (
|
||||
{ command, timeout, source },
|
||||
api,
|
||||
context,
|
||||
): Promise<AgentToolResult<BashDetails>> => {
|
||||
const toolCallId = api.callId
|
||||
const signal = context.abortSignal
|
||||
if (options?.restrictedUser && source !== 'skill') {
|
||||
throw new Error(
|
||||
'Bash commands requested directly by the user (source="user") are not permitted for your role. ' +
|
||||
'Only commands required by a loaded skill (source="skill") can be executed.',
|
||||
)
|
||||
}
|
||||
if (options?.teamId) {
|
||||
await quotaService.checkQuota(
|
||||
{
|
||||
teamId: options.teamId,
|
||||
userId: options.userId,
|
||||
role: options.role,
|
||||
resource: 'agent_bash_call_count',
|
||||
resourceData: { command },
|
||||
},
|
||||
1,
|
||||
)
|
||||
}
|
||||
options?.clearBlockedHost?.()
|
||||
const wrappedCommand = await sandboxService.wrapCommand(command, { commandId: toolCallId })
|
||||
|
||||
const output = new OutputAccumulator({ tempFilePrefix: 'shumai-bash' })
|
||||
let acceptingOutput = true
|
||||
await api.details({ exitCode: null }, context)
|
||||
|
||||
const handleData = (data: Buffer) => {
|
||||
if (!acceptingOutput) return
|
||||
output.append(data)
|
||||
api.output(data)
|
||||
}
|
||||
|
||||
const finishOutput = async () => {
|
||||
acceptingOutput = false
|
||||
output.finish()
|
||||
const snapshot = output.snapshot({ persistIfTruncated: true })
|
||||
await output.closeTempFile()
|
||||
return snapshot
|
||||
}
|
||||
|
||||
const formatOutput = (
|
||||
snapshot: Awaited<ReturnType<typeof finishOutput>>,
|
||||
emptyText = '(no output)',
|
||||
) => {
|
||||
const truncation = snapshot.truncation
|
||||
let text = snapshot.content || emptyText
|
||||
let details: BashDetails | undefined
|
||||
if (truncation.truncated) {
|
||||
details = { exitCode: null, truncation, fullOutputPath: snapshot.fullOutputPath }
|
||||
const startLine = truncation.totalLines - truncation.outputLines + 1
|
||||
const endLine = truncation.totalLines
|
||||
if (truncation.lastLinePartial) {
|
||||
const lastLineSize = formatSize(output.getLastLineBytes())
|
||||
text += `\n\n[Showing last ${formatSize(truncation.outputBytes)} of line ${endLine} (line is ${lastLineSize}). Full output: ${snapshot.fullOutputPath}]`
|
||||
} else if (truncation.truncatedBy === 'lines') {
|
||||
text += `\n\n[Showing lines ${startLine}-${endLine} of ${truncation.totalLines}. Full output: ${snapshot.fullOutputPath}]`
|
||||
} else {
|
||||
text += `\n\n[Showing lines ${startLine}-${endLine} of ${truncation.totalLines} (${formatSize(DEFAULT_MAX_BYTES)} limit). Full output: ${snapshot.fullOutputPath}]`
|
||||
}
|
||||
}
|
||||
return { text, details }
|
||||
}
|
||||
|
||||
const appendStatus = (text: string, status: string) => `${text ? `${text}\n\n` : ''}${status}`
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn('bash', ['-c', wrappedCommand], {
|
||||
cwd,
|
||||
detached: true,
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
env: {
|
||||
PATH: process.env.PATH || '',
|
||||
HOME: process.env.HOME || '',
|
||||
...skillEnvs,
|
||||
},
|
||||
})
|
||||
|
||||
let timedOut = false
|
||||
let timeoutHandle: NodeJS.Timeout | undefined
|
||||
|
||||
if (timeout !== undefined && timeout > 0) {
|
||||
timeoutHandle = setTimeout(() => {
|
||||
timedOut = true
|
||||
if (child.pid) {
|
||||
try {
|
||||
process.kill(-child.pid, 'SIGKILL')
|
||||
} catch {
|
||||
child.kill('SIGKILL')
|
||||
}
|
||||
}
|
||||
}, timeout * 1000)
|
||||
}
|
||||
|
||||
child.stdout?.on('data', handleData)
|
||||
child.stderr?.on('data', handleData)
|
||||
|
||||
const onAbort = () => {
|
||||
if (child.pid) {
|
||||
try {
|
||||
process.kill(-child.pid, 'SIGKILL')
|
||||
} catch {
|
||||
child.kill('SIGKILL')
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
signal?.addEventListener('abort', onAbort, { once: true })
|
||||
|
||||
child.on('error', async (err) => {
|
||||
if (timeoutHandle) clearTimeout(timeoutHandle)
|
||||
signal?.removeEventListener('abort', onAbort)
|
||||
const snapshot = await finishOutput()
|
||||
logger.error(
|
||||
{ err, command, content: snapshot.content },
|
||||
'Sandboxed bash command encountered process error',
|
||||
)
|
||||
reject(err)
|
||||
})
|
||||
|
||||
child.on('close', async (code) => {
|
||||
if (timeoutHandle) clearTimeout(timeoutHandle)
|
||||
signal?.removeEventListener('abort', onAbort)
|
||||
|
||||
const blocked = options?.getBlockedHost?.(toolCallId)
|
||||
const snapshot = await finishOutput()
|
||||
const { text: outputText, details: truncationDetails } = formatOutput(snapshot)
|
||||
|
||||
if (options?.teamId) {
|
||||
try {
|
||||
await quotaService.consumeQuota(
|
||||
{
|
||||
teamId: options.teamId,
|
||||
userId: options.userId,
|
||||
role: options.role,
|
||||
resource: 'agent_bash_call_count',
|
||||
resourceData: { command },
|
||||
},
|
||||
1,
|
||||
)
|
||||
} catch (err) {
|
||||
logger.error({ err }, 'Failed to record quota usage for bash command')
|
||||
}
|
||||
}
|
||||
|
||||
if (signal?.aborted) {
|
||||
logger.error({ command, outputText }, 'Sandboxed bash command aborted')
|
||||
reject(new Error(appendStatus(outputText, 'Command aborted')))
|
||||
} else if (timedOut) {
|
||||
logger.error({ timeout, command, outputText }, 'Sandboxed bash command timed out')
|
||||
reject(
|
||||
new Error(
|
||||
appendStatus(
|
||||
outputText,
|
||||
`Sandboxed bash command timed out after ${timeout} seconds`,
|
||||
),
|
||||
),
|
||||
)
|
||||
} else if (blocked) {
|
||||
reject(
|
||||
new Error(
|
||||
`Network request to ${blocked} is blocked, please ask admin to allow it in sandbox settings.`,
|
||||
),
|
||||
)
|
||||
} else if (code !== 0 && code !== null) {
|
||||
logger.error(
|
||||
{ exitCode: code, command, outputText },
|
||||
'Sandboxed bash command exited with failure code',
|
||||
)
|
||||
reject(
|
||||
new Error(
|
||||
appendStatus(outputText, `Sandboxed bash command exited with code ${code}`),
|
||||
),
|
||||
)
|
||||
} else {
|
||||
resolve({
|
||||
output: [
|
||||
{
|
||||
type: 'text',
|
||||
text: outputText,
|
||||
},
|
||||
],
|
||||
details: {
|
||||
exitCode: code,
|
||||
...(truncationDetails ?? {}),
|
||||
},
|
||||
})
|
||||
}
|
||||
})
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { OutputAccumulator } from './output-accumulator'
|
||||
|
||||
describe('OutputAccumulator', () => {
|
||||
it('does not accumulate rawChunks in memory when disableTempFile is true', () => {
|
||||
const acc = new OutputAccumulator({
|
||||
maxBytes: 1024,
|
||||
maxLines: 50,
|
||||
disableTempFile: true,
|
||||
})
|
||||
|
||||
const chunk = Buffer.from(`${'x'.repeat(2047)}\n`, 'utf-8')
|
||||
for (let i = 0; i < 256; i++) {
|
||||
acc.append(chunk)
|
||||
}
|
||||
acc.finish()
|
||||
|
||||
const internal = acc as unknown as { rawChunks: Buffer[]; tailBytes: number }
|
||||
expect(internal.rawChunks).toEqual([])
|
||||
expect(internal.tailBytes).toBeLessThanOrEqual(1024 * 4)
|
||||
|
||||
const snapshot = acc.snapshot({ persistIfTruncated: false })
|
||||
expect(snapshot.truncation.truncated).toBe(true)
|
||||
expect(snapshot.fullOutputPath).toBeUndefined()
|
||||
expect(Buffer.byteLength(snapshot.content, 'utf-8')).toBeLessThanOrEqual(1024)
|
||||
})
|
||||
})
|
||||
@@ -13,6 +13,7 @@ export interface OutputAccumulatorOptions {
|
||||
maxLines?: number
|
||||
maxBytes?: number
|
||||
tempFilePrefix?: string
|
||||
disableTempFile?: boolean
|
||||
}
|
||||
|
||||
export interface OutputSnapshot {
|
||||
@@ -35,6 +36,7 @@ export class OutputAccumulator {
|
||||
private readonly maxBytes: number
|
||||
private readonly maxRollingBytes: number
|
||||
private readonly tempFilePrefix: string
|
||||
private readonly disableTempFile: boolean
|
||||
private readonly decoder = new TextDecoder()
|
||||
|
||||
private rawChunks: Buffer[] = []
|
||||
@@ -57,6 +59,7 @@ export class OutputAccumulator {
|
||||
this.maxBytes = options.maxBytes ?? DEFAULT_MAX_BYTES
|
||||
this.maxRollingBytes = Math.max(this.maxBytes * 2, 1)
|
||||
this.tempFilePrefix = options.tempFilePrefix ?? 'shumai-bash'
|
||||
this.disableTempFile = options.disableTempFile ?? false
|
||||
}
|
||||
|
||||
append(data: Buffer): void {
|
||||
@@ -70,7 +73,7 @@ export class OutputAccumulator {
|
||||
if (this.tempFileStream || this.shouldUseTempFile()) {
|
||||
this.ensureTempFile()
|
||||
this.tempFileStream?.write(data)
|
||||
} else if (data.length > 0) {
|
||||
} else if (!this.disableTempFile && data.length > 0) {
|
||||
this.rawChunks.push(data)
|
||||
}
|
||||
}
|
||||
@@ -202,6 +205,9 @@ export class OutputAccumulator {
|
||||
}
|
||||
|
||||
private shouldUseTempFile(): boolean {
|
||||
if (this.disableTempFile) {
|
||||
return false
|
||||
}
|
||||
return (
|
||||
this.totalRawBytes > this.maxBytes ||
|
||||
this.totalDecodedBytes > this.maxBytes ||
|
||||
@@ -210,7 +216,7 @@ export class OutputAccumulator {
|
||||
}
|
||||
|
||||
private ensureTempFile(): void {
|
||||
if (this.tempFilePath) {
|
||||
if (this.disableTempFile || this.tempFilePath) {
|
||||
return
|
||||
}
|
||||
this.tempFilePath = defaultTempFilePath(this.tempFilePrefix)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@shumai/api",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "./src/index.ts",
|
||||
|
||||
@@ -30,6 +30,9 @@ import uploadRoute, { localUploadRoute } from './upload'
|
||||
import versionStackRoute from './versionStack'
|
||||
import kanbanRoute from './kanban'
|
||||
import mediaGenerationRoute from './media-generation'
|
||||
import runnerWsRoute, { runnerWebsocket } from './runner-ws'
|
||||
|
||||
export { runnerWebsocket }
|
||||
|
||||
type User = Prisma.UserGetPayload<Record<string, never>>
|
||||
|
||||
@@ -42,6 +45,7 @@ const apiRoute = new Hono<{ Variables: { user: User } }>()
|
||||
.route('/', publicShareRoute)
|
||||
.route('/', localUploadRoute)
|
||||
.route('/', mcpOauthCallbackRoute)
|
||||
.route('/', runnerWsRoute)
|
||||
|
||||
// Programmatic endpoint for the CLI: API token required, session cookies not accepted
|
||||
.use('/ops/*', requireApiTokenMiddleware)
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
import { Hono } from 'hono'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { runnerService } from '@shumai/core/src/runner'
|
||||
import runnerWsRoute from './runner-ws'
|
||||
|
||||
vi.mock('@shumai/core/src/runner', () => ({
|
||||
runnerService: {
|
||||
authenticateRunnerKey: vi.fn(),
|
||||
registerConnection: vi.fn(),
|
||||
handleMessage: vi.fn(),
|
||||
handleDisconnect: vi.fn(),
|
||||
},
|
||||
}))
|
||||
|
||||
describe('runnerWsRoute', () => {
|
||||
const app = new Hono().route('/', runnerWsRoute)
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
it('returns 401 when runner key is missing', async () => {
|
||||
const res = await app.request('/runners/ws')
|
||||
expect(res.status).toBe(401)
|
||||
expect(await res.json()).toEqual({ message: 'Missing runner key' })
|
||||
})
|
||||
|
||||
it('returns 401 when runner key is invalid', async () => {
|
||||
vi.mocked(runnerService.authenticateRunnerKey).mockResolvedValue(null)
|
||||
const res = await app.request('/runners/ws', {
|
||||
headers: { Authorization: 'Bearer shm_rn_invalid' },
|
||||
})
|
||||
expect(res.status).toBe(401)
|
||||
expect(await res.json()).toEqual({ message: 'Invalid runner key' })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,75 @@
|
||||
import { Hono } from 'hono'
|
||||
import { type AuthenticatedRunner, runnerService, type RunnerSocket } from '@shumai/core/src/runner'
|
||||
|
||||
export interface RunnerWsData {
|
||||
runner: AuthenticatedRunner
|
||||
}
|
||||
|
||||
interface BunServerLike {
|
||||
upgrade(req: Request, options?: { data?: RunnerWsData }): boolean
|
||||
}
|
||||
|
||||
function extractRunnerKey(authHeader?: string | null, customHeader?: string | null): string | null {
|
||||
if (authHeader && authHeader.toLowerCase().startsWith('bearer ')) {
|
||||
return authHeader.slice(7).trim()
|
||||
}
|
||||
if (customHeader) {
|
||||
return customHeader.trim()
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
const runnerWsRoute = new Hono().get('/runners/ws', async (c) => {
|
||||
const rawKey = extractRunnerKey(c.req.header('authorization'), c.req.header('x-runner-key'))
|
||||
if (!rawKey) {
|
||||
return c.json({ message: 'Missing runner key' }, 401)
|
||||
}
|
||||
|
||||
const runner = await runnerService.authenticateRunnerKey(rawKey)
|
||||
if (!runner) {
|
||||
return c.json({ message: 'Invalid runner key' }, 401)
|
||||
}
|
||||
|
||||
const env = c.env as BunServerLike | { server?: BunServerLike } | undefined
|
||||
const server: BunServerLike | undefined =
|
||||
env && typeof (env as BunServerLike).upgrade === 'function'
|
||||
? (env as BunServerLike)
|
||||
: env && typeof (env as { server?: BunServerLike }).server?.upgrade === 'function'
|
||||
? (env as { server?: BunServerLike }).server
|
||||
: undefined
|
||||
|
||||
if (!server) {
|
||||
return c.json({ message: 'WebSocket upgrade is not supported in this environment' }, 426)
|
||||
}
|
||||
|
||||
const upgraded = server.upgrade(c.req.raw, {
|
||||
data: { runner },
|
||||
})
|
||||
if (upgraded) {
|
||||
return new Response(null)
|
||||
}
|
||||
return c.json({ message: 'WebSocket upgrade failed' }, 400)
|
||||
})
|
||||
|
||||
export const runnerWebsocket = {
|
||||
open(ws: RunnerSocket & { data?: RunnerWsData }) {
|
||||
const runner = ws.data?.runner
|
||||
if (!runner) {
|
||||
ws.close(1008, 'Unauthorized')
|
||||
return
|
||||
}
|
||||
void runnerService.registerConnection(runner, ws)
|
||||
},
|
||||
message(ws: RunnerSocket & { data?: RunnerWsData }, message: string | Buffer | Uint8Array) {
|
||||
const runner = ws.data?.runner
|
||||
if (!runner) return
|
||||
runnerService.handleMessage(runner.id, ws, message)
|
||||
},
|
||||
close(ws: RunnerSocket & { data?: RunnerWsData }, code: number) {
|
||||
const runner = ws.data?.runner
|
||||
if (!runner) return
|
||||
runnerService.handleDisconnect(runner.id, ws, code)
|
||||
},
|
||||
}
|
||||
|
||||
export default runnerWsRoute
|
||||
+147
-53
@@ -75,6 +75,16 @@ vi.mock('@shumai/core/src/workflowTask/workflowTask', () => ({
|
||||
},
|
||||
}))
|
||||
|
||||
vi.mock('@shumai/core/src/runner', () => ({
|
||||
runnerService: {
|
||||
listRunners: vi.fn(),
|
||||
createRunner: vi.fn(),
|
||||
updateRunner: vi.fn(),
|
||||
rotateRunnerKey: vi.fn(),
|
||||
deleteRunner: vi.fn(),
|
||||
},
|
||||
}))
|
||||
|
||||
describe('team api', () => {
|
||||
const app = new Hono().use('*', authMiddleware).route('/', teamRoute)
|
||||
let mockGetUserTeams: any // eslint-disable-line @typescript-eslint/no-explicit-any
|
||||
@@ -84,8 +94,6 @@ describe('team api', () => {
|
||||
let mockUpdateSettings: any // eslint-disable-line @typescript-eslint/no-explicit-any
|
||||
let mockListUserMetadata: any // eslint-disable-line @typescript-eslint/no-explicit-any
|
||||
let mockUpsertUserMetadata: any // eslint-disable-line @typescript-eslint/no-explicit-any
|
||||
let mockGetSandboxSettings: any // eslint-disable-line @typescript-eslint/no-explicit-any
|
||||
let mockUpdateSandboxSettings: any // eslint-disable-line @typescript-eslint/no-explicit-any
|
||||
let mockUpdateMe: any // eslint-disable-line @typescript-eslint/no-explicit-any
|
||||
|
||||
beforeEach(() => {
|
||||
@@ -96,8 +104,6 @@ describe('team api', () => {
|
||||
mockUpdateSettings = vi.spyOn(teamService, 'updateSettings')
|
||||
mockListUserMetadata = vi.spyOn(userMetadataService, 'listMetadata')
|
||||
mockUpsertUserMetadata = vi.spyOn(userMetadataService, 'upsertMetadata')
|
||||
mockGetSandboxSettings = vi.spyOn(teamService, 'getSandboxSettings')
|
||||
mockUpdateSandboxSettings = vi.spyOn(teamService, 'updateSandboxSettings')
|
||||
mockUpdateMe = vi.spyOn(teamService, 'updateMe')
|
||||
vi.mocked(authzService.hasPermission).mockResolvedValue(undefined)
|
||||
})
|
||||
@@ -483,63 +489,151 @@ describe('team api', () => {
|
||||
expect(mockUpsertUserMetadata).toHaveBeenCalledWith('user1', 't1', 'key1', 'value1')
|
||||
})
|
||||
|
||||
it('GET /teams/:teamId/sandbox returns sandbox settings', async () => {
|
||||
mockGetSandboxSettings.mockResolvedValue({
|
||||
networkSandboxEnabled: false,
|
||||
allowedDomains: ['example.com'],
|
||||
pendingDomains: ['pending.com'],
|
||||
describe('Runners', () => {
|
||||
it('GET /teams/:teamId/runners returns team runners', async () => {
|
||||
const { runnerService } = await import('@shumai/core/src/runner')
|
||||
vi.mocked(runnerService.listRunners).mockResolvedValue([
|
||||
{
|
||||
id: 'r1',
|
||||
teamId: 't1',
|
||||
name: 'Local Mac Mini',
|
||||
keyPrefix: 'shm_rn_12345678',
|
||||
isDefault: true,
|
||||
status: 'connected',
|
||||
lastConnectedAt: '2026-11-14T00:00:00.000Z',
|
||||
lastDisconnectedAt: null,
|
||||
createdAt: '2026-11-14T00:00:00.000Z',
|
||||
updatedAt: '2026-11-14T00:00:00.000Z',
|
||||
},
|
||||
])
|
||||
|
||||
const res = await app.request('/teams/t1/runners')
|
||||
|
||||
expect(res.status).toBe(200)
|
||||
const data = await res.json()
|
||||
expect(data).toHaveLength(1)
|
||||
expect(data[0].name).toBe('Local Mac Mini')
|
||||
expect(authzService.hasPermission).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
type: ResourceType.Team,
|
||||
id: 't1',
|
||||
permission: Permission.Read,
|
||||
}),
|
||||
)
|
||||
expect(runnerService.listRunners).toHaveBeenCalledWith('t1')
|
||||
})
|
||||
|
||||
const res = await app.request('/teams/t1/sandbox')
|
||||
it('POST /teams/:teamId/runners creates a runner and returns key', async () => {
|
||||
const { runnerService } = await import('@shumai/core/src/runner')
|
||||
vi.mocked(runnerService.createRunner).mockResolvedValue({
|
||||
runner: {
|
||||
id: 'r1',
|
||||
teamId: 't1',
|
||||
name: 'Local Mac Mini',
|
||||
keyPrefix: 'shm_rn_12345678',
|
||||
isDefault: true,
|
||||
status: 'disconnected',
|
||||
lastConnectedAt: null,
|
||||
lastDisconnectedAt: null,
|
||||
createdAt: '2026-11-14T00:00:00.000Z',
|
||||
updatedAt: '2026-11-14T00:00:00.000Z',
|
||||
},
|
||||
key: 'shm_rn_1234567890abcdef1234567890abcdef1234567890abcdef',
|
||||
})
|
||||
|
||||
expect(res.status).toBe(200)
|
||||
const data = await res.json()
|
||||
expect(data.networkSandboxEnabled).toBe(false)
|
||||
expect(data.allowedDomains).toEqual(['example.com'])
|
||||
expect(data.pendingDomains).toEqual(['pending.com'])
|
||||
expect(authzService.hasPermission).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
type: ResourceType.Team,
|
||||
id: 't1',
|
||||
permission: Permission.Read,
|
||||
}),
|
||||
)
|
||||
expect(mockGetSandboxSettings).toHaveBeenCalledWith('t1')
|
||||
})
|
||||
const res = await app.request('/teams/t1/runners', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ name: 'Local Mac Mini', isDefault: true }),
|
||||
})
|
||||
|
||||
it('PUT /teams/:teamId/sandbox updates sandbox settings', async () => {
|
||||
mockUpdateSandboxSettings.mockResolvedValue({
|
||||
networkSandboxEnabled: true,
|
||||
allowedDomains: ['new.com'],
|
||||
pendingDomains: ['new-pending.com'],
|
||||
expect(res.status).toBe(200)
|
||||
const data = await res.json()
|
||||
expect(data.runner.id).toBe('r1')
|
||||
expect(data.key).toMatch(/^shm_rn_/)
|
||||
expect(authzService.hasPermission).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
type: ResourceType.Team,
|
||||
id: 't1',
|
||||
permission: Permission.Admin,
|
||||
}),
|
||||
)
|
||||
expect(runnerService.createRunner).toHaveBeenCalledWith('t1', {
|
||||
name: 'Local Mac Mini',
|
||||
isDefault: true,
|
||||
})
|
||||
})
|
||||
|
||||
const res = await app.request('/teams/t1/sandbox', {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
networkSandboxEnabled: true,
|
||||
allowedDomains: ['new.com'],
|
||||
pendingDomains: ['new-pending.com'],
|
||||
}),
|
||||
it('PATCH /teams/:teamId/runners/:runnerId updates a runner', async () => {
|
||||
const { runnerService } = await import('@shumai/core/src/runner')
|
||||
vi.mocked(runnerService.updateRunner).mockResolvedValue({
|
||||
id: 'r1',
|
||||
teamId: 't1',
|
||||
name: 'Renamed Runner',
|
||||
keyPrefix: 'shm_rn_12345678',
|
||||
isDefault: true,
|
||||
status: 'disconnected',
|
||||
lastConnectedAt: null,
|
||||
lastDisconnectedAt: null,
|
||||
createdAt: '2026-11-14T00:00:00.000Z',
|
||||
updatedAt: '2026-11-14T00:00:00.000Z',
|
||||
})
|
||||
|
||||
const res = await app.request('/teams/t1/runners/r1', {
|
||||
method: 'PATCH',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ name: 'Renamed Runner', isDefault: true }),
|
||||
})
|
||||
|
||||
expect(res.status).toBe(200)
|
||||
const data = await res.json()
|
||||
expect(data.name).toBe('Renamed Runner')
|
||||
expect(runnerService.updateRunner).toHaveBeenCalledWith('t1', 'r1', {
|
||||
name: 'Renamed Runner',
|
||||
isDefault: true,
|
||||
})
|
||||
})
|
||||
|
||||
expect(res.status).toBe(200)
|
||||
const data = await res.json()
|
||||
expect(data.networkSandboxEnabled).toBe(true)
|
||||
expect(data.allowedDomains).toEqual(['new.com'])
|
||||
expect(data.pendingDomains).toEqual(['new-pending.com'])
|
||||
expect(authzService.hasPermission).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
type: ResourceType.Team,
|
||||
id: 't1',
|
||||
permission: Permission.Admin,
|
||||
}),
|
||||
)
|
||||
expect(mockUpdateSandboxSettings).toHaveBeenCalledWith('t1', {
|
||||
networkSandboxEnabled: true,
|
||||
allowedDomains: ['new.com'],
|
||||
pendingDomains: ['new-pending.com'],
|
||||
it('POST /teams/:teamId/runners/:runnerId/rotate-key rotates runner key', async () => {
|
||||
const { runnerService } = await import('@shumai/core/src/runner')
|
||||
vi.mocked(runnerService.rotateRunnerKey).mockResolvedValue({
|
||||
runner: {
|
||||
id: 'r1',
|
||||
teamId: 't1',
|
||||
name: 'Local Mac Mini',
|
||||
keyPrefix: 'shm_rn_abcdef12',
|
||||
isDefault: true,
|
||||
status: 'disconnected',
|
||||
lastConnectedAt: null,
|
||||
lastDisconnectedAt: null,
|
||||
createdAt: '2026-11-14T00:00:00.000Z',
|
||||
updatedAt: '2026-11-14T00:00:00.000Z',
|
||||
},
|
||||
key: 'shm_rn_abcdef1234567890abcdef1234567890abcdef1234567890',
|
||||
})
|
||||
|
||||
const res = await app.request('/teams/t1/runners/r1/rotate-key', {
|
||||
method: 'POST',
|
||||
})
|
||||
|
||||
expect(res.status).toBe(200)
|
||||
const data = await res.json()
|
||||
expect(data.key).toMatch(/^shm_rn_abcdef12/)
|
||||
expect(runnerService.rotateRunnerKey).toHaveBeenCalledWith('t1', 'r1')
|
||||
})
|
||||
|
||||
it('DELETE /teams/:teamId/runners/:runnerId deletes a runner', async () => {
|
||||
const { runnerService } = await import('@shumai/core/src/runner')
|
||||
vi.mocked(runnerService.deleteRunner).mockResolvedValue(undefined)
|
||||
|
||||
const res = await app.request('/teams/t1/runners/r1', {
|
||||
method: 'DELETE',
|
||||
})
|
||||
|
||||
expect(res.status).toBe(200)
|
||||
const data = await res.json()
|
||||
expect(data.success).toBe(true)
|
||||
expect(runnerService.deleteRunner).toHaveBeenCalledWith('t1', 'r1')
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
@@ -1,13 +1,15 @@
|
||||
import { Hono } from 'hono'
|
||||
import { zValidator } from '@hono/zod-validator'
|
||||
import { authzService, Permission, ResourceType } from '@shumai/core/src/authz/authz'
|
||||
import { runnerService } from '@shumai/core/src/runner'
|
||||
import { teamService } from '@shumai/core/src/team/team'
|
||||
import { userMetadataService } from '@shumai/core/src/user-metadata/user-metadata'
|
||||
import {
|
||||
getUserTeamsRequestSchema,
|
||||
updateTeamSettingsRequestSchema,
|
||||
listMembersQuerySchema,
|
||||
updateSandboxSettingsRequestSchema,
|
||||
createRunnerRequestSchema,
|
||||
updateRunnerRequestSchema,
|
||||
updateTurntableSettingsRequestSchema,
|
||||
updateGotenbergSettingsRequestSchema,
|
||||
updateMeRequestSchema,
|
||||
@@ -210,7 +212,7 @@ const route = new Hono<{ Variables: { user: User } }>()
|
||||
return c.json(settings)
|
||||
},
|
||||
)
|
||||
.get('/teams/:teamId/sandbox', async (c) => {
|
||||
.get('/teams/:teamId/runners', async (c) => {
|
||||
const user = c.get('user')
|
||||
const teamId = c.req.param('teamId')
|
||||
|
||||
@@ -221,15 +223,31 @@ const route = new Hono<{ Variables: { user: User } }>()
|
||||
id: teamId,
|
||||
})
|
||||
|
||||
const sandbox = await teamService.getSandboxSettings(teamId)
|
||||
return c.json(sandbox)
|
||||
const runners = await runnerService.listRunners(teamId)
|
||||
return c.json(runners)
|
||||
})
|
||||
.put(
|
||||
'/teams/:teamId/sandbox',
|
||||
zValidator('json', updateSandboxSettingsRequestSchema),
|
||||
.post('/teams/:teamId/runners', zValidator('json', createRunnerRequestSchema), async (c) => {
|
||||
const user = c.get('user')
|
||||
const teamId = c.req.param('teamId')
|
||||
const req = c.req.valid('json')
|
||||
|
||||
await authzService.hasPermission({
|
||||
user,
|
||||
permission: Permission.Admin,
|
||||
type: ResourceType.Team,
|
||||
id: teamId,
|
||||
})
|
||||
|
||||
const created = await runnerService.createRunner(teamId, req)
|
||||
return c.json(created)
|
||||
})
|
||||
.patch(
|
||||
'/teams/:teamId/runners/:runnerId',
|
||||
zValidator('json', updateRunnerRequestSchema),
|
||||
async (c) => {
|
||||
const user = c.get('user')
|
||||
const teamId = c.req.param('teamId')
|
||||
const runnerId = c.req.param('runnerId')
|
||||
const req = c.req.valid('json')
|
||||
|
||||
await authzService.hasPermission({
|
||||
@@ -239,10 +257,40 @@ const route = new Hono<{ Variables: { user: User } }>()
|
||||
id: teamId,
|
||||
})
|
||||
|
||||
const sandbox = await teamService.updateSandboxSettings(teamId, req)
|
||||
return c.json(sandbox)
|
||||
const updated = await runnerService.updateRunner(teamId, runnerId, req)
|
||||
return c.json(updated)
|
||||
},
|
||||
)
|
||||
.post('/teams/:teamId/runners/:runnerId/rotate-key', async (c) => {
|
||||
const user = c.get('user')
|
||||
const teamId = c.req.param('teamId')
|
||||
const runnerId = c.req.param('runnerId')
|
||||
|
||||
await authzService.hasPermission({
|
||||
user,
|
||||
permission: Permission.Admin,
|
||||
type: ResourceType.Team,
|
||||
id: teamId,
|
||||
})
|
||||
|
||||
const rotated = await runnerService.rotateRunnerKey(teamId, runnerId)
|
||||
return c.json(rotated)
|
||||
})
|
||||
.delete('/teams/:teamId/runners/:runnerId', async (c) => {
|
||||
const user = c.get('user')
|
||||
const teamId = c.req.param('teamId')
|
||||
const runnerId = c.req.param('runnerId')
|
||||
|
||||
await authzService.hasPermission({
|
||||
user,
|
||||
permission: Permission.Admin,
|
||||
type: ResourceType.Team,
|
||||
id: teamId,
|
||||
})
|
||||
|
||||
await runnerService.deleteRunner(teamId, runnerId)
|
||||
return c.json({ success: true })
|
||||
})
|
||||
.get('/teams/:teamId/turntable-settings', async (c) => {
|
||||
const user = c.get('user')
|
||||
const teamId = c.req.param('teamId')
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@shumai/core",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "./src/index.ts",
|
||||
@@ -24,7 +24,6 @@
|
||||
"@ai-sdk/replicate": "3.0.34",
|
||||
"@ai-sdk/togetherai": "3.0.41",
|
||||
"@ai-sdk/xai": "4.0.49",
|
||||
"@anthropic-ai/sandbox-runtime": "0.0.74",
|
||||
"@aws-sdk/client-s3": "3.1121.0",
|
||||
"@aws-sdk/s3-request-presigner": "3.1121.0",
|
||||
"@better-auth/mcp": "1.7.12",
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
import { prisma } from '@shumai/db'
|
||||
import { setupTestDbHooks } from '@shumai/db/test'
|
||||
import { describe, expect, test } from 'vitest'
|
||||
import AdmZip from 'adm-zip'
|
||||
import { describe, expect, test, vi } from 'vitest'
|
||||
import { runnerService } from '../runner/runner-service'
|
||||
import { s3Service } from '../s3/s3'
|
||||
import { AgentService } from './agent'
|
||||
import { appendSessionEntries } from './session-test-utils'
|
||||
|
||||
@@ -258,6 +261,109 @@ describe('AgentService', () => {
|
||||
expect(updated.user.image).toBe('files/01JK23456789ABCDEF01234567')
|
||||
})
|
||||
|
||||
test('Create and update Agent with runnerId and reset to null when runner is deleted', async () => {
|
||||
const db = prisma
|
||||
const svc = new AgentService()
|
||||
const { team, provider, model } = await setupTestData(db)
|
||||
|
||||
const { runner } = await runnerService.createRunner(team.id, { name: 'Bound Runner' })
|
||||
const otherTeam = await db.team.create({ data: { name: 'Other Team' } })
|
||||
const { runner: otherRunner } = await runnerService.createRunner(otherTeam.id, {
|
||||
name: 'Foreign Runner',
|
||||
})
|
||||
|
||||
// Rejects runner from another team
|
||||
await expect(
|
||||
svc.createAgent({
|
||||
teamId: team.id,
|
||||
name: 'Bad Runner Agent',
|
||||
type: 'chat',
|
||||
enabled: true,
|
||||
providerId: provider.id,
|
||||
modelId: model.id,
|
||||
runnerId: otherRunner.id,
|
||||
thinkingLevel: 'off',
|
||||
}),
|
||||
).rejects.toThrow('runner not found in team')
|
||||
|
||||
const agent = await svc.createAgent({
|
||||
teamId: team.id,
|
||||
name: 'Runner Agent',
|
||||
type: 'chat',
|
||||
enabled: true,
|
||||
providerId: provider.id,
|
||||
modelId: model.id,
|
||||
runnerId: runner.id,
|
||||
thinkingLevel: 'off',
|
||||
})
|
||||
expect(agent?.runnerId).toBe(runner.id)
|
||||
|
||||
// Clear runnerId back to null (use team default)
|
||||
const cleared = await svc.updateAgent({
|
||||
agentId: agent!.id,
|
||||
name: 'Runner Agent',
|
||||
type: 'chat',
|
||||
enabled: true,
|
||||
providerId: provider.id,
|
||||
modelId: model.id,
|
||||
runnerId: null,
|
||||
thinkingLevel: 'off',
|
||||
})
|
||||
expect(cleared.runnerId).toBeNull()
|
||||
|
||||
// Bind again and delete runner -> onDelete: SetNull sets agent.runnerId to null
|
||||
await svc.updateAgent({
|
||||
agentId: agent!.id,
|
||||
name: 'Runner Agent',
|
||||
type: 'chat',
|
||||
enabled: true,
|
||||
providerId: provider.id,
|
||||
modelId: model.id,
|
||||
runnerId: runner.id,
|
||||
thinkingLevel: 'off',
|
||||
})
|
||||
await runnerService.deleteRunner(team.id, runner.id)
|
||||
const afterDelete = await db.agent.findUniqueOrThrow({ where: { id: agent!.id } })
|
||||
expect(afterDelete.runnerId).toBeNull()
|
||||
})
|
||||
|
||||
test('getSkillContent extracts SKILL.md from S3 zip archive without local disk cache', async () => {
|
||||
const db = prisma
|
||||
const svc = new AgentService()
|
||||
const { team } = await setupTestData(db)
|
||||
|
||||
const asset = await db.asset.create({
|
||||
data: {
|
||||
name: 'skill.zip',
|
||||
type: 'file',
|
||||
status: 'uploaded',
|
||||
storageKey: { create: { key: 'skills/zip-skill.zip' } },
|
||||
},
|
||||
})
|
||||
const skill = await db.skill.create({
|
||||
data: {
|
||||
name: 'Zip Skill',
|
||||
teamId: team.id,
|
||||
assetId: asset.id,
|
||||
hash: 'zip-hash',
|
||||
},
|
||||
})
|
||||
|
||||
const zip = new AdmZip()
|
||||
zip.addFile('sub/SKILL.md', Buffer.from('# Zip Skill Docs', 'utf-8'))
|
||||
const getObjectSpy = vi.spyOn(s3Service, 'getObject').mockResolvedValue({
|
||||
buffer: zip.toBuffer(),
|
||||
contentType: 'application/zip',
|
||||
})
|
||||
|
||||
try {
|
||||
const content = await svc.getSkillContent(skill.id)
|
||||
expect(content).toBe('# Zip Skill Docs')
|
||||
} finally {
|
||||
getObjectSpy.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
test('Update Agent preserves deniedTools when omitted, and allows explicit empty array', async () => {
|
||||
const db = prisma
|
||||
const svc = new AgentService()
|
||||
|
||||
@@ -19,8 +19,6 @@ import {
|
||||
UpdateAgentParams,
|
||||
} from '@shumai/dtos'
|
||||
import AdmZip from 'adm-zip'
|
||||
import * as fs from 'fs'
|
||||
import * as path from 'path'
|
||||
import { getAllowedAgentRoles } from './permissions'
|
||||
import { getPresetAvatarBuffer, getPresetAvatarStorageKey, isPresetAvatarId } from './presets'
|
||||
import { agentSessionService } from './session-service'
|
||||
@@ -41,6 +39,7 @@ interface AgentInfoSource {
|
||||
permission: TeamMemberRole
|
||||
providerId: string | null
|
||||
modelId: string | null
|
||||
runnerId?: string | null
|
||||
config: unknown
|
||||
soul: string | null
|
||||
user: { name: string; image: string | null }
|
||||
@@ -57,6 +56,8 @@ function extractS3Key(urlOrKey?: string | null): string | null | undefined {
|
||||
}
|
||||
|
||||
export class AgentService {
|
||||
private skillContentCache = new Map<string, string>()
|
||||
|
||||
constructor(private readonly prismaClient: typeof prisma = prisma) {}
|
||||
|
||||
async getSkillContent(skillId: string): Promise<string> {
|
||||
@@ -68,54 +69,44 @@ export class AgentService {
|
||||
throw new Error(`Skill with ID ${skillId} not found.`)
|
||||
}
|
||||
|
||||
const skillDir = path.join(process.cwd(), '.pi', 'skills', skill.id)
|
||||
const hashFile = path.join(skillDir, '.hash')
|
||||
let needsDownload = true
|
||||
|
||||
if (fs.existsSync(hashFile)) {
|
||||
const localHash = fs.readFileSync(hashFile, 'utf8')
|
||||
if (localHash === skill.hash) {
|
||||
needsDownload = false
|
||||
}
|
||||
const cacheKey = `${skill.id}:${skill.hash}`
|
||||
const cached = this.skillContentCache.get(cacheKey)
|
||||
if (cached !== undefined) {
|
||||
return cached
|
||||
}
|
||||
|
||||
if (needsDownload) {
|
||||
if (fs.existsSync(skillDir)) {
|
||||
fs.rmSync(skillDir, { recursive: true, force: true })
|
||||
}
|
||||
fs.mkdirSync(skillDir, { recursive: true })
|
||||
const asset = await this.prismaClient.asset.findUnique({
|
||||
where: { id: skill.assetId },
|
||||
include: { storageKey: true },
|
||||
})
|
||||
|
||||
const asset = await this.prismaClient.asset.findUnique({
|
||||
where: { id: skill.assetId },
|
||||
include: { storageKey: true },
|
||||
})
|
||||
if (!asset || !asset.storageKey?.key) {
|
||||
throw new Error('Skill asset not found or has no key')
|
||||
}
|
||||
|
||||
if (!asset || !asset.storageKey?.key) {
|
||||
throw new Error('Skill asset not found or has no key')
|
||||
}
|
||||
const { buffer: zipBuffer } = await s3Service.getObject(
|
||||
process.env.S3_BUCKET || 'shumai',
|
||||
asset.storageKey.key,
|
||||
)
|
||||
|
||||
const { buffer: zipBuffer } = await s3Service.getObject(
|
||||
process.env.S3_BUCKET || 'shumai',
|
||||
asset.storageKey.key,
|
||||
const zip = new AdmZip(zipBuffer)
|
||||
const entries = zip.getEntries()
|
||||
const skillMdEntry =
|
||||
entries.find((e) => !e.isDirectory && e.entryName === 'SKILL.md') ||
|
||||
entries.find((e) => !e.isDirectory && e.entryName === 'skill.md') ||
|
||||
entries.find(
|
||||
(e) =>
|
||||
!e.isDirectory &&
|
||||
(e.entryName.endsWith('/SKILL.md') || e.entryName.endsWith('/skill.md')),
|
||||
)
|
||||
|
||||
const zip = new AdmZip(zipBuffer)
|
||||
zip.extractAllTo(skillDir, true)
|
||||
|
||||
fs.writeFileSync(hashFile, skill.hash)
|
||||
if (!skillMdEntry) {
|
||||
throw new Error(`Skill downloaded but SKILL.md not found in skill ${skill.id}`)
|
||||
}
|
||||
|
||||
// Read SKILL.md
|
||||
let skillMdPath = path.join(skillDir, 'SKILL.md')
|
||||
if (!fs.existsSync(skillMdPath)) {
|
||||
skillMdPath = path.join(skillDir, 'skill.md')
|
||||
}
|
||||
|
||||
if (!fs.existsSync(skillMdPath)) {
|
||||
throw new Error(`Skill downloaded but SKILL.md not found in ${skillDir}`)
|
||||
}
|
||||
|
||||
return fs.readFileSync(skillMdPath, 'utf8')
|
||||
const content = skillMdEntry.getData().toString('utf8')
|
||||
this.skillContentCache.set(cacheKey, content)
|
||||
return content
|
||||
}
|
||||
|
||||
async getSkillEnvs(skillId: string): Promise<Record<string, string>> {
|
||||
@@ -150,6 +141,7 @@ export class AgentService {
|
||||
avatar,
|
||||
providerId,
|
||||
modelId,
|
||||
runnerId,
|
||||
thinkingLevel,
|
||||
systemPrompt,
|
||||
soul,
|
||||
@@ -206,6 +198,15 @@ export class AgentService {
|
||||
})
|
||||
if (!team) throw new Error('team not found')
|
||||
|
||||
if (runnerId) {
|
||||
const runner = await tx.runner.findFirst({
|
||||
where: { id: runnerId, teamId },
|
||||
})
|
||||
if (!runner) {
|
||||
throw new Error('runner not found in team')
|
||||
}
|
||||
}
|
||||
|
||||
const agentConfig: PrismaJson.AgentConfig = {
|
||||
provider: providerId || '',
|
||||
model: modelId || '',
|
||||
@@ -233,6 +234,7 @@ export class AgentService {
|
||||
permission: permission || 'reviewer',
|
||||
providerId: providerId || null,
|
||||
modelId: modelId || null,
|
||||
runnerId: runnerId || null,
|
||||
soul,
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
config: agentConfig as any,
|
||||
@@ -280,6 +282,7 @@ export class AgentService {
|
||||
avatar,
|
||||
providerId,
|
||||
modelId,
|
||||
runnerId,
|
||||
thinkingLevel,
|
||||
systemPrompt,
|
||||
soul,
|
||||
@@ -380,6 +383,15 @@ export class AgentService {
|
||||
where: { agentId },
|
||||
})
|
||||
|
||||
if (runnerId) {
|
||||
const runner = await tx.runner.findFirst({
|
||||
where: { id: runnerId, teamId: agent.teamId },
|
||||
})
|
||||
if (!runner) {
|
||||
throw new Error('runner not found in team')
|
||||
}
|
||||
}
|
||||
|
||||
return tx.agent.update({
|
||||
where: { id: agentId },
|
||||
data: {
|
||||
@@ -388,6 +400,7 @@ export class AgentService {
|
||||
...(permission ? { permission } : {}),
|
||||
providerId: providerId || null,
|
||||
modelId: modelId || null,
|
||||
...(runnerId !== undefined ? { runnerId: runnerId || null } : {}),
|
||||
soul,
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
config: agentConfig as any,
|
||||
@@ -478,6 +491,7 @@ export class AgentService {
|
||||
avatarPreset: config.avatarPreset,
|
||||
providerId: agent.providerId || undefined,
|
||||
modelId: agent.modelId || undefined,
|
||||
runnerId: agent.runnerId ?? null,
|
||||
thinkingLevel: config.thinkingLevel || 'off',
|
||||
systemPrompt: config.systemPrompt,
|
||||
soul: agent.soul || undefined,
|
||||
|
||||
@@ -8,6 +8,7 @@ import { s3Service, getStorageBackend } from '@shumai/core/src/s3/s3'
|
||||
import * as archiveStorageModule from '@shumai/core/src/s3/archive-storage'
|
||||
import { uploadService } from '@shumai/core/src/upload/upload'
|
||||
import { metadataService } from '@shumai/core/src/metadata/metadata'
|
||||
import { watermarkService } from '@shumai/core/src/watermark/watermark'
|
||||
import { AssetService } from './asset'
|
||||
|
||||
import { ensureJpegInStorage } from '@shumai/core/src/s3/preview-converter'
|
||||
@@ -22,6 +23,7 @@ vi.mock('@shumai/core/src/s3/s3', () => ({
|
||||
deletePrefix: vi.fn().mockResolvedValue(1),
|
||||
copyObject: vi.fn().mockResolvedValue(undefined),
|
||||
getStoragePath: vi.fn((bucket: string, key: string) => `/mock/data/${bucket}/${key}`),
|
||||
listObjects: vi.fn().mockResolvedValue([]),
|
||||
},
|
||||
getStorageBackend: vi.fn().mockReturnValue('local'),
|
||||
}))
|
||||
@@ -47,6 +49,31 @@ describe('AssetService', () => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
it('checks generated file expiration hourly while continuing regular cleanup', async () => {
|
||||
vi.useFakeTimers({ toFake: ['Date', 'setTimeout', 'clearTimeout'] })
|
||||
const startedAt = Date.now()
|
||||
const regularCleanup = vi
|
||||
.spyOn(watermarkService, 'purgeOrphanWatermarkConfigs')
|
||||
.mockResolvedValue(0)
|
||||
try {
|
||||
await assetService.startCleanupJob()
|
||||
await vi.waitFor(() => expect(s3Service.listObjects).toHaveBeenCalledTimes(1))
|
||||
expect(s3Service.listObjects).toHaveBeenCalledWith(expect.any(String), 'temp/generated/')
|
||||
|
||||
await vi.advanceTimersByTimeAsync(5_000)
|
||||
await vi.waitFor(() => expect(regularCleanup).toHaveBeenCalledTimes(2))
|
||||
expect(s3Service.listObjects).toHaveBeenCalledTimes(1)
|
||||
|
||||
vi.setSystemTime(startedAt + 60 * 60 * 1000)
|
||||
await vi.advanceTimersByTimeAsync(5_000)
|
||||
await vi.waitFor(() => expect(s3Service.listObjects).toHaveBeenCalledTimes(2))
|
||||
} finally {
|
||||
assetService.stopCleanupJob()
|
||||
regularCleanup.mockRestore()
|
||||
vi.useRealTimers()
|
||||
}
|
||||
})
|
||||
|
||||
const setupBasicAssets = async () => {
|
||||
const user = await prisma.user.create({
|
||||
data: {
|
||||
|
||||
@@ -66,6 +66,7 @@ import { resolveEffectiveRole } from '@shumai/core/src/authz/authz'
|
||||
import { metadataService } from '@shumai/core/src/metadata/metadata'
|
||||
import { uploadService } from '@shumai/core/src/upload/upload'
|
||||
import path from 'path'
|
||||
import { decodeTime } from 'ulid'
|
||||
import { stemFromKey } from '@shumai/core/src/utils/filename'
|
||||
import { rewriteM3u8WithPresignedUrls } from '@shumai/core/src/transcode/transcode'
|
||||
|
||||
@@ -1584,7 +1585,9 @@ export class AssetService {
|
||||
private cleanupJobsRunning = false
|
||||
|
||||
async startCleanupJob() {
|
||||
if (this.cleanupJobsRunning) return
|
||||
this.cleanupJobsRunning = true
|
||||
let nextGeneratedCleanupAt = 0
|
||||
|
||||
const runCleanup = async () => {
|
||||
if (!this.cleanupJobsRunning) return
|
||||
@@ -1594,6 +1597,11 @@ export class AssetService {
|
||||
await this.purgePendingAssets()
|
||||
await this.purgeUnreferencedStorageKeys()
|
||||
await watermarkService.purgeOrphanWatermarkConfigs()
|
||||
const now = Date.now()
|
||||
if (now >= nextGeneratedCleanupAt) {
|
||||
nextGeneratedCleanupAt = now + 60 * 60 * 1000
|
||||
await this.purgeExpiredGeneratedFiles()
|
||||
}
|
||||
} catch (e: unknown) {
|
||||
console.error('Error in asset cleanup job:', e)
|
||||
}
|
||||
@@ -1773,6 +1781,37 @@ export class AssetService {
|
||||
}
|
||||
}
|
||||
|
||||
private async purgeExpiredGeneratedFiles(retentionMs = 24 * 60 * 60 * 1000) {
|
||||
const bucket = process.env.S3_BUCKET || 'shumai'
|
||||
const keys = await s3Service.listObjects(bucket, 'temp/generated/')
|
||||
if (keys.length === 0) return
|
||||
|
||||
const now = Date.now()
|
||||
for (const key of keys) {
|
||||
try {
|
||||
const parts = key.split('/')
|
||||
const idSegment = parts[2]
|
||||
let createdAtMs: number | null = null
|
||||
if (idSegment && /^[0-9A-HJKMNP-TV-Z]{26}$/i.test(idSegment)) {
|
||||
try {
|
||||
createdAtMs = decodeTime(idSegment.toUpperCase())
|
||||
} catch {
|
||||
createdAtMs = null
|
||||
}
|
||||
}
|
||||
if (createdAtMs === null) {
|
||||
const info = await s3Service.headObject(bucket, key)
|
||||
createdAtMs = info.lastModified.getTime()
|
||||
}
|
||||
if (now - createdAtMs > retentionMs) {
|
||||
await s3Service.deleteObject(bucket, key)
|
||||
}
|
||||
} catch (err) {
|
||||
logger.warn({ err, key }, 'Failed to purge expired generated file')
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async restoreAssets(ids: string[]): Promise<void> {
|
||||
for (const id of ids) {
|
||||
await this.prismaClient.$transaction(async (tx) => {
|
||||
|
||||
@@ -20,7 +20,6 @@ describe('AssetService - Recently Viewed', () => {
|
||||
settings: {
|
||||
transcode: { videoStrategy: 'best_match' },
|
||||
},
|
||||
sandbox: { create: {} },
|
||||
},
|
||||
})
|
||||
await prisma.teamMember.create({
|
||||
|
||||
@@ -20,43 +20,46 @@ describe('Auth Signup Locale Hook', () => {
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
it('should save locale metadata for the user upon signup', async () => {
|
||||
const email = `signup-locale-${Date.now()}@example.com`
|
||||
it.each(['zh', 'ko'])(
|
||||
'should save %s locale metadata for the user upon signup',
|
||||
async (locale) => {
|
||||
const email = `signup-locale-${locale}-${Date.now()}@example.com`
|
||||
|
||||
// We must use any here because 'locale' is a custom property we extract in
|
||||
// our auth hooks, but better-auth's signUpEmail typescript interface does
|
||||
// not recognize it.
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const res = await (auth.api.signUpEmail as any)({
|
||||
headers: new Headers(),
|
||||
body: {
|
||||
email,
|
||||
password: 'testpassword',
|
||||
name: 'Locale Tester',
|
||||
locale: 'zh',
|
||||
},
|
||||
})
|
||||
// We must use any here because 'locale' is a custom property we extract in
|
||||
// our auth hooks, but better-auth's signUpEmail typescript interface does
|
||||
// not recognize it.
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const res = await (auth.api.signUpEmail as any)({
|
||||
headers: new Headers(),
|
||||
body: {
|
||||
email,
|
||||
password: 'testpassword',
|
||||
name: 'Locale Tester',
|
||||
locale,
|
||||
},
|
||||
})
|
||||
|
||||
if (res.error) {
|
||||
console.error('SIGNUP ERROR:', res.error)
|
||||
}
|
||||
if (res.error) {
|
||||
console.error('SIGNUP ERROR:', res.error)
|
||||
}
|
||||
|
||||
expect(res).toBeDefined()
|
||||
expect(res.user).toBeDefined()
|
||||
expect(res.user.email).toBe(email)
|
||||
expect(res).toBeDefined()
|
||||
expect(res.user).toBeDefined()
|
||||
expect(res.user.email).toBe(email)
|
||||
|
||||
// Check that the userMetadata record was created with key 'locale' and value 'zh'
|
||||
const teamMembers = await prisma.teamMember.findMany({
|
||||
where: { userId: res.user.id },
|
||||
})
|
||||
expect(teamMembers.length).toBeGreaterThan(0)
|
||||
// Check that the userMetadata record was created with key 'locale' and the requested value
|
||||
const teamMembers = await prisma.teamMember.findMany({
|
||||
where: { userId: res.user.id },
|
||||
})
|
||||
expect(teamMembers.length).toBeGreaterThan(0)
|
||||
|
||||
for (const member of teamMembers) {
|
||||
const meta = await userMetadataService.getMetadata(res.user.id, member.teamId, 'locale')
|
||||
expect(meta).toBeDefined()
|
||||
expect(meta?.value).toBe('zh')
|
||||
}
|
||||
})
|
||||
for (const member of teamMembers) {
|
||||
const meta = await userMetadataService.getMetadata(res.user.id, member.teamId, 'locale')
|
||||
expect(meta).toBeDefined()
|
||||
expect(meta?.value).toBe(locale)
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
it('should not save locale metadata if locale is invalid or missing', async () => {
|
||||
const email = `signup-no-locale-${Date.now()}@example.com`
|
||||
|
||||
@@ -201,7 +201,7 @@ export const auth = betterAuth({
|
||||
}
|
||||
|
||||
const locale = body?.locale
|
||||
if (locale === 'en' || locale === 'zh') {
|
||||
if (locale === 'en' || locale === 'zh' || locale === 'ko') {
|
||||
const joinedTeams = await prisma.teamMember.findMany({
|
||||
where: { userId: user.id },
|
||||
select: { teamId: true },
|
||||
|
||||
@@ -18,7 +18,6 @@ describe('ChatService', () => {
|
||||
data: {
|
||||
name: 'Test Team',
|
||||
settings: { transcode: { videoStrategy: 'best_match' }, chatbotAgentId: 'test-agent-id' },
|
||||
sandbox: { create: {} },
|
||||
},
|
||||
})
|
||||
|
||||
@@ -537,7 +536,6 @@ describe('ChatService', () => {
|
||||
data: {
|
||||
name: 'Team Roles',
|
||||
settings: { transcode: { videoStrategy: 'best_match' }, chatbotAgentId: 'test-agent-id' },
|
||||
sandbox: { create: {} },
|
||||
},
|
||||
})
|
||||
const project = await prisma.project.create({
|
||||
|
||||
@@ -17,7 +17,6 @@ describe('CollectionService', () => {
|
||||
settings: {
|
||||
transcode: { videoStrategy: 'best_match' },
|
||||
},
|
||||
sandbox: { create: {} },
|
||||
},
|
||||
})
|
||||
await prisma.teamMember.create({
|
||||
|
||||
@@ -33,7 +33,7 @@ export * from './mcp/mcp-service'
|
||||
export * from './quota/quota-service'
|
||||
export * from './kanban/kanban'
|
||||
export * from './media-generation/media-generation'
|
||||
export * from './sandbox'
|
||||
export * from './runner'
|
||||
export * from './asset/comments-export'
|
||||
export * from './turntable/turntable'
|
||||
export * from './utils/model3d'
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
export * from './types'
|
||||
export * from './runner-service'
|
||||
@@ -0,0 +1,859 @@
|
||||
import { prisma } from '@shumai/db'
|
||||
import * as advisoryLocks from '@shumai/db/src/advisory-lock'
|
||||
import { setupTestDbHooks } from '@shumai/db/test'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { s3Service } from '../s3/s3'
|
||||
import {
|
||||
RunnerService,
|
||||
generateRunnerKey,
|
||||
hashRunnerKey,
|
||||
validateRunnerWorkspacePath,
|
||||
} from './runner-service'
|
||||
import type { RunnerSocket } from './types'
|
||||
|
||||
function createMockWs() {
|
||||
const sent: string[] = []
|
||||
let closed: { code?: number; reason?: string } | null = null
|
||||
const ws: RunnerSocket = {
|
||||
send: vi.fn((data: string) => {
|
||||
sent.push(data)
|
||||
}),
|
||||
close: vi.fn((code?: number, reason?: string) => {
|
||||
closed = { code, reason }
|
||||
}),
|
||||
}
|
||||
return {
|
||||
ws,
|
||||
sent,
|
||||
getClosed: () => closed,
|
||||
lastSentJson: () => {
|
||||
const last = sent[sent.length - 1]
|
||||
return last ? JSON.parse(last) : null
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
function reconcile(
|
||||
service: RunnerService,
|
||||
runnerId: string,
|
||||
ws: RunnerSocket,
|
||||
activeIds: string[] = [],
|
||||
) {
|
||||
service.handleMessage(
|
||||
runnerId,
|
||||
ws,
|
||||
JSON.stringify({
|
||||
type: 'reconcile',
|
||||
environment: { platform: 'linux', shell: '/bin/sh' },
|
||||
activeExecutions: activeIds.map((executionId) => ({ executionId, nextOffset: 0 })),
|
||||
completedExecutions: [],
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
describe('RunnerService', () => {
|
||||
setupTestDbHooks()
|
||||
|
||||
let service: RunnerService
|
||||
let teamId: string
|
||||
|
||||
beforeEach(async () => {
|
||||
service = new RunnerService()
|
||||
const user = await prisma.user.create({
|
||||
data: { name: 'Runner Owner', email: `runner-${Date.now()}@example.com`, password: 'pw' },
|
||||
})
|
||||
const team = await prisma.team.create({
|
||||
data: { name: 'Runner Test Team', members: { create: { userId: user.id, role: 'owner' } } },
|
||||
})
|
||||
teamId = team.id
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
service.stopHeartbeatMonitor()
|
||||
await service.stopServerTopology()
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
it('allows replicas without runners and enforces a single server while runners exist', async () => {
|
||||
const replica = new RunnerService()
|
||||
try {
|
||||
await service.startServerTopology()
|
||||
await replica.startServerTopology()
|
||||
await expect(service.createRunner(teamId, { name: 'Blocked' })).rejects.toMatchObject({
|
||||
status: 409,
|
||||
message: expect.stringContaining('single main Shumai server'),
|
||||
})
|
||||
expect(await service.listRunners(teamId)).toEqual([])
|
||||
|
||||
await replica.stopServerTopology()
|
||||
const { runner } = await service.createRunner(teamId, { name: 'Exclusive runner' })
|
||||
await expect(replica.startServerTopology()).rejects.toThrow('single main Shumai server')
|
||||
|
||||
await service.stopServerTopology()
|
||||
await replica.startServerTopology()
|
||||
await expect(service.startServerTopology()).rejects.toThrow('single main Shumai server')
|
||||
|
||||
const acquireLease = advisoryLocks.tryAcquireUpgradeableAdvisoryLock
|
||||
vi.spyOn(advisoryLocks, 'tryAcquireUpgradeableAdvisoryLock').mockImplementationOnce(
|
||||
async (lockKey, exclusive) => {
|
||||
// The newcomer has already looked up the existing runner.
|
||||
await replica.deleteRunner(teamId, runner.id)
|
||||
return acquireLease(lockKey, exclusive)
|
||||
},
|
||||
)
|
||||
await service.startServerTopology()
|
||||
await expect(replica.createRunner(teamId, { name: 'Blocked again' })).rejects.toMatchObject({
|
||||
status: 409,
|
||||
})
|
||||
} finally {
|
||||
await replica.stopServerTopology()
|
||||
}
|
||||
})
|
||||
|
||||
describe('Key & workspace path helpers', () => {
|
||||
it('keeps validated OS and shell metadata only for the current connection', async () => {
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'Windows runner' })
|
||||
const connection = createMockWs()
|
||||
const authenticated = {
|
||||
id: runner.id,
|
||||
teamId,
|
||||
name: runner.name,
|
||||
keyHash: hashRunnerKey(key),
|
||||
}
|
||||
expect(service.getRunnerEnvironment(runner.id)).toBeUndefined()
|
||||
await service.registerConnection(authenticated, connection.ws)
|
||||
const environment = { platform: 'win32', shell: 'C:\\Windows\\System32\\cmd.exe' }
|
||||
const reconcile = {
|
||||
type: 'reconcile',
|
||||
environment,
|
||||
activeExecutions: [],
|
||||
completedExecutions: [],
|
||||
}
|
||||
service.handleMessage(runner.id, connection.ws, JSON.stringify(reconcile))
|
||||
expect(service.getRunnerEnvironment(runner.id)).toEqual(environment)
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
connection.ws,
|
||||
JSON.stringify({ ...reconcile, environment: { platform: 'invalid', shell: '' } }),
|
||||
)
|
||||
expect(service.getRunnerEnvironment(runner.id)).toEqual(environment)
|
||||
const replacement = createMockWs()
|
||||
await service.registerConnection(authenticated, replacement.ws)
|
||||
service.handleMessage(runner.id, connection.ws, JSON.stringify(reconcile))
|
||||
expect(service.getRunnerEnvironment(runner.id)).toBeUndefined()
|
||||
service.handleMessage(runner.id, replacement.ws, JSON.stringify(reconcile))
|
||||
service.handleDisconnect(runner.id, replacement.ws)
|
||||
expect(service.getRunnerEnvironment(runner.id)).toBeUndefined()
|
||||
})
|
||||
|
||||
it('generates and hashes runner keys', () => {
|
||||
const { key, keyPrefix, keyHash } = generateRunnerKey()
|
||||
expect(key).toMatch(/^shm_rn_[0-9a-f]{48}$/)
|
||||
expect(keyPrefix).toBe(key.slice(0, 14))
|
||||
expect(keyHash).toBe(hashRunnerKey(key))
|
||||
})
|
||||
|
||||
it('validates runner workspace paths strictly under .pi/', () => {
|
||||
expect(validateRunnerWorkspacePath('.pi/file.txt')).toBe('.pi/file.txt')
|
||||
expect(validateRunnerWorkspacePath('.pi/skills/s1')).toBe('.pi/skills/s1')
|
||||
expect(() => validateRunnerWorkspacePath('/etc/passwd')).toThrow(/must be relative/)
|
||||
expect(() => validateRunnerWorkspacePath('.pi/../../etc/passwd')).toThrow(
|
||||
/must stay within \.pi\//,
|
||||
)
|
||||
expect(() => validateRunnerWorkspacePath('outside.txt')).toThrow(/must stay within \.pi\//)
|
||||
})
|
||||
})
|
||||
|
||||
describe('CRUD & Default Runner Invariant', () => {
|
||||
it('auto-promotes first runner to default and preserves invariant across CRUD', async () => {
|
||||
const resolvedEmpty = await service.resolveRunnerForAgent(teamId)
|
||||
expect(resolvedEmpty.hasConfiguredRunner).toBe(false)
|
||||
|
||||
const r1 = await service.createRunner(teamId, { name: 'Mac Mini 1' })
|
||||
expect(r1.runner.isDefault).toBe(true)
|
||||
expect(r1.runner.status).toBe('disconnected')
|
||||
expect(r1.key).toMatch(/^shm_rn_[0-9a-f]{48}$/)
|
||||
|
||||
// Creating second runner without isDefault keeps first as default
|
||||
const r2 = await service.createRunner(teamId, { name: 'Mac Mini 2' })
|
||||
expect(r2.runner.isDefault).toBe(false)
|
||||
|
||||
let list = await service.listRunners(teamId)
|
||||
expect(list).toHaveLength(2)
|
||||
expect(list.find((r) => r.id === r1.runner.id)?.isDefault).toBe(true)
|
||||
expect(list.find((r) => r.id === r2.runner.id)?.isDefault).toBe(false)
|
||||
|
||||
// Promoting r2 demotes r1
|
||||
const updatedR2 = await service.updateRunner(teamId, r2.runner.id, {
|
||||
name: 'Mac Mini 2 Pro',
|
||||
isDefault: true,
|
||||
})
|
||||
expect(updatedR2.name).toBe('Mac Mini 2 Pro')
|
||||
expect(updatedR2.isDefault).toBe(true)
|
||||
|
||||
list = await service.listRunners(teamId)
|
||||
expect(list.find((r) => r.id === r1.runner.id)?.isDefault).toBe(false)
|
||||
expect(list.find((r) => r.id === r2.runner.id)?.isDefault).toBe(true)
|
||||
|
||||
// Deleting the default runner (r2) promotes the remaining runner (r1)
|
||||
await service.deleteRunner(teamId, r2.runner.id)
|
||||
list = await service.listRunners(teamId)
|
||||
expect(list).toHaveLength(1)
|
||||
expect(list[0].id).toBe(r1.runner.id)
|
||||
expect(list[0].isDefault).toBe(true)
|
||||
})
|
||||
|
||||
it('preserves a concurrent default promotion during name-only updates and enforces single default in DB', async () => {
|
||||
const older = await service.createRunner(teamId, { name: 'Older Runner' })
|
||||
const newer = await service.createRunner(teamId, { name: 'Newer Default', isDefault: true })
|
||||
|
||||
// Simulate rename-only update alongside promoting older runner
|
||||
await service.updateRunner(teamId, older.runner.id, { isDefault: true })
|
||||
const renamedNewer = await service.updateRunner(teamId, newer.runner.id, {
|
||||
name: 'Newer Default Renamed',
|
||||
})
|
||||
expect(renamedNewer.isDefault).toBe(false)
|
||||
|
||||
// Concurrent rename + promote should serialize cleanly and never produce two defaults
|
||||
await Promise.all([
|
||||
service.updateRunner(teamId, older.runner.id, { name: 'Older Renamed' }),
|
||||
service.updateRunner(teamId, newer.runner.id, { isDefault: true }),
|
||||
])
|
||||
|
||||
const list = await service.listRunners(teamId)
|
||||
expect(list.filter((r) => r.isDefault)).toHaveLength(1)
|
||||
expect(list.find((r) => r.id === newer.runner.id)?.isDefault).toBe(true)
|
||||
expect(list.find((r) => r.id === older.runner.id)?.name).toBe('Older Renamed')
|
||||
|
||||
// Partial unique index enforces single default even on direct DB writes
|
||||
await expect(
|
||||
prisma.runner.update({
|
||||
where: { id: older.runner.id },
|
||||
data: { isDefault: true },
|
||||
}),
|
||||
).rejects.toThrow()
|
||||
})
|
||||
|
||||
it('authenticates runner keys and invalidates old key on rotation', async () => {
|
||||
const created = await service.createRunner(teamId, { name: 'Runner A' })
|
||||
const verified = await service.authenticateRunnerKey(created.key)
|
||||
expect(verified).toEqual({
|
||||
id: created.runner.id,
|
||||
teamId,
|
||||
name: 'Runner A',
|
||||
keyHash: hashRunnerKey(created.key),
|
||||
})
|
||||
|
||||
const rotated = await service.rotateRunnerKey(teamId, created.runner.id)
|
||||
expect(rotated.key).not.toBe(created.key)
|
||||
expect(await service.authenticateRunnerKey(created.key)).toBeNull()
|
||||
expect(await service.authenticateRunnerKey(rotated.key)).toEqual({
|
||||
id: created.runner.id,
|
||||
teamId,
|
||||
name: 'Runner A',
|
||||
keyHash: hashRunnerKey(rotated.key),
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('WebSocket connection lifecycle & eviction', () => {
|
||||
it('waits for reconciliation before reporting online or dispatching a command', async () => {
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'Handshaking runner' })
|
||||
const conn = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
|
||||
conn.ws,
|
||||
)
|
||||
|
||||
expect(conn.lastSentJson()).toHaveProperty('type', 'welcome')
|
||||
expect(service.isRunnerConnected(runner.id)).toBe(false)
|
||||
await expect(service.exec(runner.id, { command: 'echo ready' })).rejects.toThrow(
|
||||
'Runner is not connected',
|
||||
)
|
||||
expect(conn.sent).toHaveLength(1)
|
||||
|
||||
reconcile(service, runner.id, conn.ws)
|
||||
expect(service.isRunnerConnected(runner.id)).toBe(true)
|
||||
})
|
||||
|
||||
it('evicts existing connection with 4000 when a new connection arrives', async () => {
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'Runner WS' })
|
||||
const keyHash = hashRunnerKey(key)
|
||||
const conn1 = createMockWs()
|
||||
const conn2 = createMockWs()
|
||||
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash },
|
||||
conn1.ws,
|
||||
)
|
||||
reconcile(service, runner.id, conn1.ws)
|
||||
expect(service.isRunnerConnected(runner.id)).toBe(true)
|
||||
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash },
|
||||
conn2.ws,
|
||||
)
|
||||
expect(conn1.getClosed()).toEqual({
|
||||
code: 4000,
|
||||
reason: 'Replaced by a newer connection',
|
||||
})
|
||||
reconcile(service, runner.id, conn2.ws)
|
||||
expect(service.isRunnerConnected(runner.id)).toBe(true)
|
||||
})
|
||||
|
||||
it('closes active connection with 1008 when key is rotated or runner is deleted', async () => {
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'Runner Rotate' })
|
||||
const conn1 = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
|
||||
conn1.ws,
|
||||
)
|
||||
|
||||
const rotated = await service.rotateRunnerKey(teamId, runner.id)
|
||||
expect(conn1.getClosed()?.code).toBe(1008)
|
||||
expect(service.isRunnerConnected(runner.id)).toBe(false)
|
||||
|
||||
const conn2 = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(rotated.key) },
|
||||
conn2.ws,
|
||||
)
|
||||
await service.deleteRunner(teamId, runner.id)
|
||||
expect(conn2.getClosed()?.code).toBe(1008)
|
||||
})
|
||||
|
||||
it('rejects outstanding handshake if key was rotated or runner deleted before open()', async () => {
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'Runner Race' })
|
||||
const authenticated = await service.authenticateRunnerKey(key)
|
||||
expect(authenticated).not.toBeNull()
|
||||
|
||||
// Key rotates before registerConnection finishes
|
||||
await service.rotateRunnerKey(teamId, runner.id)
|
||||
const staleConn = createMockWs()
|
||||
await service.registerConnection(authenticated!, staleConn.ws)
|
||||
|
||||
expect(staleConn.getClosed()?.code).toBe(1008)
|
||||
expect(service.isRunnerConnected(runner.id)).toBe(false)
|
||||
expect(staleConn.sent).toHaveLength(0)
|
||||
|
||||
// Deleting runner after authentication also blocks registration
|
||||
const { runner: rDel, key: kDel } = await service.createRunner(teamId, { name: 'Runner Del' })
|
||||
const authedDel = await service.authenticateRunnerKey(kDel)
|
||||
await service.deleteRunner(teamId, rDel.id)
|
||||
|
||||
const delConn = createMockWs()
|
||||
await service.registerConnection(authedDel!, delConn.ws)
|
||||
expect(delConn.getClosed()?.code).toBe(1008)
|
||||
expect(service.isRunnerConnected(rDel.id)).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('Agent Runner Resolution', () => {
|
||||
it('resolves team default runner when agentRunnerId is null and specific runner when set', async () => {
|
||||
const r1 = await service.createRunner(teamId, { name: 'Default Runner' })
|
||||
const r2 = await service.createRunner(teamId, { name: 'Dedicated GPU Runner' })
|
||||
|
||||
const resolvedDefault = await service.resolveRunnerForAgent(teamId, null)
|
||||
expect(resolvedDefault.runner?.id).toBe(r1.runner.id)
|
||||
expect(resolvedDefault.isBoundToAgent).toBe(false)
|
||||
|
||||
const resolvedPinned = await service.resolveRunnerForAgent(teamId, r2.runner.id)
|
||||
expect(resolvedPinned.runner?.id).toBe(r2.runner.id)
|
||||
expect(resolvedPinned.isBoundToAgent).toBe(true)
|
||||
|
||||
// When offline, requireOnlineRunner throws a clear message naming the runner
|
||||
await expect(service.requireOnlineRunner(teamId, r2.runner.id)).rejects.toThrow(
|
||||
/Dedicated GPU Runner.*currently offline/,
|
||||
)
|
||||
|
||||
const conn2 = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: r2.runner.id, teamId, name: r2.runner.name, keyHash: hashRunnerKey(r2.key) },
|
||||
conn2.ws,
|
||||
)
|
||||
reconcile(service, r2.runner.id, conn2.ws)
|
||||
const online = await service.requireOnlineRunner(teamId, r2.runner.id)
|
||||
expect(online.id).toBe(r2.runner.id)
|
||||
})
|
||||
})
|
||||
|
||||
describe('RPC Execution & File Operations over WebSocket', () => {
|
||||
it('executes bash commands, streams stdout/stderr, and returns exit result', async () => {
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'Exec Runner' })
|
||||
const conn = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
|
||||
conn.ws,
|
||||
)
|
||||
reconcile(service, runner.id, conn.ws)
|
||||
|
||||
const chunks: Array<{ stream: string; text: string }> = []
|
||||
const execPromise = service.exec(runner.id, {
|
||||
command: 'echo hi',
|
||||
env: { FOO: 'bar' },
|
||||
onOutput: (text, stream) => chunks.push({ stream, text }),
|
||||
})
|
||||
|
||||
const startMsg = conn.lastSentJson()
|
||||
expect(startMsg).toMatchObject({
|
||||
type: 'exec.start',
|
||||
command: 'echo hi',
|
||||
env: { FOO: 'bar' },
|
||||
})
|
||||
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.output',
|
||||
executionId: startMsg.executionId,
|
||||
stream: 'stdout',
|
||||
offset: 0,
|
||||
data: 'hello ',
|
||||
}),
|
||||
)
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.output',
|
||||
executionId: startMsg.executionId,
|
||||
stream: 'stderr',
|
||||
offset: 6,
|
||||
data: 'warn',
|
||||
}),
|
||||
)
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.result',
|
||||
executionId: startMsg.executionId,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
fullOutputPath: '.pi/logs/bash-123.log',
|
||||
}),
|
||||
)
|
||||
|
||||
const res = await execPromise
|
||||
expect(res).toMatchObject({
|
||||
executionId: startMsg.executionId,
|
||||
output: 'hello warn',
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
fullOutputPath: '.pi/logs/bash-123.log',
|
||||
})
|
||||
expect(chunks).toEqual([
|
||||
{ stream: 'stdout', text: 'hello ' },
|
||||
{ stream: 'stderr', text: 'warn' },
|
||||
])
|
||||
})
|
||||
|
||||
it('omits retained output when collectOutput is false and bounds tail when maxBufferedOutputBytes is set', async () => {
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'Bounded Runner' })
|
||||
const keyHash = hashRunnerKey(key)
|
||||
const conn = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash },
|
||||
conn.ws,
|
||||
)
|
||||
|
||||
reconcile(service, runner.id, conn.ws)
|
||||
const streamed: string[] = []
|
||||
const noCollectPromise = service.exec(runner.id, {
|
||||
command: 'yes | head -n 100',
|
||||
collectOutput: false,
|
||||
onOutput: (chunk) => streamed.push(chunk),
|
||||
})
|
||||
const msg1 = conn.lastSentJson()
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.output',
|
||||
executionId: msg1.executionId,
|
||||
stream: 'stdout',
|
||||
offset: 0,
|
||||
data: 'a'.repeat(4096),
|
||||
}),
|
||||
)
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.result',
|
||||
executionId: msg1.executionId,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
}),
|
||||
)
|
||||
const noCollectRes = await noCollectPromise
|
||||
expect(noCollectRes.output).toBe('')
|
||||
expect(streamed.join('')).toHaveLength(4096)
|
||||
|
||||
const boundedPromise = service.exec(runner.id, {
|
||||
command: 'echo bounded',
|
||||
maxBufferedOutputBytes: 6,
|
||||
})
|
||||
const msg2 = conn.lastSentJson()
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.output',
|
||||
executionId: msg2.executionId,
|
||||
stream: 'stdout',
|
||||
offset: 0,
|
||||
data: '1234567890',
|
||||
}),
|
||||
)
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.result',
|
||||
executionId: msg2.executionId,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
}),
|
||||
)
|
||||
const boundedRes = await boundedPromise
|
||||
expect(boundedRes.output).toBe('567890')
|
||||
})
|
||||
|
||||
it('sends exec.cancel when AbortSignal fires', async () => {
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'Cancel Runner' })
|
||||
const conn = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
|
||||
conn.ws,
|
||||
)
|
||||
reconcile(service, runner.id, conn.ws)
|
||||
|
||||
const abortController = new AbortController()
|
||||
const execPromise = service.exec(runner.id, {
|
||||
command: 'sleep 30',
|
||||
signal: abortController.signal,
|
||||
})
|
||||
|
||||
const startMsg = conn.lastSentJson()
|
||||
abortController.abort()
|
||||
|
||||
const cancelMsg = conn.lastSentJson()
|
||||
expect(cancelMsg).toEqual({
|
||||
type: 'exec.cancel',
|
||||
executionId: startMsg.executionId,
|
||||
})
|
||||
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.result',
|
||||
executionId: startMsg.executionId,
|
||||
exitCode: null,
|
||||
signal: 'SIGKILL',
|
||||
timedOut: false,
|
||||
aborted: true,
|
||||
}),
|
||||
)
|
||||
|
||||
const res = await execPromise
|
||||
expect(res.aborted).toBe(true)
|
||||
})
|
||||
|
||||
it.each([false, true])(
|
||||
'replays cancellation on reconnect after notifying caller=%s',
|
||||
async (reported) => {
|
||||
const { runner, key } = await service.createRunner(teamId, {
|
||||
name: 'Cancelled while offline',
|
||||
})
|
||||
const authenticated = {
|
||||
id: runner.id,
|
||||
teamId,
|
||||
name: runner.name,
|
||||
keyHash: hashRunnerKey(key),
|
||||
}
|
||||
const conn = createMockWs()
|
||||
await service.registerConnection(authenticated, conn.ws)
|
||||
reconcile(service, runner.id, conn.ws)
|
||||
const abort = new AbortController()
|
||||
const execution = service.exec(runner.id, { command: 'sleep 30', signal: abort.signal })
|
||||
const { executionId } = conn.lastSentJson()
|
||||
|
||||
service.handleDisconnect(runner.id, conn.ws)
|
||||
abort.abort()
|
||||
if (reported) await expect(execution).resolves.toMatchObject({ aborted: true })
|
||||
const reconnected = createMockWs()
|
||||
await service.registerConnection(authenticated, reconnected.ws)
|
||||
expect(reconnected.lastSentJson()).toMatchObject({ inFlightExecutionIds: [executionId] })
|
||||
reconcile(service, runner.id, reconnected.ws, [executionId])
|
||||
const forwarded = reconnected.lastSentJson()
|
||||
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
reconnected.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.result',
|
||||
executionId,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
}),
|
||||
)
|
||||
expect(forwarded).toEqual({ type: 'exec.cancel', executionId })
|
||||
await expect(execution).resolves.toMatchObject({ aborted: true })
|
||||
},
|
||||
)
|
||||
|
||||
it('restages a generated input that was removed from the runner workspace', async () => {
|
||||
vi.spyOn(s3Service, 'headObject').mockResolvedValue({
|
||||
key: 'temp/generated/example/input.jpg',
|
||||
size: 100,
|
||||
lastModified: new Date(),
|
||||
contentType: 'image/jpeg',
|
||||
eTag: 'fixture',
|
||||
})
|
||||
vi.spyOn(s3Service, 'presign').mockResolvedValue('https://signed.example.com/input.jpg')
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'Generated file runner' })
|
||||
const conn = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
|
||||
conn.ws,
|
||||
)
|
||||
reconcile(service, runner.id, conn.ws)
|
||||
|
||||
const destPath = '.pi/generated/example/input.jpg'
|
||||
const acknowledgeDownload = () => {
|
||||
const download = conn.lastSentJson()
|
||||
expect(download).toMatchObject({ type: 'file.download', destPath })
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'file.download.result',
|
||||
executionId: download.executionId,
|
||||
ok: true,
|
||||
}),
|
||||
)
|
||||
}
|
||||
const first = service.ensureGeneratedFileOnRunner(runner.id, destPath)
|
||||
await vi.waitFor(() => expect(conn.lastSentJson()?.type).toBe('file.download'))
|
||||
acknowledgeDownload()
|
||||
await expect(first).resolves.toBe(destPath)
|
||||
|
||||
const second = service.ensureGeneratedFileOnRunner(runner.id, destPath)
|
||||
const stat = conn.lastSentJson()
|
||||
expect(stat).toMatchObject({ type: 'file.stat', srcPath: destPath })
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'file.stat.result',
|
||||
executionId: stat.executionId,
|
||||
ok: true,
|
||||
exists: false,
|
||||
}),
|
||||
)
|
||||
await vi.waitFor(() => expect(conn.lastSentJson()?.type).toBe('file.download'))
|
||||
acknowledgeDownload()
|
||||
await expect(second).resolves.toBe(destPath)
|
||||
|
||||
const third = service.ensureGeneratedFileOnRunner(runner.id, destPath)
|
||||
const present = conn.lastSentJson()
|
||||
expect(present).toMatchObject({ type: 'file.stat', srcPath: destPath })
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'file.stat.result',
|
||||
executionId: present.executionId,
|
||||
ok: true,
|
||||
exists: true,
|
||||
isFile: true,
|
||||
}),
|
||||
)
|
||||
await expect(third).resolves.toBe(destPath)
|
||||
expect(conn.lastSentJson()).toEqual(present)
|
||||
})
|
||||
|
||||
it('handles downloadToRunner, uploadFromRunner, statOnRunner, and session caching', async () => {
|
||||
vi.spyOn(s3Service, 'presign').mockResolvedValue('https://signed.example.com/file')
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'File Runner' })
|
||||
const conn = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
|
||||
conn.ws,
|
||||
)
|
||||
reconcile(service, runner.id, conn.ws)
|
||||
|
||||
// 1. downloadToRunner
|
||||
const dlPromise = service.downloadToRunner(runner.id, {
|
||||
url: 'https://signed.example.com/asset',
|
||||
destPath: '.pi/video.mp4',
|
||||
})
|
||||
const dlReq = conn.lastSentJson()
|
||||
expect(dlReq.type).toBe('file.download')
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'file.download.result',
|
||||
executionId: dlReq.executionId,
|
||||
ok: true,
|
||||
size: 1024,
|
||||
mimeType: 'video/mp4',
|
||||
}),
|
||||
)
|
||||
await expect(dlPromise).resolves.toMatchObject({
|
||||
ok: true,
|
||||
size: 1024,
|
||||
mimeType: 'video/mp4',
|
||||
})
|
||||
|
||||
// 2. statOnRunner
|
||||
const statPromise = service.statOnRunner(runner.id, '.pi/out.png')
|
||||
const statReq = conn.lastSentJson()
|
||||
expect(statReq.type).toBe('file.stat')
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'file.stat.result',
|
||||
executionId: statReq.executionId,
|
||||
ok: true,
|
||||
exists: true,
|
||||
size: 512,
|
||||
mimeType: 'image/png',
|
||||
}),
|
||||
)
|
||||
await expect(statPromise).resolves.toMatchObject({
|
||||
ok: true,
|
||||
exists: true,
|
||||
size: 512,
|
||||
mimeType: 'image/png',
|
||||
})
|
||||
|
||||
// 3. uploadFromRunner
|
||||
const ulPromise = service.uploadFromRunner(runner.id, {
|
||||
srcPath: '.pi/out.png',
|
||||
url: 'https://signed.example.com/put',
|
||||
contentType: 'image/png',
|
||||
})
|
||||
const ulReq = conn.lastSentJson()
|
||||
expect(ulReq.type).toBe('file.upload')
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'file.upload.result',
|
||||
executionId: ulReq.executionId,
|
||||
ok: true,
|
||||
size: 512,
|
||||
}),
|
||||
)
|
||||
await expect(ulPromise).resolves.toMatchObject({ ok: true, size: 512 })
|
||||
|
||||
// 4. ensureSkillOnRunner caches by skillId:hash per connection
|
||||
const asset = await prisma.asset.create({
|
||||
data: {
|
||||
name: 'skill.zip',
|
||||
type: 'file',
|
||||
status: 'uploaded',
|
||||
storageKey: { create: { key: 'skills/s1.zip' } },
|
||||
},
|
||||
})
|
||||
const skillPromise1 = service.ensureSkillOnRunner(runner.id, {
|
||||
id: 'skill-1',
|
||||
assetId: asset.id,
|
||||
hash: 'h1',
|
||||
})
|
||||
await vi.waitFor(() => expect(conn.lastSentJson()?.type).toBe('file.download'))
|
||||
const skillReq = conn.lastSentJson()
|
||||
expect(skillReq).toMatchObject({
|
||||
type: 'file.download',
|
||||
destPath: '.pi/skills/skill-1',
|
||||
extractZip: true,
|
||||
expectedHash: 'h1',
|
||||
})
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn.ws,
|
||||
JSON.stringify({
|
||||
type: 'file.download.result',
|
||||
executionId: skillReq.executionId,
|
||||
ok: true,
|
||||
size: 100,
|
||||
}),
|
||||
)
|
||||
await skillPromise1
|
||||
|
||||
const countBeforeSecondCall = conn.sent.length
|
||||
await service.ensureSkillOnRunner(runner.id, {
|
||||
id: 'skill-1',
|
||||
assetId: asset.id,
|
||||
hash: 'h1',
|
||||
})
|
||||
expect(conn.sent.length).toBe(countBeforeSecondCall)
|
||||
})
|
||||
|
||||
it('recovers in-flight execution across a brief disconnect when runner reconciles', async () => {
|
||||
const { runner, key } = await service.createRunner(teamId, { name: 'Reconnect Runner' })
|
||||
const keyHash = hashRunnerKey(key)
|
||||
const conn1 = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash },
|
||||
conn1.ws,
|
||||
)
|
||||
reconcile(service, runner.id, conn1.ws)
|
||||
|
||||
const execPromise = service.exec(runner.id, { command: 'long-job' })
|
||||
const startMsg = conn1.lastSentJson()
|
||||
|
||||
// Disconnect conn1 (starts grace timer)
|
||||
service.handleDisconnect(runner.id, conn1.ws)
|
||||
|
||||
// Reconnect conn2 within grace period and send reconcile with active executionId
|
||||
const conn2 = createMockWs()
|
||||
await service.registerConnection(
|
||||
{ id: runner.id, teamId, name: runner.name, keyHash },
|
||||
conn2.ws,
|
||||
)
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn2.ws,
|
||||
JSON.stringify({
|
||||
type: 'reconcile',
|
||||
environment: { platform: 'linux', shell: '/bin/sh' },
|
||||
activeExecutions: [{ executionId: startMsg.executionId, nextOffset: 0 }],
|
||||
completedExecutions: [],
|
||||
}),
|
||||
)
|
||||
|
||||
// Finish execution on conn2
|
||||
service.handleMessage(
|
||||
runner.id,
|
||||
conn2.ws,
|
||||
JSON.stringify({
|
||||
type: 'exec.result',
|
||||
executionId: startMsg.executionId,
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
}),
|
||||
)
|
||||
|
||||
const result = await execPromise
|
||||
expect(result.exitCode).toBe(0)
|
||||
})
|
||||
})
|
||||
})
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,76 @@
|
||||
import { HTTPException } from 'hono/http-exception'
|
||||
import { prisma } from '@shumai/db'
|
||||
import { tryAcquireUpgradeableAdvisoryLock } from '@shumai/db/src/advisory-lock'
|
||||
import { logger } from '@shumai/core/src/logger'
|
||||
|
||||
const RUNNER_TOPOLOGY_KEY = 'shumai:runner-topology'
|
||||
const SINGLE_SERVER_REQUIRED =
|
||||
'Agent runners require a single main Shumai server per database. Stop the other main server replicas.'
|
||||
|
||||
async function hasConfiguredRunners() {
|
||||
return (await prisma.runner.findFirst({ select: { id: true } })) !== null
|
||||
}
|
||||
|
||||
/** Each main server holds a shared lease. A configured runner requires an exclusive lease. */
|
||||
export class RunnerTopology {
|
||||
private mutations = Promise.resolve()
|
||||
|
||||
private constructor(
|
||||
private readonly lease: NonNullable<
|
||||
Awaited<ReturnType<typeof tryAcquireUpgradeableAdvisoryLock>>
|
||||
>,
|
||||
) {}
|
||||
|
||||
static async start() {
|
||||
const configured = await hasConfiguredRunners()
|
||||
let lease = await tryAcquireUpgradeableAdvisoryLock(RUNNER_TOPOLOGY_KEY, configured)
|
||||
if (!lease && configured) {
|
||||
// If the last runner was just deleted, the incumbent may now hold a shared lease.
|
||||
lease = await tryAcquireUpgradeableAdvisoryLock(RUNNER_TOPOLOGY_KEY, false)
|
||||
}
|
||||
if (!lease) throw new Error(SINGLE_SERVER_REQUIRED)
|
||||
try {
|
||||
// A concurrently stopping server may have changed the catalog.
|
||||
if (await hasConfiguredRunners()) {
|
||||
if (!(await lease.tryUpgrade())) throw new Error(SINGLE_SERVER_REQUIRED)
|
||||
} else {
|
||||
await lease.downgrade()
|
||||
}
|
||||
return new RunnerTopology(lease)
|
||||
} catch (error) {
|
||||
await lease.release()
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
get signal() {
|
||||
return this.lease.signal
|
||||
}
|
||||
|
||||
/** Serialize runner creation and deletion until the exclusive lease can be safely downgraded. */
|
||||
async changeRunners<T>(operation: () => Promise<T>, adding = false): Promise<T> {
|
||||
const previous = this.mutations
|
||||
const finished = Promise.withResolvers<void>()
|
||||
this.mutations = finished.promise
|
||||
await previous
|
||||
try {
|
||||
if (adding && !(await this.lease.tryUpgrade())) {
|
||||
throw new HTTPException(409, { message: SINGLE_SERVER_REQUIRED })
|
||||
}
|
||||
return await operation()
|
||||
} finally {
|
||||
try {
|
||||
if (this.lease.exclusive && !(await hasConfiguredRunners())) await this.lease.downgrade()
|
||||
} catch (error) {
|
||||
logger.warn({ err: error }, 'Failed to downgrade the runner server lease')
|
||||
} finally {
|
||||
finished.resolve()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async close() {
|
||||
await this.mutations
|
||||
await this.lease.release()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
import type { RunnerInfo } from '@shumai/dtos'
|
||||
|
||||
export interface RunnerSocket {
|
||||
send(data: string): void
|
||||
close(code?: number, reason?: string): void
|
||||
}
|
||||
|
||||
export interface AuthenticatedRunner {
|
||||
id: string
|
||||
teamId: string
|
||||
name: string
|
||||
keyHash: string
|
||||
}
|
||||
|
||||
export interface RunnerExecOptions {
|
||||
command: string
|
||||
timeoutSec?: number
|
||||
env?: Record<string, string>
|
||||
signal?: AbortSignal
|
||||
onOutput?: (chunk: string, stream: 'stdout' | 'stderr') => void
|
||||
/**
|
||||
* Whether `exec()` should retain streamed output in memory to return in `RunnerExecResult.output`.
|
||||
* Streaming callers should pass `false` to avoid duplicate full-output retention on the server.
|
||||
* Defaults to `true`.
|
||||
*/
|
||||
collectOutput?: boolean
|
||||
/**
|
||||
* Maximum UTF-8 bytes of output retained in memory when `collectOutput` is true.
|
||||
* Older output is trimmed from the front so server memory remains bounded.
|
||||
*/
|
||||
maxBufferedOutputBytes?: number
|
||||
}
|
||||
|
||||
export interface RunnerExecResult {
|
||||
executionId: string
|
||||
output: string
|
||||
exitCode: number | null
|
||||
signal: string | null
|
||||
timedOut: boolean
|
||||
aborted: boolean
|
||||
fullOutputPath?: string
|
||||
error?: string
|
||||
}
|
||||
|
||||
export interface ResolvedRunnerTarget {
|
||||
hasConfiguredRunner: boolean
|
||||
runner: RunnerInfo | null
|
||||
isBoundToAgent: boolean
|
||||
}
|
||||
@@ -717,6 +717,50 @@ describe('S3Service implementations', () => {
|
||||
expect(keysAll.length).toBe(3)
|
||||
})
|
||||
|
||||
it('lists a nested prefix without scanning unrelated directories', async () => {
|
||||
await localS3.putObject('test-bucket', 'temp/generated/one.jpg', 'one', 3)
|
||||
await localS3.putObject('test-bucket', 'files/unrelated.txt', 'abc', 3)
|
||||
const readDirectory = vi.spyOn(fs.promises, 'readdir')
|
||||
try {
|
||||
await expect(localS3.listObjects('test-bucket', 'temp/generated/')).resolves.toEqual([
|
||||
'temp/generated/one.jpg',
|
||||
])
|
||||
expect(readDirectory.mock.calls.map(([directory]) => directory)).toEqual([
|
||||
path.join(TEST_BASE_PATH, 'test-bucket', 'temp', 'generated'),
|
||||
])
|
||||
readDirectory.mockClear()
|
||||
await expect(localS3.listObjects('test-bucket', 'temp/missing/')).resolves.toEqual([])
|
||||
expect(readDirectory.mock.calls.map(([directory]) => directory)).not.toContain(
|
||||
path.join(TEST_BASE_PATH, 'test-bucket'),
|
||||
)
|
||||
await expect(localS3.listObjects('test-bucket', 'temp/generated/on')).resolves.toEqual([
|
||||
'temp/generated/one.jpg',
|
||||
])
|
||||
} finally {
|
||||
readDirectory.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('continues listing when a file disappears during a local prefix scan', async () => {
|
||||
const removedKey = 'temp/generated/removed.jpg'
|
||||
const remainingKey = 'temp/generated/remaining.jpg'
|
||||
await localS3.putObject('test-bucket', removedKey, 'old', 3)
|
||||
await localS3.putObject('test-bucket', remainingKey, 'new', 3)
|
||||
const stat = fs.promises.stat
|
||||
const removedPath = path.join(TEST_BASE_PATH, 'test-bucket', removedKey)
|
||||
const statSpy = vi.spyOn(fs.promises, 'stat').mockImplementation(async (target, options) => {
|
||||
if (target === removedPath) await fs.promises.unlink(removedPath)
|
||||
return stat(target, options)
|
||||
})
|
||||
try {
|
||||
await expect(localS3.listObjects('test-bucket', 'temp/generated/')).resolves.toEqual([
|
||||
remainingKey,
|
||||
])
|
||||
} finally {
|
||||
statSpy.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('should handle headObject correctly', async () => {
|
||||
await localS3.putObject('test-bucket', 'test.txt', 'hello world', 11)
|
||||
const head = await localS3.headObject('test-bucket', 'test.txt')
|
||||
|
||||
@@ -1012,16 +1012,33 @@ export class LocalStorageService implements S3Service {
|
||||
async listObjects(bucket: string, prefix: string): Promise<string[]> {
|
||||
const bucketDir = path.join(this.basePath, bucket)
|
||||
const keys: string[] = []
|
||||
|
||||
if (!fs.existsSync(bucketDir)) {
|
||||
const directoryPrefix = prefix.slice(0, prefix.lastIndexOf('/') + 1)
|
||||
const directory = path.join(bucketDir, directoryPrefix.slice(0, -1))
|
||||
const relativeDirectory = path.relative(bucketDir, directory).split(path.sep).join('/')
|
||||
if (
|
||||
directoryPrefix &&
|
||||
(!relativeDirectory ||
|
||||
relativeDirectory === '..' ||
|
||||
relativeDirectory.startsWith('../') ||
|
||||
relativeDirectory + '/' !== directoryPrefix)
|
||||
)
|
||||
return keys
|
||||
}
|
||||
|
||||
const ignoreMissing = (error: unknown) => {
|
||||
if (
|
||||
error instanceof Error &&
|
||||
'code' in error &&
|
||||
(error.code === 'ENOENT' || error.code === 'ENOTDIR')
|
||||
)
|
||||
return undefined
|
||||
throw error
|
||||
}
|
||||
const walkDir = async (currentDir: string) => {
|
||||
const files = await fs.promises.readdir(currentDir)
|
||||
const files = (await fs.promises.readdir(currentDir).catch(ignoreMissing)) ?? []
|
||||
for (const file of files) {
|
||||
const fullPath = path.join(currentDir, file)
|
||||
const stat = await fs.promises.stat(fullPath)
|
||||
const stat = await fs.promises.stat(fullPath).catch(ignoreMissing)
|
||||
if (!stat) continue
|
||||
|
||||
if (stat.isDirectory()) {
|
||||
await walkDir(fullPath)
|
||||
@@ -1037,7 +1054,7 @@ export class LocalStorageService implements S3Service {
|
||||
}
|
||||
}
|
||||
|
||||
await walkDir(bucketDir)
|
||||
await walkDir(directory)
|
||||
return keys
|
||||
}
|
||||
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
export * from './types'
|
||||
export * from './providers/local-provider'
|
||||
export * from './sandbox-service'
|
||||
@@ -1,177 +0,0 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { LocalSandboxProvider } from './local-provider'
|
||||
import { SandboxManager } from '@anthropic-ai/sandbox-runtime'
|
||||
import type { SandboxConfig, SandboxAskCallback } from '../types'
|
||||
|
||||
vi.mock('@anthropic-ai/sandbox-runtime', () => ({
|
||||
SandboxManager: {
|
||||
initialize: vi.fn(),
|
||||
wrapWithSandbox: vi.fn(),
|
||||
getConfig: vi.fn(),
|
||||
updateConfig: vi.fn(),
|
||||
reset: vi.fn(),
|
||||
isSandboxingEnabled: vi.fn(),
|
||||
getSandboxViolationStore: vi.fn(),
|
||||
},
|
||||
}))
|
||||
|
||||
describe('LocalSandboxProvider', () => {
|
||||
let provider: LocalSandboxProvider
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
provider = new LocalSandboxProvider()
|
||||
})
|
||||
|
||||
it('delegates initialize to SandboxManager', async () => {
|
||||
const config: SandboxConfig = {
|
||||
network: { allowedDomains: ['example.com'], deniedDomains: [] },
|
||||
filesystem: {
|
||||
denyRead: ['.env'],
|
||||
allowWrite: ['/tmp'],
|
||||
denyWrite: ['.env'],
|
||||
},
|
||||
enableWeakerNestedSandbox: true,
|
||||
}
|
||||
const callback: SandboxAskCallback = async () => true
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mock resolves void
|
||||
vi.mocked(SandboxManager.initialize as any).mockResolvedValue(undefined)
|
||||
vi.mocked(SandboxManager.isSandboxingEnabled).mockReturnValue(true)
|
||||
|
||||
await provider.initialize(config, callback)
|
||||
|
||||
expect(SandboxManager.initialize).toHaveBeenCalledWith(
|
||||
{
|
||||
network: config.network,
|
||||
filesystem: config.filesystem,
|
||||
enableWeakerNestedSandbox: true,
|
||||
},
|
||||
callback,
|
||||
)
|
||||
expect(provider.isInitialized()).toBe(true)
|
||||
})
|
||||
|
||||
it('delegates wrapCommand with and without commandId', async () => {
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mock wrapWithSandbox
|
||||
vi.mocked(SandboxManager.wrapWithSandbox as any).mockResolvedValue('wrapped-command')
|
||||
|
||||
const res1 = await provider.wrapCommand('echo hello')
|
||||
expect(res1).toBe('wrapped-command')
|
||||
expect(SandboxManager.wrapWithSandbox).toHaveBeenCalledWith(
|
||||
'echo hello',
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
)
|
||||
|
||||
const res2 = await provider.wrapCommand('curl example.com', { commandId: 'cmd-123' })
|
||||
expect(res2).toBe('wrapped-command')
|
||||
expect(SandboxManager.wrapWithSandbox).toHaveBeenCalledWith(
|
||||
'curl example.com',
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
{ commandId: 'cmd-123' },
|
||||
)
|
||||
})
|
||||
|
||||
describe('updateConfig', () => {
|
||||
it('throws when called before initialization (no config in manager)', async () => {
|
||||
vi.mocked(SandboxManager.getConfig).mockReturnValue(undefined)
|
||||
|
||||
await expect(
|
||||
provider.updateConfig({ network: { allowedDomains: ['new.com'], deniedDomains: [] } }),
|
||||
).rejects.toThrow('Cannot update sandbox config before initialization')
|
||||
})
|
||||
|
||||
it('merges partial network and filesystem configuration', async () => {
|
||||
const currentConfig = {
|
||||
network: { allowedDomains: ['old.com'], deniedDomains: [] },
|
||||
filesystem: { denyRead: ['.env'], allowWrite: ['/tmp'], denyWrite: [] },
|
||||
enableWeakerNestedSandbox: false,
|
||||
}
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mock getConfig return
|
||||
vi.mocked(SandboxManager.getConfig).mockReturnValue(currentConfig as any)
|
||||
|
||||
await provider.updateConfig({
|
||||
network: { allowedDomains: ['new.com', 'old.com'], deniedDomains: [] },
|
||||
})
|
||||
|
||||
expect(SandboxManager.updateConfig).toHaveBeenCalledWith({
|
||||
network: { allowedDomains: ['new.com', 'old.com'], deniedDomains: [] },
|
||||
filesystem: { denyRead: ['.env'], allowWrite: ['/tmp'], denyWrite: [] },
|
||||
enableWeakerNestedSandbox: false,
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
it('resets SandboxManager and clears initialized state', async () => {
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mock resolves
|
||||
vi.mocked(SandboxManager.initialize as any).mockResolvedValue(undefined)
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mock resolves
|
||||
vi.mocked(SandboxManager.reset as any).mockResolvedValue(undefined)
|
||||
vi.mocked(SandboxManager.isSandboxingEnabled).mockReturnValue(true)
|
||||
|
||||
await provider.initialize({
|
||||
network: { allowedDomains: [], deniedDomains: [] },
|
||||
filesystem: { denyRead: [], allowWrite: [], denyWrite: [] },
|
||||
})
|
||||
expect(provider.isInitialized()).toBe(true)
|
||||
|
||||
await provider.reset()
|
||||
expect(SandboxManager.reset).toHaveBeenCalled()
|
||||
expect(provider.isInitialized()).toBe(false)
|
||||
})
|
||||
|
||||
describe('getBlockedHostForCommand', () => {
|
||||
it('extracts blocked host from deny network-outbound lines', () => {
|
||||
const mockStore = {
|
||||
getViolationsForCommand: vi
|
||||
.fn()
|
||||
.mockReturnValue([
|
||||
{ line: 'something else' },
|
||||
{ line: 'deny network-outbound evil.com:443' },
|
||||
]),
|
||||
}
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mock store
|
||||
vi.mocked(SandboxManager.getSandboxViolationStore).mockReturnValue(mockStore as any)
|
||||
|
||||
const blocked = provider.getBlockedHostForCommand('cmd-1')
|
||||
expect(blocked).toBe('evil.com')
|
||||
expect(mockStore.getViolationsForCommand).toHaveBeenCalledWith('cmd-1')
|
||||
})
|
||||
|
||||
it('extracts blocked host from url matches', () => {
|
||||
const mockStore = {
|
||||
getViolationsForCommand: vi
|
||||
.fn()
|
||||
.mockReturnValue([{ line: 'violation: https://blocked-url.com/some/path' }]),
|
||||
}
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mock store
|
||||
vi.mocked(SandboxManager.getSandboxViolationStore).mockReturnValue(mockStore as any)
|
||||
|
||||
const blocked = provider.getBlockedHostForCommand('cmd-2')
|
||||
expect(blocked).toBe('blocked-url.com')
|
||||
})
|
||||
|
||||
it('returns null if no violations or no matching regex', () => {
|
||||
const mockStore = {
|
||||
getViolationsForCommand: vi.fn().mockReturnValue([]),
|
||||
}
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- mock store
|
||||
vi.mocked(SandboxManager.getSandboxViolationStore).mockReturnValue(mockStore as any)
|
||||
|
||||
expect(provider.getBlockedHostForCommand('cmd-3')).toBeNull()
|
||||
})
|
||||
|
||||
it('handles violation store errors gracefully', () => {
|
||||
vi.mocked(SandboxManager.getSandboxViolationStore).mockImplementation(() => {
|
||||
throw new Error('store failure')
|
||||
})
|
||||
|
||||
expect(provider.getBlockedHostForCommand('cmd-err')).toBeNull()
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,86 +0,0 @@
|
||||
import { SandboxManager } from '@anthropic-ai/sandbox-runtime'
|
||||
import type {
|
||||
SandboxProvider,
|
||||
SandboxConfig,
|
||||
SandboxAskCallback,
|
||||
WrapCommandOptions,
|
||||
} from '../types'
|
||||
|
||||
export class LocalSandboxProvider implements SandboxProvider {
|
||||
private initialized = false
|
||||
|
||||
async initialize(config: SandboxConfig, callback?: SandboxAskCallback): Promise<void> {
|
||||
await SandboxManager.initialize(
|
||||
{
|
||||
network: config.network,
|
||||
filesystem: config.filesystem,
|
||||
enableWeakerNestedSandbox: config.enableWeakerNestedSandbox,
|
||||
},
|
||||
callback,
|
||||
)
|
||||
this.initialized = true
|
||||
}
|
||||
|
||||
async wrapCommand(command: string, options?: WrapCommandOptions): Promise<string> {
|
||||
return SandboxManager.wrapWithSandbox(
|
||||
command,
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
options?.commandId ? { commandId: options.commandId } : undefined,
|
||||
)
|
||||
}
|
||||
|
||||
async updateConfig(config: Partial<SandboxConfig>): Promise<void> {
|
||||
const current = SandboxManager.getConfig()
|
||||
if (!current) {
|
||||
throw new Error('Cannot update sandbox config before initialization')
|
||||
}
|
||||
const updated = {
|
||||
...current,
|
||||
...(config.network && {
|
||||
network: {
|
||||
...current.network,
|
||||
...config.network,
|
||||
},
|
||||
}),
|
||||
...(config.filesystem && {
|
||||
filesystem: {
|
||||
...current.filesystem,
|
||||
...config.filesystem,
|
||||
},
|
||||
}),
|
||||
...(config.enableWeakerNestedSandbox !== undefined && {
|
||||
enableWeakerNestedSandbox: config.enableWeakerNestedSandbox,
|
||||
}),
|
||||
}
|
||||
SandboxManager.updateConfig(updated)
|
||||
}
|
||||
|
||||
async reset(): Promise<void> {
|
||||
await SandboxManager.reset()
|
||||
this.initialized = false
|
||||
}
|
||||
|
||||
isInitialized(): boolean {
|
||||
return this.initialized && SandboxManager.isSandboxingEnabled()
|
||||
}
|
||||
|
||||
getBlockedHostForCommand(commandId: string): string | null {
|
||||
try {
|
||||
const store = SandboxManager.getSandboxViolationStore()
|
||||
const violations = store.getViolationsForCommand(commandId)
|
||||
for (let i = violations.length - 1; i >= 0; i--) {
|
||||
const line = violations[i]?.line
|
||||
if (!line) continue
|
||||
const m1 = line.match(/deny network-outbound ([^:\s]+)/)
|
||||
if (m1?.[1]) return m1[1]
|
||||
const m2 = line.match(/https?:\/\/([^:/\s]+)/)
|
||||
if (m2?.[1]) return m2[1]
|
||||
}
|
||||
} catch {
|
||||
// Violation store query failed or not available
|
||||
}
|
||||
return null
|
||||
}
|
||||
}
|
||||
@@ -1,219 +0,0 @@
|
||||
import { describe, it, expect, beforeEach } from 'vitest'
|
||||
import { prisma } from '@shumai/db'
|
||||
import { setupTestDbHooks } from '@shumai/db/test'
|
||||
import { sandboxService } from './sandbox-service'
|
||||
import type {
|
||||
SandboxProvider,
|
||||
SandboxConfig,
|
||||
SandboxAskCallback,
|
||||
WrapCommandOptions,
|
||||
} from './types'
|
||||
import { teamService } from '../team/team'
|
||||
|
||||
class MockProvider implements SandboxProvider {
|
||||
initialized = false
|
||||
config: SandboxConfig | null = null
|
||||
initializeCallCount = 0
|
||||
updateConfigCallCount = 0
|
||||
resetCallCount = 0
|
||||
wrapCommandCalls: Array<{ command: string; options?: WrapCommandOptions }> = []
|
||||
violationsByCommandId = new Map<string, string>()
|
||||
initDelayPromise: Promise<void> | null = null
|
||||
callback: SandboxAskCallback | undefined
|
||||
|
||||
async initialize(config: SandboxConfig, callback?: SandboxAskCallback): Promise<void> {
|
||||
if (this.initDelayPromise) {
|
||||
await this.initDelayPromise
|
||||
}
|
||||
this.initialized = true
|
||||
this.config = config
|
||||
this.callback = callback
|
||||
this.initializeCallCount++
|
||||
}
|
||||
|
||||
async wrapCommand(command: string, options?: WrapCommandOptions): Promise<string> {
|
||||
this.wrapCommandCalls.push({ command, options })
|
||||
return `wrapped-${command}`
|
||||
}
|
||||
|
||||
async updateConfig(config: Partial<SandboxConfig>): Promise<void> {
|
||||
this.updateConfigCallCount++
|
||||
if (this.config) {
|
||||
if (config.network) {
|
||||
this.config.network = { ...this.config.network, ...config.network }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async reset(): Promise<void> {
|
||||
this.resetCallCount++
|
||||
this.initialized = false
|
||||
this.config = null
|
||||
}
|
||||
|
||||
isInitialized(): boolean {
|
||||
return this.initialized
|
||||
}
|
||||
|
||||
getBlockedHostForCommand(commandId: string): string | null {
|
||||
return this.violationsByCommandId.get(commandId) ?? null
|
||||
}
|
||||
}
|
||||
|
||||
describe('SandboxService', () => {
|
||||
setupTestDbHooks()
|
||||
|
||||
let mockProvider: MockProvider
|
||||
|
||||
beforeEach(async () => {
|
||||
mockProvider = new MockProvider()
|
||||
sandboxService.setProvider(mockProvider)
|
||||
})
|
||||
|
||||
it('initializes provider once on ensureInitialized', async () => {
|
||||
const team = await teamService.ensureDefaultTeam()
|
||||
|
||||
await sandboxService.ensureInitialized({ teamId: team.id })
|
||||
expect(mockProvider.initializeCallCount).toBe(1)
|
||||
expect(mockProvider.isInitialized()).toBe(true)
|
||||
|
||||
// Second call should be a no-op
|
||||
await sandboxService.ensureInitialized({ teamId: team.id })
|
||||
expect(mockProvider.initializeCallCount).toBe(1)
|
||||
})
|
||||
|
||||
it('handles concurrent ensureInitialized calls without duplicate initializations', async () => {
|
||||
const team = await teamService.ensureDefaultTeam()
|
||||
|
||||
// Simulate 5 concurrent calls
|
||||
await Promise.all([
|
||||
sandboxService.ensureInitialized({ teamId: team.id }),
|
||||
sandboxService.ensureInitialized({ teamId: team.id }),
|
||||
sandboxService.ensureInitialized({ teamId: team.id }),
|
||||
sandboxService.ensureInitialized({ teamId: team.id }),
|
||||
sandboxService.ensureInitialized({ teamId: team.id }),
|
||||
])
|
||||
|
||||
expect(mockProvider.initializeCallCount).toBe(1)
|
||||
})
|
||||
|
||||
it('hot-reloads configuration only when domains change', async () => {
|
||||
const team = await teamService.ensureDefaultTeam()
|
||||
|
||||
// 1. Initial sync initializes provider
|
||||
await sandboxService.syncAllowedDomains(['github.com', 'pypi.org'], team.id)
|
||||
expect(mockProvider.initializeCallCount).toBe(1)
|
||||
expect(mockProvider.updateConfigCallCount).toBe(0)
|
||||
expect(sandboxService.getCurrentAllowedDomains()).toEqual(['github.com', 'pypi.org'])
|
||||
|
||||
// 2. Same domains (even in different order) should be a no-op
|
||||
await sandboxService.syncAllowedDomains(['pypi.org', 'github.com'], team.id)
|
||||
expect(mockProvider.updateConfigCallCount).toBe(0)
|
||||
|
||||
// 3. Different domains should trigger updateConfig without reset
|
||||
await sandboxService.syncAllowedDomains(['github.com', 'pypi.org', 'npm.org'], team.id)
|
||||
expect(mockProvider.updateConfigCallCount).toBe(1)
|
||||
expect(mockProvider.resetCallCount).toBe(0)
|
||||
expect(sandboxService.getCurrentAllowedDomains()).toEqual(['github.com', 'pypi.org', 'npm.org'])
|
||||
})
|
||||
|
||||
it('integrates with teamService.updateSandboxSettings for live hot-reloading', async () => {
|
||||
const team = await teamService.ensureDefaultTeam()
|
||||
|
||||
// Initialize sandbox first
|
||||
await sandboxService.ensureInitialized({ teamId: team.id })
|
||||
expect(mockProvider.initializeCallCount).toBe(1)
|
||||
expect(mockProvider.updateConfigCallCount).toBe(0)
|
||||
|
||||
// Update sandbox settings via teamService
|
||||
await teamService.updateSandboxSettings(team.id, {
|
||||
networkSandboxEnabled: true,
|
||||
allowedDomains: ['custom-api.example.com'],
|
||||
})
|
||||
|
||||
// Verify DB updated
|
||||
const updated = await prisma.sandbox.findUnique({ where: { teamId: team.id } })
|
||||
expect(updated?.allowedDomains).toEqual(['custom-api.example.com'])
|
||||
expect(updated?.networkSandboxEnabled).toBe(true)
|
||||
|
||||
// Verify sandboxService hot-reloaded the provider live
|
||||
expect(mockProvider.updateConfigCallCount).toBe(1)
|
||||
expect(mockProvider.resetCallCount).toBe(0)
|
||||
expect(sandboxService.getCurrentAllowedDomains()).toEqual(['custom-api.example.com'])
|
||||
})
|
||||
|
||||
it('delegates wrapCommand and extracts blocked host for commandId', async () => {
|
||||
const team = await teamService.ensureDefaultTeam()
|
||||
await sandboxService.ensureInitialized({ teamId: team.id })
|
||||
|
||||
const result = await sandboxService.wrapCommand('echo hello', { commandId: 'tool-call-1' })
|
||||
expect(result).toBe('wrapped-echo hello')
|
||||
expect(mockProvider.wrapCommandCalls).toEqual([
|
||||
{ command: 'echo hello', options: { commandId: 'tool-call-1' } },
|
||||
])
|
||||
|
||||
// Set mock violation for command
|
||||
mockProvider.violationsByCommandId.set('tool-call-1', 'blocked.malicious.com')
|
||||
|
||||
expect(sandboxService.getBlockedHost('tool-call-1')).toBe('blocked.malicious.com')
|
||||
expect(sandboxService.getBlockedHost('tool-call-2')).toBe('')
|
||||
})
|
||||
|
||||
it('does not leak lastBlockedHost to command with no violations', async () => {
|
||||
const team = await teamService.ensureDefaultTeam()
|
||||
await prisma.sandbox.upsert({
|
||||
where: { teamId: team.id },
|
||||
create: { teamId: team.id, networkSandboxEnabled: true },
|
||||
update: { networkSandboxEnabled: true },
|
||||
})
|
||||
await sandboxService.ensureInitialized({ teamId: team.id })
|
||||
|
||||
// Simulate global ask callback recording a blocked host
|
||||
if (mockProvider.callback) {
|
||||
await mockProvider.callback({ host: 'blocked.global.com', port: 443 })
|
||||
}
|
||||
|
||||
// Legacy un-scoped call returns lastBlockedHost
|
||||
expect(sandboxService.getBlockedHost()).toBe('blocked.global.com')
|
||||
|
||||
// Command-scoped call for clean command MUST return ''
|
||||
expect(sandboxService.getBlockedHost('clean-cmd')).toBe('')
|
||||
|
||||
// Command-scoped call with recorded violation returns that violation
|
||||
mockProvider.violationsByCommandId.set('violating-cmd', 'violating.host.com')
|
||||
expect(sandboxService.getBlockedHost('violating-cmd')).toBe('violating.host.com')
|
||||
})
|
||||
|
||||
it('serializes concurrent syncAllowedDomains during delayed initialization and applies latest configuration', async () => {
|
||||
const team = await teamService.ensureDefaultTeam()
|
||||
|
||||
let resolveInit: () => void = () => {}
|
||||
mockProvider.initDelayPromise = new Promise<void>((resolve) => {
|
||||
resolveInit = resolve
|
||||
})
|
||||
|
||||
// Start two concurrent sync requests before initialization completes
|
||||
const sync1 = sandboxService.syncAllowedDomains(['first.com'], team.id)
|
||||
const sync2 = sandboxService.syncAllowedDomains(['second.com'], team.id)
|
||||
|
||||
// Complete the provider initialization
|
||||
resolveInit()
|
||||
await Promise.all([sync1, sync2])
|
||||
|
||||
// Provider was initialized once, and then updated to the second request
|
||||
expect(mockProvider.initializeCallCount).toBe(1)
|
||||
expect(mockProvider.updateConfigCallCount).toBe(1)
|
||||
expect(sandboxService.getCurrentAllowedDomains()).toEqual(['second.com'])
|
||||
})
|
||||
|
||||
it('resets cleanly when reset is invoked', async () => {
|
||||
const team = await teamService.ensureDefaultTeam()
|
||||
await sandboxService.ensureInitialized({ teamId: team.id })
|
||||
expect(sandboxService.isInitialized()).toBe(true)
|
||||
|
||||
await sandboxService.reset()
|
||||
expect(sandboxService.isInitialized()).toBe(false)
|
||||
expect(mockProvider.resetCallCount).toBe(1)
|
||||
expect(sandboxService.getCurrentAllowedDomains()).toEqual([])
|
||||
})
|
||||
})
|
||||
@@ -1,224 +0,0 @@
|
||||
import * as fs from 'fs'
|
||||
import * as os from 'os'
|
||||
import * as path from 'path'
|
||||
import { logger } from '../logger'
|
||||
import { prisma } from '@shumai/db'
|
||||
import type {
|
||||
SandboxProvider,
|
||||
SandboxConfig,
|
||||
SandboxAskCallback,
|
||||
WrapCommandOptions,
|
||||
} from './types'
|
||||
import { LocalSandboxProvider } from './providers/local-provider'
|
||||
|
||||
export class SandboxService {
|
||||
private provider: SandboxProvider
|
||||
private isInit = false
|
||||
private initPromise: Promise<void> | null = null
|
||||
private currentAllowedDomains: string[] = []
|
||||
private teamId: string | null = null
|
||||
private lastBlockedHost = ''
|
||||
private syncQueue: Promise<void> = Promise.resolve()
|
||||
|
||||
constructor(provider?: SandboxProvider) {
|
||||
this.provider = provider ?? new LocalSandboxProvider()
|
||||
}
|
||||
|
||||
setProvider(provider: SandboxProvider): void {
|
||||
this.provider = provider
|
||||
this.isInit = false
|
||||
this.initPromise = null
|
||||
this.currentAllowedDomains = []
|
||||
this.lastBlockedHost = ''
|
||||
this.syncQueue = Promise.resolve()
|
||||
}
|
||||
|
||||
getProvider(): SandboxProvider {
|
||||
return this.provider
|
||||
}
|
||||
|
||||
isInitialized(): boolean {
|
||||
return this.isInit && this.provider.isInitialized()
|
||||
}
|
||||
|
||||
getCurrentAllowedDomains(): string[] {
|
||||
return [...this.currentAllowedDomains]
|
||||
}
|
||||
|
||||
async ensureInitialized(initialConfig?: {
|
||||
allowedDomains?: string[]
|
||||
teamId?: string
|
||||
}): Promise<void> {
|
||||
if (this.isInitialized()) return
|
||||
|
||||
if (this.initPromise) {
|
||||
await this.initPromise
|
||||
return
|
||||
}
|
||||
|
||||
this.initPromise = (async () => {
|
||||
try {
|
||||
let teamId = initialConfig?.teamId ?? this.teamId
|
||||
if (!teamId) {
|
||||
const defaultTeam = await prisma.team.findFirst({ select: { id: true } })
|
||||
teamId = defaultTeam?.id ?? null
|
||||
}
|
||||
this.teamId = teamId
|
||||
|
||||
let allowedDomains = initialConfig?.allowedDomains
|
||||
if (!allowedDomains && teamId) {
|
||||
const sandbox = await prisma.sandbox.findUnique({
|
||||
where: { teamId },
|
||||
})
|
||||
allowedDomains = sandbox?.networkSandboxEnabled ? sandbox.allowedDomains : ['*']
|
||||
}
|
||||
allowedDomains = allowedDomains ?? ['*']
|
||||
|
||||
const piDir = path.join(process.cwd(), '.pi')
|
||||
if (!fs.existsSync(piDir)) fs.mkdirSync(piDir, { recursive: true })
|
||||
|
||||
const callback: SandboxAskCallback = async ({ host }) => {
|
||||
return this.handleAskCallback(host)
|
||||
}
|
||||
|
||||
const config: SandboxConfig = {
|
||||
network: {
|
||||
allowedDomains,
|
||||
deniedDomains: [],
|
||||
},
|
||||
filesystem: {
|
||||
denyRead: ['.env', '.env.*', '*.pem', '*.key'],
|
||||
allowWrite: [piDir, os.tmpdir()],
|
||||
denyWrite: ['.env', '.env.*', '*.pem', '*.key'],
|
||||
},
|
||||
enableWeakerNestedSandbox: process.env.ENABLE_WEAKER_NESTED_SANDBOX === 'true',
|
||||
}
|
||||
|
||||
await this.provider.initialize(config, callback)
|
||||
this.currentAllowedDomains = [...allowedDomains]
|
||||
this.isInit = true
|
||||
logger.debug({ allowedDomains }, 'Sandbox initialized successfully')
|
||||
} catch (err) {
|
||||
logger.error({ err }, 'Failed to initialize sandbox')
|
||||
throw err
|
||||
} finally {
|
||||
this.initPromise = null
|
||||
}
|
||||
})()
|
||||
|
||||
await this.initPromise
|
||||
}
|
||||
|
||||
async syncAllowedDomains(incomingDomains: string[], teamId?: string): Promise<void> {
|
||||
const runSync = async () => {
|
||||
if (teamId) {
|
||||
this.teamId = teamId
|
||||
}
|
||||
|
||||
if (!this.isInitialized()) {
|
||||
await this.ensureInitialized({ allowedDomains: incomingDomains, teamId })
|
||||
}
|
||||
|
||||
if (this.areDomainsEqual(this.currentAllowedDomains, incomingDomains)) {
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
await this.provider.updateConfig({
|
||||
network: {
|
||||
allowedDomains: incomingDomains,
|
||||
deniedDomains: [],
|
||||
},
|
||||
})
|
||||
this.currentAllowedDomains = [...incomingDomains]
|
||||
logger.debug({ incomingDomains }, 'Sandbox allowed domains hot-reloaded successfully')
|
||||
} catch (err) {
|
||||
logger.error({ err, incomingDomains }, 'Failed to hot-reload sandbox allowed domains')
|
||||
throw err
|
||||
}
|
||||
}
|
||||
|
||||
const previous = this.syncQueue
|
||||
const current = (async () => {
|
||||
try {
|
||||
await previous
|
||||
} catch {
|
||||
// Ignore errors from previous syncs so subsequent calls proceed
|
||||
}
|
||||
return runSync()
|
||||
})()
|
||||
|
||||
this.syncQueue = current
|
||||
await current
|
||||
}
|
||||
|
||||
async wrapCommand(command: string, options?: WrapCommandOptions): Promise<string> {
|
||||
if (!this.isInitialized()) {
|
||||
await this.ensureInitialized()
|
||||
}
|
||||
return this.provider.wrapCommand(command, options)
|
||||
}
|
||||
|
||||
getBlockedHost(commandId?: string): string {
|
||||
if (commandId) {
|
||||
return this.provider.getBlockedHostForCommand(commandId) ?? ''
|
||||
}
|
||||
return this.lastBlockedHost
|
||||
}
|
||||
|
||||
clearBlockedHost(): void {
|
||||
this.lastBlockedHost = ''
|
||||
}
|
||||
|
||||
async reset(): Promise<void> {
|
||||
await this.provider.reset()
|
||||
this.isInit = false
|
||||
this.initPromise = null
|
||||
this.currentAllowedDomains = []
|
||||
this.lastBlockedHost = ''
|
||||
this.teamId = null
|
||||
this.syncQueue = Promise.resolve()
|
||||
}
|
||||
|
||||
private async handleAskCallback(host: string): Promise<boolean> {
|
||||
const teamId = this.teamId
|
||||
if (teamId) {
|
||||
try {
|
||||
const sandbox = await prisma.sandbox.findUnique({
|
||||
where: { teamId },
|
||||
})
|
||||
if (sandbox && !sandbox.networkSandboxEnabled) {
|
||||
return true
|
||||
}
|
||||
const pendingDomains = sandbox?.pendingDomains || []
|
||||
if (!pendingDomains.includes(host)) {
|
||||
await prisma.sandbox.upsert({
|
||||
where: { teamId },
|
||||
create: {
|
||||
teamId,
|
||||
pendingDomains: [host],
|
||||
},
|
||||
update: {
|
||||
pendingDomains: {
|
||||
push: host,
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
} catch (err) {
|
||||
logger.error({ err, host, teamId }, 'Failed to update sandbox pending domains')
|
||||
}
|
||||
}
|
||||
|
||||
this.lastBlockedHost = host
|
||||
return false
|
||||
}
|
||||
|
||||
private areDomainsEqual(a: string[], b: string[]): boolean {
|
||||
if (a.length !== b.length) return false
|
||||
const setA = new Set(a)
|
||||
return b.every((domain) => setA.has(domain))
|
||||
}
|
||||
}
|
||||
|
||||
export const sandboxService = new SandboxService()
|
||||
@@ -1,31 +0,0 @@
|
||||
export interface SandboxNetworkConfig {
|
||||
allowedDomains: string[]
|
||||
deniedDomains: string[]
|
||||
}
|
||||
|
||||
export interface SandboxFilesystemConfig {
|
||||
denyRead: string[]
|
||||
allowWrite: string[]
|
||||
denyWrite: string[]
|
||||
}
|
||||
|
||||
export interface SandboxConfig {
|
||||
network: SandboxNetworkConfig
|
||||
filesystem: SandboxFilesystemConfig
|
||||
enableWeakerNestedSandbox?: boolean
|
||||
}
|
||||
|
||||
export type SandboxAskCallback = (params: { host: string; port?: number }) => Promise<boolean>
|
||||
|
||||
export interface WrapCommandOptions {
|
||||
commandId?: string
|
||||
}
|
||||
|
||||
export interface SandboxProvider {
|
||||
initialize(config: SandboxConfig, callback?: SandboxAskCallback): Promise<void>
|
||||
wrapCommand(command: string, options?: WrapCommandOptions): Promise<string>
|
||||
updateConfig(config: Partial<SandboxConfig>): Promise<void>
|
||||
reset(): Promise<void>
|
||||
isInitialized(): boolean
|
||||
getBlockedHostForCommand(commandId: string): string | null
|
||||
}
|
||||
@@ -481,65 +481,6 @@ describe('TeamService', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('should create sandbox when team is created', async () => {
|
||||
const user = await prisma.user.create({
|
||||
data: { name: 'Test User', email: `test-sandbox-${Date.now()}@example.com`, password: 'pw' },
|
||||
})
|
||||
|
||||
const team = await teamService.createTeam(user, {
|
||||
name: 'Sandbox Team',
|
||||
})
|
||||
|
||||
const sandbox = await prisma.sandbox.findUnique({
|
||||
where: { teamId: team.id },
|
||||
})
|
||||
|
||||
expect(sandbox).toBeDefined()
|
||||
expect(sandbox?.allowedDomains).toContain('github.com')
|
||||
})
|
||||
|
||||
it('should get and update sandbox settings', async () => {
|
||||
const user = await prisma.user.create({
|
||||
data: { name: 'Test User', email: `test-settings-${Date.now()}@example.com`, password: 'pw' },
|
||||
})
|
||||
const team = await teamService.createTeam(user, {
|
||||
name: 'Settings Team',
|
||||
})
|
||||
|
||||
// Should have default settings (networkSandboxEnabled is false by default for new teams)
|
||||
const settings = await teamService.getSandboxSettings(team.id)
|
||||
expect(settings.networkSandboxEnabled).toBe(false)
|
||||
expect(settings.allowedDomains).toContain('github.com')
|
||||
expect(settings.pendingDomains).toEqual([])
|
||||
|
||||
// Should update settings
|
||||
const updated = await teamService.updateSandboxSettings(team.id, {
|
||||
networkSandboxEnabled: true,
|
||||
allowedDomains: ['example.com'],
|
||||
pendingDomains: ['pending.com'],
|
||||
})
|
||||
expect(updated.networkSandboxEnabled).toBe(true)
|
||||
expect(updated.allowedDomains).toEqual(['example.com'])
|
||||
expect(updated.pendingDomains).toEqual(['pending.com'])
|
||||
|
||||
// Should persist to DB
|
||||
const persisted = await teamService.getSandboxSettings(team.id)
|
||||
expect(persisted.networkSandboxEnabled).toBe(true)
|
||||
expect(persisted.allowedDomains).toEqual(['example.com'])
|
||||
expect(persisted.pendingDomains).toEqual(['pending.com'])
|
||||
})
|
||||
|
||||
it('should return default settings if sandbox is missing for old team', async () => {
|
||||
const team = await prisma.team.create({
|
||||
data: { name: 'Old Team' },
|
||||
})
|
||||
|
||||
const settings = await teamService.getSandboxSettings(team.id)
|
||||
expect(settings.networkSandboxEnabled).toBe(false)
|
||||
expect(settings.allowedDomains).toEqual([])
|
||||
expect(settings.pendingDomains).toEqual([])
|
||||
})
|
||||
|
||||
describe('hasWritableRoleInAnyTeam', () => {
|
||||
it('should return true for user with owner role', async () => {
|
||||
const user = await prisma.user.create({
|
||||
|
||||
@@ -7,7 +7,6 @@ import { getAvatarUrl } from '@shumai/core/src/user/avatar'
|
||||
import { HTTPException } from 'hono/http-exception'
|
||||
import { getAllowedAgentRoles } from '@shumai/core/src/agent/permissions'
|
||||
import { AgentService, agentService } from '@shumai/core/src/agent/agent'
|
||||
import { sandboxService } from '@shumai/core/src/sandbox/sandbox-service'
|
||||
import { turntableService } from '@shumai/core/src/turntable/turntable'
|
||||
import { gotenbergService } from '@shumai/core/src/gotenberg/gotenberg'
|
||||
import {
|
||||
@@ -24,8 +23,6 @@ import {
|
||||
TeamInfo,
|
||||
GetMeResponse,
|
||||
UserInfo,
|
||||
SandboxSettings,
|
||||
UpdateSandboxSettingsRequest,
|
||||
TeamSettingsResponse,
|
||||
AutoArchiveMethod,
|
||||
AutoArchiveSettings,
|
||||
@@ -50,7 +47,6 @@ export class TeamService {
|
||||
threads: 0,
|
||||
},
|
||||
},
|
||||
sandbox: { create: {} },
|
||||
},
|
||||
})
|
||||
await providerService.initBuiltinProviders(team.id)
|
||||
@@ -68,7 +64,6 @@ export class TeamService {
|
||||
name: req.name,
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
settings: {} as any,
|
||||
sandbox: { create: {} },
|
||||
},
|
||||
})
|
||||
|
||||
@@ -513,50 +508,6 @@ export class TeamService {
|
||||
return this.getSettings(teamId)
|
||||
}
|
||||
|
||||
async getSandboxSettings(teamId: string): Promise<SandboxSettings> {
|
||||
const sandbox = await prisma.sandbox.findUnique({
|
||||
where: { teamId },
|
||||
})
|
||||
return {
|
||||
networkSandboxEnabled: sandbox?.networkSandboxEnabled ?? false,
|
||||
allowedDomains: sandbox?.allowedDomains || [],
|
||||
pendingDomains: sandbox?.pendingDomains || [],
|
||||
}
|
||||
}
|
||||
|
||||
async updateSandboxSettings(
|
||||
teamId: string,
|
||||
settings: UpdateSandboxSettingsRequest,
|
||||
): Promise<SandboxSettings> {
|
||||
const sandbox = await prisma.sandbox.upsert({
|
||||
where: { teamId },
|
||||
create: {
|
||||
teamId,
|
||||
networkSandboxEnabled: settings.networkSandboxEnabled ?? false,
|
||||
allowedDomains: settings.allowedDomains ?? [],
|
||||
pendingDomains: settings.pendingDomains ?? [],
|
||||
},
|
||||
update: {
|
||||
...(settings.networkSandboxEnabled !== undefined && {
|
||||
networkSandboxEnabled: settings.networkSandboxEnabled,
|
||||
}),
|
||||
...(settings.allowedDomains !== undefined && {
|
||||
allowedDomains: settings.allowedDomains,
|
||||
}),
|
||||
...(settings.pendingDomains !== undefined && {
|
||||
pendingDomains: settings.pendingDomains,
|
||||
}),
|
||||
},
|
||||
})
|
||||
const effectiveDomains = sandbox.networkSandboxEnabled ? sandbox.allowedDomains : ['*']
|
||||
await sandboxService.syncAllowedDomains(effectiveDomains, teamId)
|
||||
return {
|
||||
networkSandboxEnabled: sandbox.networkSandboxEnabled,
|
||||
allowedDomains: sandbox.allowedDomains,
|
||||
pendingDomains: sandbox.pendingDomains,
|
||||
}
|
||||
}
|
||||
|
||||
async getSignupInfo() {
|
||||
const count = await prisma.user.count()
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@shumai/db",
|
||||
"version": "0.6.6",
|
||||
"version": "0.7.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "./src/index.ts",
|
||||
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
/*
|
||||
Warnings:
|
||||
|
||||
- You are about to drop the `sandboxes` table. If the table is not empty, all the data it contains will be lost.
|
||||
|
||||
*/
|
||||
-- DropForeignKey
|
||||
ALTER TABLE "sandboxes" DROP CONSTRAINT "sandboxes_team_id_fkey";
|
||||
|
||||
-- AlterTable
|
||||
ALTER TABLE "agents" ADD COLUMN "runner_id" TEXT;
|
||||
|
||||
-- DropTable
|
||||
DROP TABLE "sandboxes";
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "runners" (
|
||||
"id" TEXT NOT NULL,
|
||||
"name" TEXT NOT NULL,
|
||||
"description" TEXT,
|
||||
"key_prefix" TEXT NOT NULL,
|
||||
"key_hash" TEXT NOT NULL,
|
||||
"is_default" BOOLEAN NOT NULL DEFAULT false,
|
||||
"last_connected_at" TIMESTAMP(3),
|
||||
"last_disconnected_at" TIMESTAMP(3),
|
||||
"created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updated_at" TIMESTAMP(3) NOT NULL,
|
||||
"team_id" TEXT NOT NULL,
|
||||
|
||||
CONSTRAINT "runners_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "runners_key_hash_key" ON "runners"("key_hash");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE INDEX "runners_team_id_idx" ON "runners"("team_id");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "runners_team_id_name_key" ON "runners"("team_id", "name");
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "runners" ADD CONSTRAINT "runners_team_id_fkey" FOREIGN KEY ("team_id") REFERENCES "teams"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "agents" ADD CONSTRAINT "agents_runner_id_fkey" FOREIGN KEY ("runner_id") REFERENCES "runners"("id") ON DELETE SET NULL ON UPDATE CASCADE;
|
||||
+2
@@ -0,0 +1,2 @@
|
||||
-- Enforce single default runner per team at the database level
|
||||
CREATE UNIQUE INDEX "runners_one_default_per_team_idx" ON "runners"("team_id") WHERE "is_default" = true;
|
||||
@@ -342,7 +342,7 @@ model Team {
|
||||
providers Provider[] @relation("TeamToProvider")
|
||||
skills Skill[] @relation("TeamToSkill")
|
||||
mcpServers McpServer[] @relation("TeamToMcpServer")
|
||||
sandbox Sandbox? @relation("TeamToSandbox")
|
||||
runners Runner[] @relation("TeamToRunner")
|
||||
agents Agent[] @relation("TeamToAgent")
|
||||
aiUsages AiUsage[] @relation("TeamToAiUsage")
|
||||
auditLogs AuditLog[] @relation("TeamToAuditLog")
|
||||
@@ -355,18 +355,26 @@ model Team {
|
||||
@@map("teams")
|
||||
}
|
||||
|
||||
model Sandbox {
|
||||
id String @id @default(ulid())
|
||||
networkSandboxEnabled Boolean @default(false) @map("network_sandbox_enabled")
|
||||
allowedDomains String[] @default(["npmjs.org", "*.npmjs.org", "registry.npmjs.org", "registry.yarnpkg.com", "pypi.org", "*.pypi.org", "github.com", "*.github.com", "api.github.com", "raw.githubusercontent.com"])
|
||||
pendingDomains String[] @default([]) @map("pending_domains")
|
||||
createdAt DateTime @default(now()) @map("created_at")
|
||||
updatedAt DateTime @updatedAt @map("updated_at")
|
||||
model Runner {
|
||||
id String @id @default(ulid())
|
||||
name String
|
||||
description String?
|
||||
keyPrefix String @map("key_prefix")
|
||||
keyHash String @unique @map("key_hash")
|
||||
isDefault Boolean @default(false) @map("is_default")
|
||||
lastConnectedAt DateTime? @map("last_connected_at")
|
||||
lastDisconnectedAt DateTime? @map("last_disconnected_at")
|
||||
createdAt DateTime @default(now()) @map("created_at")
|
||||
updatedAt DateTime @updatedAt @map("updated_at")
|
||||
|
||||
teamId String @unique @map("team_id")
|
||||
team Team @relation("TeamToSandbox", fields: [teamId], references: [id], onDelete: Cascade)
|
||||
teamId String @map("team_id")
|
||||
team Team @relation("TeamToRunner", fields: [teamId], references: [id], onDelete: Cascade)
|
||||
|
||||
@@map("sandboxes")
|
||||
agents Agent[] @relation("RunnerToAgent")
|
||||
|
||||
@@unique([teamId, name])
|
||||
@@index([teamId])
|
||||
@@map("runners")
|
||||
}
|
||||
|
||||
model Skill {
|
||||
@@ -891,6 +899,8 @@ model Agent {
|
||||
provider Provider? @relation("ProviderToAgent", fields: [providerId], references: [id], onDelete: SetNull)
|
||||
modelId String? @map("model_id")
|
||||
modelRef Model? @relation("ModelToAgent", fields: [modelId], references: [id], onDelete: SetNull)
|
||||
runnerId String? @map("runner_id")
|
||||
runner Runner? @relation("RunnerToAgent", fields: [runnerId], references: [id], onDelete: SetNull)
|
||||
|
||||
teamId String @map("team_id")
|
||||
team Team @relation("TeamToAgent", fields: [teamId], references: [id], onDelete: Cascade)
|
||||
|
||||
@@ -39,3 +39,68 @@ export async function acquireAdvisoryLock(key: string, signal?: AbortSignal) {
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/** Hold a shared server lease, with a non-blocking upgrade to exclusive ownership. */
|
||||
export async function tryAcquireUpgradeableAdvisoryLock(key: string, exclusive: boolean) {
|
||||
const client = await locks.connect()
|
||||
const lost = new AbortController()
|
||||
const onError = (error: Error) => lost.abort(error)
|
||||
client.on('error', onError)
|
||||
|
||||
const tryExclusive = async () => {
|
||||
const result = await client.query<{ acquired: boolean }>(
|
||||
'SELECT pg_try_advisory_lock(hashtextextended($1, 0)) AS acquired',
|
||||
[key],
|
||||
)
|
||||
return result.rows[0]?.acquired === true
|
||||
}
|
||||
let acquired = false
|
||||
try {
|
||||
if (exclusive && !(await tryExclusive())) return null
|
||||
const result = await client.query<{ acquired: boolean }>(
|
||||
'SELECT pg_try_advisory_lock_shared(hashtextextended($1, 0)) AS acquired',
|
||||
[key],
|
||||
)
|
||||
lost.signal.throwIfAborted()
|
||||
acquired = result.rows[0]?.acquired === true
|
||||
if (!acquired) return null
|
||||
} finally {
|
||||
if (!acquired) {
|
||||
client.off('error', onError)
|
||||
client.release(true)
|
||||
}
|
||||
}
|
||||
|
||||
let released = false
|
||||
const assertHeld = () => {
|
||||
if (released) throw new Error('The server lease has been released')
|
||||
lost.signal.throwIfAborted()
|
||||
}
|
||||
return {
|
||||
signal: lost.signal,
|
||||
get exclusive() {
|
||||
return exclusive
|
||||
},
|
||||
async tryUpgrade() {
|
||||
assertHeld()
|
||||
if (!exclusive) exclusive = await tryExclusive()
|
||||
return exclusive
|
||||
},
|
||||
async downgrade() {
|
||||
assertHeld()
|
||||
if (!exclusive) return
|
||||
await client.query('SELECT pg_advisory_unlock(hashtextextended($1, 0))', [key])
|
||||
exclusive = false
|
||||
},
|
||||
async release() {
|
||||
if (released) return
|
||||
released = true
|
||||
try {
|
||||
if (!lost.signal.aborted) await client.query('SELECT pg_advisory_unlock_all()')
|
||||
} finally {
|
||||
client.off('error', onError)
|
||||
client.release(true)
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user