Compare commits

...
5 Commits
Author SHA1 Message Date
yiling fb5cc441c7 feat(webui): add Korean translations for agent runners 2026-11-18 12:41:22 +08:00
yiling a9281cbc75 fix(runner): enforce ownership and safe reconnects
Gate dispatch on reconciliation, retain cancellations across disconnects, and stop replaced clients. Guard the single-server runner topology with a database lease. Restage missing generated inputs, report signal exits, and bound generated-file cleanup.
2026-11-18 12:04:29 +08:00
yiling fd4dc211f8 feat(cli): discover Bash for Windows runners
Use Pi-style Bash discovery and launch behavior, exercise the packaged CLI shell, and wait for pending ZIP metadata reads before closing cancelled downloads.
2026-11-18 10:57:04 +08:00
yiling 776f6962b6 chore(cli): upgrade @anthropic-ai/sandbox-runtime to 0.0.83 2026-11-18 10:19:06 +08:00
Yiling-J 3e24fe593e feat(cli): add runner start with bundled SRT sandboxing (#535)
* refactor(agent): expose runner bash without skill gating

* feat(cli): add runner start with optional SRT sandboxing

* refactor(cli): remove runner Python dependency

* feat(cli): bundle SRT and use platform-native runner shells

* refactor(cli): delegate runner processes to Execa
2026-11-18 09:44:54 +08:00
50 changed files with 3483 additions and 403 deletions
+3 -1
View File
@@ -8,7 +8,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Added
- **agent**: Support binding external Runners per team so agents can execute shell commands and exchange files securely over WebSocket without requiring a built-in server sandbox, and allow assigning a default or dedicated Runner per agent in Team Settings.
- **cli**: Run remote shell commands and exchange files with `shumai-cli runner start` on macOS, Linux, and Windows, with optional SRT sandboxing. On Windows, the runner automatically uses Bash when available and falls back to Command Prompt.
- **agent**: Connect remote runners on single-server deployments to run shell commands and exchange files. Assign the team's default runner or a dedicated runner in Agent Settings. A runner's workspace is shared by everyone who can use it.
- **i18n**: Add Korean as a display language, selectable on the sign-up page and in team settings.
### Fixed
@@ -17,6 +18,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Changed
- **agent**: Make bash available to editors and reviewers as soon as a runner is configured, without first reading a skill.
- **agent**: ⚠️ **Behavior Change**: Replace the built-in server-side bash sandbox and domain allowlist settings with external team Runners; agent `bash` and `download_asset` tools are enabled automatically once at least one Runner is configured for the team, while domain and OS-level sandboxing rules move to the Runner side.
- **cli**: Show copyable commands and shell configuration instructions when the standalone CLI installation directory is missing from PATH.
+39 -1
View File
@@ -86,6 +86,44 @@ export SHUMAI_ALLOW_DELETE="true"
---
## Agent runner
Create a runner in **Settings → Agent Runners** and copy its one-time key. On macOS, Linux, or Windows, start it in an existing dedicated workspace:
```sh
export SHUMAI_API_SERVER="https://shumai.example.com"
export SHUMAI_RUNNER_KEY="your-runner-key"
shumai-cli runner start --cwd /path/to/workspace
```
For Windows PowerShell:
```powershell
$env:SHUMAI_API_SERVER = "https://shumai.example.com"
$env:SHUMAI_RUNNER_KEY = "your-runner-key"
shumai-cli runner start --cwd C:\workspace
```
Use `--url` or `--key` to override those variables. A runner key is separate from `SHUMAI_API_KEY`. The runner stays in the foreground; Ctrl+C stops it and its active commands. Remote servers require HTTPS; localhost and `host.docker.internal` support HTTP for development. No inbound port is opened.
Commands use `/bin/sh` on macOS and Linux. On Windows, the runner checks for Git Bash in the standard installation directories, then for `bash.exe` on PATH, then falls back to `cmd.exe`. It prints the selected shell at startup and reports it to the agent. Commands run with the local user's permissions unless SRT is requested.
[Anthropic Sandbox Runtime](https://github.com/anthropics/sandbox-runtime) is bundled with the CLI. Install its platform prerequisites (Linux: bubblewrap, socat, and ripgrep; macOS: ripgrep), then pass its native JSON settings:
```sh
shumai-cli runner start --srt sandbox.json
```
Windows sandboxing uses SRT's alpha Windows support. Use Git Bash if your selected shell is the legacy WSL Bash launcher. Run the one-time SRT setup before starting a sandboxed runner (it requests administrator permission):
```powershell
npx @anthropic-ai/sandbox-runtime windows-install
```
SRT settings are read and validated at startup. Invalid settings or unavailable sandbox prerequisites stop startup; commands never fall back to unrestricted execution. Only shell commands and their children are sandboxed. Runner connections and server-directed file transfers stay outside SRT, and no server domain is automatically allowed. The shared SRT runtime is cleaned up when the runner stops.
See [Agent Runners](../../docs/using-shumai/agent-sandbox.mdx) for configuration, Docker usage, limits, and security boundaries.
## Usage
> [!IMPORTANT]
@@ -330,7 +368,7 @@ bun run build:cli --target darwin-arm64 # or darwin-x64, linux-arm64, linux-x64,
bun run test:cli-release
```
Without `--target`, the build produces all six targets. Output defaults to `out/cli-release`; both commands accept `--out /absolute/path`. Unix archives require `tar`; creating Windows archives on Unix also requires `zip`. The test runs the generated installer against local HTTP fixtures, then authenticates, creates a folder, uploads and downloads random bytes, and checks failure recovery with no runtime on the executable's PATH.
Without `--target`, the build produces all six targets. Output defaults to `out/cli-release`. Set `--out /absolute/path` on either command to change it. Unix archives require `tar`. Creating Windows archives on Unix also requires `zip`. The test exercises installation, runner shell quoting, authenticated API calls, file transfers, and failure recovery. Install Git Bash to run the quoting check on Windows. With SRT configured, pass `--srt /absolute/path/to/settings.json` to repeat the quoting check in the sandbox.
The Native CLI workflow builds and tests each target on a matching runner. The npm release workflow assembles those tested archives, stages a draft GitHub Release, publishes npm packages, then publishes the completed release. Its `validate_only` dispatch input builds and tests without publishing. Published assets are verified and preserved on retries; draft assets can be replaced while resuming a failed run. Prerelease npm packages use the `next` tag and prerelease GitHub Releases do not become latest.
+24
View File
@@ -0,0 +1,24 @@
The Windows Bash discovery and related tests include material adapted from Pi.
https://github.com/earendil-works/pi
MIT License
Copyright (c) 2025 Mario Zechner
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+10
View File
@@ -7,13 +7,23 @@
"shumai-cli": "./src/index.ts"
},
"dependencies": {
"@anthropic-ai/sandbox-runtime": "0.0.83",
"@shumai/api": "workspace:*",
"@shumai/dtos": "workspace:*",
"@shumai/operations": "workspace:*",
"@shumai/tableprinter": "workspace:*",
"commander": "15.0.0",
"execa": "10.2.1",
"hono": "4.13.5",
"ulid": "3.0.2",
"ws": "8.22.2",
"yauzl": "3.4.0",
"zod": "4.5.4"
},
"devDependencies": {
"@types/adm-zip": "0.5.8",
"@types/ws": "8.18.2",
"@types/yauzl": "3.4.0",
"adm-zip": "0.6.1"
}
}
+12
View File
@@ -7,6 +7,7 @@ import { cliExtensions } from './operations/extensions'
import { registerOperationCommands } from './operations/generate'
import type { Output } from './operations/render'
import { version } from '../package.json'
import { startRunner } from './runner/client'
export interface ProgramOptions {
callOperation?: CallOperation
@@ -43,6 +44,17 @@ export function createProgram(options: ProgramOptions = {}): Command {
extensions: cliExtensions,
})
program
.command('runner')
.description('Connect a local command runner to Shumai.')
.command('start')
.description('Run in the foreground using a key created in Settings → Agent Runners.')
.option('--url <url>', 'Shumai server URL (or SHUMAI_API_SERVER).')
.option('--key <key>', 'Runner key (prefer SHUMAI_RUNNER_KEY to avoid shell history).')
.option('--cwd <directory>', 'Runner workspace directory (defaults to the current directory).')
.option('--srt <file>', 'Sandbox commands with bundled SRT using this JSON settings file.')
.action(startRunner)
program
.command('upload')
.description('Upload a local file or folder to a parent folder.')
+195
View File
@@ -0,0 +1,195 @@
import { mkdtemp, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import path from 'node:path'
import { WebSocketServer, type WebSocket } from 'ws'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { runnerToServerMessageSchema, type RunnerToServerMessage } from '@shumai/dtos'
import { RunnerClient } from './client'
import { RunnerFiles } from './files'
import { RunnerShell } from './shell'
describe('runner WebSocket client', () => {
let cwd: string
let server: WebSocketServer
let runner: RunnerClient
let running: Promise<void>
let socket: WebSocket
let messages: RunnerToServerMessage[]
let welcomeInFlight: string[]
let connections: number
let replacePrevious: boolean
let connectRunner: () => RunnerClient
beforeEach(async () => {
cwd = await mkdtemp(path.join(tmpdir(), 'shumai-client-test-'))
messages = []
welcomeInFlight = []
connections = 0
replacePrevious = false
server = new WebSocketServer({ host: '127.0.0.1', port: 0 })
await new Promise<void>((resolve) => server.once('listening', resolve))
const address = server.address()
if (!address || typeof address === 'string') throw new Error('No WebSocket address')
server.on('connection', (connection, request) => {
expect(request.headers.authorization).toBe('Bearer runner-key')
if (replacePrevious) socket.close(4000, 'Replaced by a newer connection')
socket = connection
connections++
connection.on('message', (raw) =>
messages.push(runnerToServerMessageSchema.parse(JSON.parse(raw.toString()))),
)
connection.send(
JSON.stringify({
type: 'welcome',
runnerId: 'runner',
runnerName: 'Test',
serverTime: Date.now(),
inFlightExecutionIds: welcomeInFlight,
}),
)
})
connectRunner = () => {
const files = new RunnerFiles(cwd)
return new RunnerClient(
{
url: new URL(`ws://127.0.0.1:${address.port}/api/runners/ws`),
key: 'runner-key',
cwd,
srt: undefined,
},
files,
new RunnerShell(files),
() => {},
)
}
runner = connectRunner()
running = runner.run()
// Attach an error handler immediately, including for tests exercising terminal errors.
void running.catch(() => {})
await vi.waitFor(() =>
expect(messages.some((message) => message.type === 'reconcile')).toBe(true),
)
})
afterEach(async () => {
runner.stop()
await running.catch(() => {})
for (const connection of server.clients) connection.terminate()
await new Promise<void>((resolve) => server.close(() => resolve()))
await rm(cwd, { recursive: true, force: true })
})
const send = (message: unknown) => socket.send(JSON.stringify(message))
it('responds to heartbeat, executes a command once and streams byte offsets', async () => {
expect(messages).toContainEqual(
expect.objectContaining({
type: 'reconcile',
environment: { platform: process.platform, shell: '/bin/sh' },
}),
)
send({ type: 'ping', ts: 123 })
const command = {
type: 'exec.start',
executionId: 'one',
command: 'printf "世界"; printf "error" >&2',
}
send(command)
send(command)
await vi.waitFor(() =>
expect(messages.some((message) => message.type === 'exec.result')).toBe(true),
)
expect(messages).toContainEqual({ type: 'pong', ts: 123 })
expect(messages.filter((message) => message.type === 'exec.result')).toHaveLength(1)
const chunks = messages.filter((message) => message.type === 'exec.output')
expect(chunks.map((chunk) => chunk.data).join('')).toContain('世界')
let offset = 0
for (const chunk of chunks) {
expect(chunk.offset).toBe(offset)
offset += Buffer.byteLength(chunk.data)
}
})
it('cancels an active command through the protocol', async () => {
send({ type: 'exec.start', executionId: 'cancel', command: 'echo ready; sleep 30' })
await vi.waitFor(() =>
expect(messages.some((message) => message.type === 'exec.output')).toBe(true),
)
send({ type: 'exec.cancel', executionId: 'cancel' })
await vi.waitFor(() =>
expect(messages).toContainEqual(
expect.objectContaining({ type: 'exec.result', executionId: 'cancel', aborted: true }),
),
)
})
it('reconnects and replays output before completing an in-flight command', async () => {
send({
type: 'exec.start',
executionId: 'reconnect',
command: 'echo before; sleep 0.2; echo after',
})
await vi.waitFor(() =>
expect(messages.some((message) => message.type === 'exec.output')).toBe(true),
)
welcomeInFlight = ['reconnect']
socket.terminate()
messages = []
await vi.waitFor(() => expect(connections).toBe(2), { timeout: 5000 })
await vi.waitFor(() =>
expect(messages.some((message) => message.type === 'reconcile')).toBe(true),
)
const reconciliation = messages.findIndex((message) => message.type === 'reconcile')
expect(
messages
.slice(0, reconciliation)
.filter((message) => message.type === 'exec.output')
.map((message) => message.data)
.join(''),
).toContain('after')
expect(messages[reconciliation]).toMatchObject({
completedExecutions: [expect.objectContaining({ executionId: 'reconnect', exitCode: 0 })],
})
})
it('handles file requests and returns a cached result for a duplicate request', async () => {
const request = { type: 'file.stat', executionId: 'stat', srcPath: '.pi/missing' }
send(request)
await vi.waitFor(() =>
expect(messages.some((message) => message.type === 'file.stat.result')).toBe(true),
)
send(request)
await vi.waitFor(() =>
expect(messages.filter((message) => message.type === 'file.stat.result')).toHaveLength(2),
)
expect(messages.at(-1)).toMatchObject({ ok: true, exists: false })
})
it('stops instead of reconnecting when the runner key is revoked', async () => {
socket.close(1008, 'revoked')
await expect(running).rejects.toThrow('revoked')
expect(connections).toBe(1)
})
it('stops the superseded client when a second runner connects with the same key', async () => {
replacePrevious = true
let replaced: unknown
void running.catch((error: unknown) => {
replaced = error
})
const replacement = connectRunner()
const replacementRun = replacement.run()
void replacementRun.catch(() => {})
try {
await vi.waitFor(() => expect(replaced).toBeInstanceOf(Error))
await expect(running).rejects.toThrow('replaced')
expect(connections).toBe(2)
} finally {
replacement.stop()
await replacementRun.catch(() => {})
}
})
it('rejects invalid messages and path-like execution IDs', async () => {
send({ type: 'exec.start', executionId: '../escape', command: 'echo unsafe' })
await expect(running).rejects.toThrow('invalid runner protocol')
expect(messages.some((message) => message.type === 'exec.output')).toBe(false)
})
})
+313
View File
@@ -0,0 +1,313 @@
import WebSocket from 'ws'
import { setTimeout as delay } from 'node:timers/promises'
import {
RUNNER_REPLACED_CLOSE_CODE,
serverToRunnerMessageSchema,
type RunnerExecOutputMessage,
type RunnerExecResultMessage,
type RunnerToServerMessage,
type ServerToRunnerMessage,
} from '@shumai/dtos'
import { CliError } from '../operations/errors'
import { resolveRunnerConfig, type RunnerOptions } from './config'
import { RunnerFiles, type FileMessage } from './files'
import { RunnerShell } from './shell'
const MAX_REPLAY_BYTES = 1024 * 1024
const MAX_COMPLETED = 16
const MAX_ACTIVE = 8
interface Execution {
controller: AbortController
chunks: RunnerExecOutputMessage[]
bytes: number
offset: number
result?: RunnerExecResultMessage
}
export class RunnerClient {
private socket?: WebSocket
private ready = false
private stopped = new AbortController()
private executions = new Map<string, Execution>()
private transfers = new Map<string, AbortController>()
private completedFiles = new Map<string, RunnerToServerMessage>()
private pending = new Set<Promise<void>>()
constructor(
private readonly config: Awaited<ReturnType<typeof resolveRunnerConfig>>,
private readonly files: RunnerFiles,
private readonly shell: RunnerShell,
private readonly log: (message: string) => void = console.error,
) {}
async run(signal?: AbortSignal): Promise<void> {
const stop = () => this.stop()
signal?.addEventListener('abort', stop, { once: true })
if (signal?.aborted) this.stop()
let attempt = 0
try {
while (!this.stopped.signal.aborted) {
const wasReady = await this.connect()
if (this.stopped.signal.aborted) break
attempt = wasReady ? 0 : Math.min(attempt + 1, 5)
this.log('Runner disconnected; reconnecting…')
await delay(Math.min(1000 * 2 ** attempt, 30_000), undefined, {
signal: this.stopped.signal,
}).catch(() => {})
}
} finally {
signal?.removeEventListener('abort', stop)
this.stop()
await Promise.allSettled(this.pending)
}
}
stop() {
this.stopped.abort()
this.ready = false
this.socket?.terminate()
for (const execution of this.executions.values()) execution.controller.abort()
for (const transfer of this.transfers.values()) transfer.abort()
}
private connect(): Promise<boolean> {
return new Promise((resolve, reject) => {
const socket = new WebSocket(this.config.url, {
headers: { authorization: `Bearer ${this.config.key}` },
handshakeTimeout: 10_000,
maxPayload: 1024 * 1024,
perMessageDeflate: false,
followRedirects: false,
})
this.socket = socket
let welcomed = false
let lastMessage = Date.now()
const heartbeat = setInterval(() => {
if (Date.now() - lastMessage > (welcomed ? 75_000 : 10_000)) socket.terminate()
}, 5_000)
socket.on('error', () => {
/* Close drives reconnect; never print credential-bearing request errors. */
})
socket.on('unexpected-response', (_request, response) => {
response.destroy()
socket.terminate()
if ([401, 403, 404, 426].includes(response.statusCode ?? 0))
reject(
new CliError(
`Runner connection rejected (HTTP ${response.statusCode}). Check the server URL and runner key.`,
),
)
})
socket.on('close', (code) => {
clearInterval(heartbeat)
this.ready = false
if ((code === 1008 || code === RUNNER_REPLACED_CLOSE_CODE) && !this.stopped.signal.aborted)
reject(
new CliError('Runner access was revoked or another connection replaced this runner.'),
)
else resolve(welcomed)
})
socket.on('message', (raw) => {
try {
const parsed = serverToRunnerMessageSchema.parse(JSON.parse(raw.toString()))
if ('executionId' in parsed && !/^[A-Za-z0-9_-]{1,128}$/.test(parsed.executionId))
throw new Error('Invalid execution ID.')
lastMessage = Date.now()
if (parsed.type === 'welcome') {
if (welcomed) throw new Error('Duplicate welcome.')
welcomed = true
this.ready = true
this.reconcile(parsed.inFlightExecutionIds)
this.log(`Runner connected: ${parsed.runnerName} (${parsed.runnerId})`)
} else {
if (!welcomed) throw new Error('Expected welcome before runner requests.')
this.handle(parsed)
}
} catch {
reject(new CliError('Server sent an invalid runner protocol message.'))
socket.terminate()
}
})
if (this.stopped.signal.aborted) socket.terminate()
})
}
private send(message: RunnerToServerMessage) {
if (this.ready && this.socket?.readyState === WebSocket.OPEN)
this.socket.send(JSON.stringify(message))
}
private reconcile(inFlight: string[]) {
const expected = new Set(inFlight)
for (const [id, execution] of this.executions) {
if (!expected.has(id)) {
execution.controller.abort()
if (execution.result) this.executions.delete(id)
continue
}
// Replay before results: the server deduplicates chunks by UTF-8 byte offset.
for (const chunk of execution.chunks) this.send(chunk)
}
this.send({
type: 'reconcile',
environment: this.shell.environment,
activeExecutions: [...this.executions]
.filter(([id, execution]) => expected.has(id) && !execution.result)
.map(([executionId, execution]) => ({ executionId, nextOffset: execution.offset })),
completedExecutions: [...this.executions]
.filter(([id, execution]) => expected.has(id) && execution.result)
.map(([, execution]) => execution.result!),
})
}
private handle(message: Exclude<ServerToRunnerMessage, { type: 'welcome' }>) {
if (message.type === 'ping') {
this.send({ type: 'pong', ts: message.ts })
return
}
if (message.type === 'exec.cancel') {
this.executions.get(message.executionId)?.controller.abort()
return
}
if (message.type === 'exec.start') {
const previous = this.executions.get(message.executionId)
if (previous) {
if (previous.result) this.send(previous.result)
return
}
const execution: Execution = {
controller: new AbortController(),
chunks: [],
bytes: 0,
offset: 0,
}
this.executions.set(message.executionId, execution)
this.track(
(async () => {
const busy = this.activeCount() > MAX_ACTIVE
const result = busy
? {
type: 'exec.result' as const,
executionId: message.executionId,
exitCode: null,
signal: null,
timedOut: false,
error: 'Runner is busy. Try again later.',
}
: await this.shell.run(message, execution.controller.signal, (stream, data) => {
// Small frames keep reconnect replay and WebSocket buffering bounded.
for (const part of data.match(/[\s\S]{1,8192}/gu) ?? []) {
const chunk: RunnerExecOutputMessage = {
type: 'exec.output',
executionId: message.executionId,
stream,
data: part,
offset: execution.offset,
}
const bytes = Buffer.byteLength(part)
execution.offset += bytes
execution.bytes += bytes
execution.chunks.push(chunk)
while (execution.bytes > MAX_REPLAY_BYTES) {
const removed = execution.chunks.shift()!
execution.bytes -= Buffer.byteLength(removed.data)
if (!this.ready) execution.controller.abort()
}
if ((this.socket?.bufferedAmount ?? 0) > MAX_REPLAY_BYTES) {
this.ready = false
this.socket?.terminate()
}
this.send(chunk)
}
})
execution.result = result
this.send(result)
const completed = [...this.executions].filter(([, value]) => value.result)
for (const [id] of completed.slice(0, -MAX_COMPLETED)) this.executions.delete(id)
})(),
)
return
}
this.handleFile(message)
}
private activeCount() {
return (
[...this.executions.values()].filter((execution) => !execution.result).length +
this.transfers.size
)
}
private handleFile(message: FileMessage) {
const previous = this.completedFiles.get(message.executionId)
if (previous) {
this.send(previous)
return
}
if (this.transfers.has(message.executionId)) return
const controller = new AbortController()
this.transfers.set(message.executionId, controller)
const timer = setTimeout(() => controller.abort(), 120_000)
this.track(
(async () => {
try {
const result =
this.activeCount() > MAX_ACTIVE
? this.fileBusy(message)
: await this.files.handle(message, controller.signal)
this.completedFiles.set(message.executionId, result)
if (this.completedFiles.size > MAX_COMPLETED)
this.completedFiles.delete(this.completedFiles.keys().next().value!)
this.send(result)
} finally {
clearTimeout(timer)
this.transfers.delete(message.executionId)
}
})(),
)
}
private fileBusy(message: FileMessage): RunnerToServerMessage {
const result = {
executionId: message.executionId,
ok: false,
error: 'Runner is busy. Try again later.',
}
switch (message.type) {
case 'file.download':
return { ...result, type: 'file.download.result' }
case 'file.upload':
return { ...result, type: 'file.upload.result' }
case 'file.stat':
return { ...result, type: 'file.stat.result', exists: false }
}
}
private track(task: Promise<void>) {
this.pending.add(task)
void task.finally(() => this.pending.delete(task)).catch(() => this.stop())
}
}
export async function startRunner(options: RunnerOptions) {
const config = await resolveRunnerConfig(options)
const files = new RunnerFiles(config.cwd)
const shell = new RunnerShell(files, config.srt)
// SRT resolves relative policy paths against the runner workspace.
process.chdir(config.cwd)
await shell.initialize()
console.error(`Runner shell: ${shell.environment.shell}`)
const runner = new RunnerClient(config, files, shell)
const controller = new AbortController()
const shutdown = () => controller.abort()
process.once('SIGINT', shutdown)
process.once('SIGTERM', shutdown)
try {
await runner.run(controller.signal)
} finally {
process.removeListener('SIGINT', shutdown)
process.removeListener('SIGTERM', shutdown)
await shell.close()
}
}
+64
View File
@@ -0,0 +1,64 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import { commandEnv, resolveRunnerConfig } from './config'
afterEach(() => vi.unstubAllEnvs())
describe('runner configuration', () => {
it('preserves Windows OS settings, normalizes explicit overrides, and excludes credentials', () => {
vi.stubEnv('Path', 'C:\\Windows\\System32')
vi.stubEnv('SystemRoot', 'C:\\Windows')
vi.stubEnv('USERPROFILE', 'C:\\Users\\runner')
const env = commandEnv({ PATH: 'C:\\tools', ['shumai_runner_key']: 'secret' })
expect(env).toMatchObject({
PATH: 'C:\\tools',
SystemRoot: 'C:\\Windows',
USERPROFILE: 'C:\\Users\\runner',
})
expect(env).not.toHaveProperty('Path')
expect(env).not.toHaveProperty('shumai_runner_key')
})
it('uses a dedicated runner key and preserves a server path prefix', async () => {
vi.stubEnv('SHUMAI_API_SERVER', 'https://example.com/shumai/')
vi.stubEnv('SHUMAI_RUNNER_KEY', 'runner-secret')
const config = await resolveRunnerConfig({})
expect(config.url.href).toBe('wss://example.com/shumai/api/runners/ws')
expect(config.key).toBe('runner-secret')
expect(config.cwd).toBe(process.cwd())
})
it.each([
'http://example.com',
'ftp://localhost',
'https://user:secret@example.com',
'https://example.com/?key=secret',
'not a URL',
])('rejects unsafe server URL %s', async (url) => {
await expect(resolveRunnerConfig({ url, key: 'key' })).rejects.toThrow()
})
it('allows the Docker host development address', async () => {
expect(
(await resolveRunnerConfig({ url: 'http://host.docker.internal:3000', key: 'key' })).url.href,
).toBe('ws://host.docker.internal:3000/api/runners/ws')
})
it('does not accept a personal API token in place of a missing runner key', async () => {
vi.stubEnv('SHUMAI_RUNNER_KEY', '')
vi.stubEnv('SHUMAI_API_KEY', 'personal-secret')
await expect(resolveRunnerConfig({ url: 'https://example.com' })).rejects.toThrow(
'SHUMAI_RUNNER_KEY',
)
})
it('inherits only baseline environment and explicit skill variables', () => {
vi.stubEnv('SHUMAI_RUNNER_KEY', 'runner-secret')
vi.stubEnv('SHUMAI_API_KEY', 'personal-secret')
vi.stubEnv('AWS_SECRET_ACCESS_KEY', 'aws-secret')
vi.stubEnv('BASH_ENV', '/malicious-startup')
expect(
commandEnv({ SKILL_TOKEN: 'skill-secret', SHUMAI_RUNNER_KEY: 'injected' }),
).toMatchObject({ SKILL_TOKEN: 'skill-secret' })
for (const name of ['SHUMAI_RUNNER_KEY', 'SHUMAI_API_KEY', 'AWS_SECRET_ACCESS_KEY', 'BASH_ENV'])
expect(commandEnv()).not.toHaveProperty(name)
})
})
+69
View File
@@ -0,0 +1,69 @@
import { realpath, stat } from 'node:fs/promises'
import path from 'node:path'
import { CliError } from '../operations/errors'
export interface RunnerOptions {
url?: string
key?: string
cwd?: string
srt?: string
}
export async function resolveRunnerConfig(options: RunnerOptions) {
const address = options.url ?? process.env.SHUMAI_API_SERVER
const key = options.key ?? process.env.SHUMAI_RUNNER_KEY
if (!address) throw new CliError('Set SHUMAI_API_SERVER or pass --url.')
if (!key?.trim()) throw new CliError('Set SHUMAI_RUNNER_KEY or pass --key with a runner key.')
let url: URL
try {
url = new URL(address)
} catch {
throw new CliError('Invalid Shumai server URL.')
}
if (
url.username ||
url.password ||
url.search ||
url.hash ||
!['http:', 'https:'].includes(url.protocol)
) {
throw new CliError(
'Use an http:// or https:// server URL without credentials, query, or fragment.',
)
}
const localHosts = ['localhost', '127.0.0.1', '[::1]', 'host.docker.internal']
if (url.protocol === 'http:' && !localHosts.includes(url.hostname)) {
throw new CliError(
'Remote runners require HTTPS. HTTP is only supported for local development.',
)
}
url.protocol = url.protocol === 'https:' ? 'wss:' : 'ws:'
url.pathname = `${url.pathname.replace(/\/+$/, '')}/api/runners/ws`
const cwd = await realpath(path.resolve(options.cwd ?? process.cwd()))
if (!(await stat(cwd)).isDirectory()) throw new CliError('Runner workspace must be a directory.')
return { url, key: key.trim(), cwd, srt: options.srt ? path.resolve(options.srt) : undefined }
}
/** Commands inherit baseline OS settings and explicitly supplied skill variables, never CLI credentials. */
export function commandEnv(extra: Record<string, string> = {}, inherit = true): NodeJS.ProcessEnv {
const env: NodeJS.ProcessEnv = {}
for (const [key, value] of Object.entries(inherit ? process.env : {})) {
if (
/^(PATH|PATHEXT|HOME|USER|LOGNAME|LANG|TERM|TMPDIR|TEMP|TMP|SYSTEMROOT|SYSTEMDRIVE|COMSPEC|USERPROFILE|USERNAME|APPDATA|LOCALAPPDATA|PROGRAMDATA|PROGRAMFILES|PROGRAMFILES\(X86\)|LC_.*)$/i.test(
key,
)
)
env[key] = value
}
for (const [key, value] of Object.entries(extra)) {
if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(key))
throw new Error('Invalid skill environment variable name.')
if (['SHUMAI_RUNNER_KEY', 'SHUMAI_API_KEY'].includes(key.toUpperCase())) continue
// Windows treats environment names as case-insensitive.
for (const existing of Object.keys(env)) {
if (existing.toUpperCase() === key.toUpperCase()) delete env[existing]
}
env[key] = value
}
return env
}
+364
View File
@@ -0,0 +1,364 @@
import { createHash } from 'node:crypto'
import { createServer, type Server } from 'node:http'
import { mkdtemp, mkdir, readFile, readdir, rm, stat, symlink, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import path from 'node:path'
import AdmZip from 'adm-zip'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import yauzl from 'yauzl'
import { RunnerFiles } from './files'
describe('runner workspace file transfers', () => {
let cwd: string
let files: RunnerFiles
let server: Server
let url: string
let body: Buffer
let uploaded: Buffer
beforeEach(async () => {
cwd = await mkdtemp(path.join(tmpdir(), 'shumai-files-test-'))
files = new RunnerFiles(cwd)
body = Buffer.from('download contents')
uploaded = Buffer.alloc(0)
server = createServer(async (request, response) => {
if (request.method === 'PUT') {
const chunks: Buffer[] = []
for await (const chunk of request) chunks.push(Buffer.from(chunk))
uploaded = Buffer.concat(chunks)
response.end()
} else if (request.url === '/slow') {
const timer = setInterval(() => response.write(Buffer.alloc(65536)), 20)
response.once('close', () => clearInterval(timer))
} else if (request.url === '/redirect') {
response.writeHead(302, { location: '/file' })
response.end()
} else response.end(body)
})
await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve))
const address = server.address()
if (!address || typeof address === 'string') throw new Error('Missing test HTTP address')
url = `http://127.0.0.1:${address.port}`
})
afterEach(async () => {
server.closeAllConnections()
await new Promise<void>((resolve) => server.close(() => resolve()))
await rm(cwd, { recursive: true, force: true })
})
const signal = () => new AbortController().signal
it('downloads, checks SHA-256, stats and uploads actual bytes', async () => {
expect(
await files.handle(
{
type: 'file.download',
executionId: 'download',
url,
destPath: '.pi/input/data.bin',
expectedHash: createHash('sha256').update(body).digest('hex'),
},
signal(),
),
).toMatchObject({ ok: true, size: body.length })
expect(await readFile(path.join(cwd, '.pi/input/data.bin'))).toEqual(body)
expect(
await files.handle(
{ type: 'file.stat', executionId: 'stat', srcPath: '.pi/input/data.bin' },
signal(),
),
).toMatchObject({ ok: true, exists: true, isFile: true, size: body.length })
expect(
await files.handle(
{ type: 'file.upload', executionId: 'upload', srcPath: '.pi/input/data.bin', url },
signal(),
),
).toMatchObject({ ok: true, size: body.length, name: 'data.bin' })
expect(uploaded).toEqual(body)
expect(
await files.handle(
{ type: 'file.stat', executionId: 'absent', srcPath: '.pi/absent' },
signal(),
),
).toMatchObject({ ok: true, exists: false })
})
it('does not publish mismatched downloads or follow redirects', async () => {
expect(
await files.handle(
{
type: 'file.download',
executionId: 'bad-hash',
url,
destPath: '.pi/data',
expectedHash: 'bad',
},
signal(),
),
).toMatchObject({ ok: false, error: 'Downloaded file hash does not match.' })
await expect(readFile(path.join(cwd, '.pi/data'))).rejects.toThrow()
expect(
await files.handle(
{
type: 'file.download',
executionId: 'redirect',
url: `${url}/redirect`,
destPath: '.pi/data',
},
signal(),
),
).toHaveProperty('ok', false)
})
it.each([
'../secret',
'/etc/passwd',
'.pi/../secret',
'.pi/dir/../../secret',
'.pi\\secret',
'.pi//file',
'.pi/a:b',
'.pi/a\0b',
'.pi/.. /outside',
'.pi/folder./file',
'.pi/CON.txt',
'.pi/nul .txt',
'.pi/COM1',
'.pi/lpt¹.log',
'.pi/CONOUT$',
])('rejects unsafe protocol path %s', async (srcPath) => {
expect(
await files.handle({ type: 'file.stat', executionId: 'unsafe', srcPath }, signal()),
).toHaveProperty('ok', false)
})
it('rejects parent and final symlinks, including a replaced .pi root', async () => {
await mkdir(path.join(cwd, '.pi'))
await writeFile(path.join(cwd, 'secret'), 'secret')
await symlink(cwd, path.join(cwd, '.pi/escape'))
await symlink(path.join(cwd, 'secret'), path.join(cwd, '.pi/link'))
for (const srcPath of ['.pi/escape/secret', '.pi/link']) {
expect(
await files.handle({ type: 'file.upload', executionId: 'symlink', srcPath, url }, signal()),
).toHaveProperty('ok', false)
}
await rm(path.join(cwd, '.pi'), { recursive: true })
await symlink(cwd, path.join(cwd, '.pi'))
expect(
await files.handle(
{ type: 'file.download', executionId: 'root', destPath: '.pi/file', url },
signal(),
),
).toHaveProperty('ok', false)
})
it('extracts skills and safely replaces old contents', async () => {
const zip = new AdmZip()
zip.addFile('SKILL.md', Buffer.from('# Test skill'))
zip.addFile('scripts/run.sh', Buffer.from('echo skill'))
body = zip.toBuffer()
for (let attempt = 0; attempt < 2; attempt++) {
expect(
await files.handle(
{
type: 'file.download',
executionId: `zip-${attempt}`,
url,
destPath: '.pi/skills/test',
extractZip: true,
},
signal(),
),
).toHaveProperty('ok', true)
}
expect(await readFile(path.join(cwd, '.pi/skills/test/SKILL.md'), 'utf8')).toBe('# Test skill')
})
it('rejects symlink ZIP entries and zip-slip paths without publishing them', async () => {
const zip = new AdmZip()
zip.addFile('link', Buffer.from('/etc/passwd'))
const entry = zip.getEntry('link')!
entry.header.attr = (0o120777 << 16) >>> 0
body = zip.toBuffer()
expect(
await files.handle(
{
type: 'file.download',
executionId: 'zip-link',
url,
destPath: '.pi/skills/bad',
extractZip: true,
},
signal(),
),
).toHaveProperty('ok', false)
const normal = new AdmZip()
normal.addFile('aa/secret', Buffer.from('escape'))
body = Buffer.from(
normal.toBuffer().toString('latin1').replaceAll('aa/secret', '../secret'),
'latin1',
)
expect(
await files.handle(
{
type: 'file.download',
executionId: 'zip-slip',
url,
destPath: '.pi/skills/bad',
extractZip: true,
},
signal(),
),
).toHaveProperty('ok', false)
await expect(readFile(path.join(cwd, '.pi/skills/bad/link'))).rejects.toThrow()
})
it('preserves executable script permissions and rejects oversized ZIP entries', async () => {
const zip = new AdmZip()
zip.addFile('run.sh', Buffer.from('#!/bin/sh\necho executable\n'))
zip.getEntry('run.sh')!.header.attr = (0o100755 << 16) >>> 0
body = zip.toBuffer()
expect(
await files.handle(
{
type: 'file.download',
executionId: 'exec-zip',
url,
destPath: '.pi/skills/executable',
extractZip: true,
},
signal(),
),
).toHaveProperty('ok', true)
expect((await stat(path.join(cwd, '.pi/skills/executable/run.sh'))).mode & 0o7777).toBe(0o755)
const central = body.indexOf(Buffer.from([0x50, 0x4b, 0x01, 0x02]))
body.writeUInt32LE(256 * 1024 * 1024 + 1, central + 24)
expect(
await files.handle(
{
type: 'file.download',
executionId: 'large-zip',
url,
destPath: '.pi/skills/large',
extractZip: true,
},
signal(),
),
).toMatchObject({ ok: false, error: 'Extracted ZIP exceeds 256 MiB.' })
})
it('cancels a streaming download without publishing partial bytes or leaving staging files', async () => {
await mkdir(path.join(cwd, '.pi/input'), { recursive: true })
await writeFile(path.join(cwd, '.pi/input/data'), 'original')
const controller = new AbortController()
const transfer = files.handle(
{
type: 'file.download',
executionId: 'stream-cancel',
url: `${url}/slow`,
destPath: '.pi/input/data',
},
controller.signal,
)
await vi.waitFor(async () =>
expect(
(await readdir(path.join(cwd, '.pi/input'))).some((name) => name.startsWith('.shumai-')),
).toBe(true),
)
controller.abort()
expect(await transfer).toHaveProperty('ok', false)
expect(await readFile(path.join(cwd, '.pi/input/data'), 'utf8')).toBe('original')
expect(await readdir(path.join(cwd, '.pi/input'))).toEqual(['data'])
})
it('cancels extraction without deleting the installed skill or leaving staging trees', async () => {
await mkdir(path.join(cwd, '.pi/skills/installed'), { recursive: true })
await writeFile(path.join(cwd, '.pi/skills/installed/SKILL.md'), 'original')
const zip = new AdmZip()
for (let index = 0; index < 2000; index++)
zip.addFile(`file-${index}.txt`, Buffer.alloc(32768, 65))
body = zip.toBuffer()
const controller = new AbortController()
const transfer = files.handle(
{
type: 'file.download',
executionId: 'extract-cancel',
url,
destPath: '.pi/skills/installed',
extractZip: true,
},
controller.signal,
)
await vi.waitFor(
async () => {
const temporary = (await readdir(path.join(cwd, '.pi/skills'))).find((name) =>
name.startsWith('.shumai-'),
)
expect(temporary).toBeDefined()
expect(
(await readdir(path.join(cwd, '.pi/skills', temporary!, 'extracted'))).length,
).toBeGreaterThan(0)
},
{ interval: 5, timeout: 5000 },
)
controller.abort()
expect(await transfer).toHaveProperty('ok', false)
expect(await readFile(path.join(cwd, '.pi/skills/installed/SKILL.md'), 'utf8')).toBe('original')
expect(await readdir(path.join(cwd, '.pi/skills'))).toEqual(['installed'])
}, 10000)
it('waits for an in-flight ZIP entry read before closing on cancellation', async () => {
const zip = new AdmZip()
zip.addFile('SKILL.md', Buffer.from('skill'))
body = zip.toBuffer()
const controller = new AbortController()
const pendingRead = Promise.withResolvers<() => void>()
const readEntry = yauzl.ZipFile.prototype.readEntry
const read = vi.spyOn(yauzl.ZipFile.prototype, 'readEntry').mockImplementationOnce(function (
this: yauzl.ZipFile,
) {
pendingRead.resolve(() => readEntry.call(this))
})
const close = vi.spyOn(yauzl.ZipFile.prototype, 'close')
try {
const transfer = files.handle(
{
type: 'file.download',
executionId: 'zip-entry-cancel',
url,
destPath: '.pi/skills/new',
extractZip: true,
},
controller.signal,
)
const finishRead = await pendingRead.promise
controller.abort()
await new Promise<void>((resolve) => setImmediate(resolve))
const closedDuringRead = close.mock.calls.length > 0
if (!closedDuringRead) finishRead()
expect(await transfer).toHaveProperty('ok', false)
expect(closedDuringRead).toBe(false)
expect(await readdir(path.join(cwd, '.pi/skills'))).toEqual([])
} finally {
read.mockRestore()
close.mockRestore()
}
})
it('handles cancelled transfers and refuses uploads of directories', async () => {
const controller = new AbortController()
controller.abort()
expect(
await files.handle(
{ type: 'file.download', executionId: 'cancelled', url, destPath: '.pi/data' },
controller.signal,
),
).toHaveProperty('ok', false)
expect(
await files.handle(
{ type: 'file.upload', executionId: 'directory', url, srcPath: '.pi/transfers' },
signal(),
),
).toHaveProperty('ok', false)
expect(uploaded).toHaveLength(0)
})
})
+282
View File
@@ -0,0 +1,282 @@
import { createHash } from 'node:crypto'
import { constants, createWriteStream } from 'node:fs'
import { lstat, mkdir, mkdtemp, open, rename, rm } from 'node:fs/promises'
import path from 'node:path'
import { Readable, Transform } from 'node:stream'
import { pipeline } from 'node:stream/promises'
import yauzl from 'yauzl'
import type {
RunnerFileDownloadMessage,
RunnerFileUploadMessage,
RunnerFileStatMessage,
RunnerToServerMessage,
} from '@shumai/dtos'
const MAX_FILE_BYTES = 256 * 1024 * 1024
const MAX_ZIP_ENTRIES = 10_000
export type FileMessage =
RunnerFileDownloadMessage | RunnerFileUploadMessage | RunnerFileStatMessage
function pathParts(value: string): string[] {
const parts = value.split('/')
if (
value.includes('\\') ||
value.includes('\0') ||
parts.some(
(part) =>
!part ||
part === '.' ||
part === '..' ||
part.includes(':') ||
/[. ]$/.test(part) ||
/^(?:con|prn|aux|nul|conin\$|conout\$|com[1-9¹²³]|lpt[1-9¹²³])(?: *\.|$)/i.test(part),
)
)
throw new Error('Invalid workspace path.')
return parts
}
function isMissing(error: unknown): boolean {
return error instanceof Error && 'code' in error && error.code === 'ENOENT'
}
function transferUrl(address: string): URL {
const url = new URL(address)
if (!['http:', 'https:'].includes(url.protocol) || url.username || url.password)
throw new Error('Invalid transfer URL.')
return url
}
export class RunnerFiles {
constructor(readonly cwd: string) {}
/** Basic workspace validation. Concurrent hostile filesystem mutation is outside v1's scope. */
private async resolve(relative: string, createParents = false): Promise<string> {
const parts = pathParts(relative)
if (parts[0] !== '.pi' || parts.length < 2) throw new Error('File paths must stay inside .pi/.')
let current = this.cwd
for (let index = 0; index < parts.length; index++) {
current = path.join(current, parts[index])
const parent = index < parts.length - 1
if (parent && createParents) await mkdir(current, { recursive: true, mode: 0o700 })
try {
const info = await lstat(current)
if (info.isSymbolicLink() || (parent && !info.isDirectory()))
throw new Error('Workspace paths cannot contain symlinks or non-directory parents.')
} catch (error) {
if (!isMissing(error)) throw error
}
}
return current
}
async createWriter(relative: string) {
const file = await open(
await this.resolve(relative, true),
constants.O_CREAT | constants.O_EXCL | constants.O_WRONLY | constants.O_NOFOLLOW,
0o600,
)
return {
write: async (data: string) => {
await file.writeFile(data)
},
close: () => file.close(),
}
}
async handle(message: FileMessage, signal: AbortSignal): Promise<RunnerToServerMessage> {
try {
signal.throwIfAborted()
if (message.type === 'file.download')
return {
type: 'file.download.result',
executionId: message.executionId,
ok: true,
...(await this.download(message, signal)),
}
const filename = await this.resolve(message.srcPath)
if (message.type === 'file.upload') return await this.upload(message, filename, signal)
try {
const info = await lstat(filename)
return {
type: 'file.stat.result',
executionId: message.executionId,
ok: true,
exists: true,
isFile: info.isFile(),
size: info.size,
name: path.basename(filename),
}
} catch (error) {
if (!isMissing(error)) throw error
return {
type: 'file.stat.result',
executionId: message.executionId,
ok: true,
exists: false,
}
}
} catch (error) {
const result = {
executionId: message.executionId,
ok: false,
error: error instanceof Error ? error.message : 'File operation failed.',
}
switch (message.type) {
case 'file.download':
return { ...result, type: 'file.download.result' }
case 'file.upload':
return { ...result, type: 'file.upload.result' }
case 'file.stat':
return { ...result, type: 'file.stat.result', exists: false }
}
}
}
private async upload(
message: RunnerFileUploadMessage,
filename: string,
signal: AbortSignal,
): Promise<RunnerToServerMessage> {
const file = await open(
filename,
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
)
try {
const info = await file.stat()
if (!info.isFile() || info.size > MAX_FILE_BYTES)
throw new Error('Upload must be a regular file no larger than 256 MiB.')
// Node and Bun declare different DOM stream types for the same fetch-compatible stream.
const body = info.size
? (Readable.toWeb(
file.createReadStream({ autoClose: false, end: info.size - 1 }),
) as unknown as ReadableStream<Uint8Array>)
: new Uint8Array()
const response = await fetch(transferUrl(message.url), {
method: 'PUT',
body,
...{ duplex: 'half' },
headers: {
'content-type': message.contentType ?? 'application/octet-stream',
'content-length': String(info.size),
},
signal,
redirect: 'error',
})
await response.body?.cancel()
if (!response.ok) throw new Error(`Upload failed (HTTP ${response.status}).`)
return {
type: 'file.upload.result',
executionId: message.executionId,
ok: true,
size: info.size,
name: path.basename(filename),
}
} finally {
await file.close()
}
}
private async download(message: RunnerFileDownloadMessage, signal: AbortSignal) {
const destination = await this.resolve(message.destPath, true)
const temporary = await mkdtemp(path.join(path.dirname(destination), '.shumai-'))
let preserveBackup = false
try {
const response = await fetch(transferUrl(message.url), { signal, redirect: 'error' })
if (!response.ok || !response.body) {
await response.body?.cancel()
throw new Error(`Download failed (HTTP ${response.status}).`)
}
const hash = createHash('sha256')
let size = 0
const archive = path.join(temporary, 'download')
await pipeline(
response.body,
new Transform({
transform(chunk: Buffer, _encoding, callback) {
size += chunk.length
if (size > MAX_FILE_BYTES) {
callback(new Error('Download exceeds 256 MiB.'))
return
}
hash.update(chunk)
callback(null, chunk)
},
}),
createWriteStream(archive, { flags: 'wx', mode: 0o600 }),
{ signal },
)
if (message.expectedHash && hash.digest('hex') !== message.expectedHash)
throw new Error('Downloaded file hash does not match.')
let source = archive
if (message.extractZip) {
source = path.join(temporary, 'extracted')
await mkdir(source, { mode: 0o700 })
await this.extract(archive, source, signal)
}
signal.throwIfAborted()
await this.resolve(message.destPath)
if (message.extractZip) {
const backup = path.join(temporary, 'previous')
let backedUp = false
try {
await rename(destination, backup)
backedUp = true
} catch (error) {
if (!isMissing(error)) throw error
}
try {
await rename(source, destination)
} catch (error) {
if (backedUp) {
preserveBackup = true
await rename(backup, destination)
preserveBackup = false
}
throw error
}
} else await rename(source, destination)
return { size, cached: false }
} finally {
if (!preserveBackup) await rm(temporary, { recursive: true, force: true })
}
}
private async extract(archive: string, directory: string, signal: AbortSignal) {
const zip = await yauzl.openPromise(archive, {
lazyEntries: true,
strictFileNames: true,
autoClose: false,
})
let size = 0
let count = 0
try {
signal.throwIfAborted()
// Let each metadata read finish before cancellation can close the ZIP descriptor.
for await (const entry of zip.eachEntry()) {
signal.throwIfAborted()
if (++count > MAX_ZIP_ENTRIES) throw new Error('ZIP contains too many entries.')
const parts = pathParts(entry.fileName.replace(/\/$/, ''))
const mode = entry.externalFileAttributes >>> 16
const kind = mode & 0o170000
if (kind && kind !== 0o100000 && kind !== 0o040000)
throw new Error('ZIP links and special files are forbidden.')
size += entry.uncompressedSize
if (size > MAX_FILE_BYTES) throw new Error('Extracted ZIP exceeds 256 MiB.')
const destination = path.join(directory, ...parts)
if (entry.fileName.endsWith('/')) await mkdir(destination, { recursive: true, mode: 0o700 })
else {
await mkdir(path.dirname(destination), { recursive: true, mode: 0o700 })
const input = await zip.openReadStreamPromise(entry)
await pipeline(
input,
createWriteStream(destination, { flags: 'wx', mode: 0o600 | (mode & 0o155) }),
{ signal },
)
}
}
} finally {
zip.close()
}
}
}
+106
View File
@@ -0,0 +1,106 @@
import { spawnSync, type SpawnSyncReturns } from 'node:child_process'
import { accessSync, constants } from 'node:fs'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { resolveRunnerShell } from './shell-config'
vi.mock('node:child_process', () => ({ spawnSync: vi.fn() }))
vi.mock('node:fs', { spy: true })
const access = vi.mocked(accessSync)
const where = vi.mocked(spawnSync)
const gitBash = 'D:\\Program Files\\Git\\bin\\bash.exe'
const gitBashX86 = 'D:\\Program Files (x86)\\Git\\bin\\bash.exe'
function whereResult(stdout = '', status = 0): SpawnSyncReturns<string> {
return { stdout, stderr: '', status, output: [null, stdout, ''], pid: 1, signal: null }
}
function available(...paths: string[]) {
access.mockImplementation((executable) => {
if (!paths.some((path) => executable === path)) throw new Error('File not found')
})
}
beforeEach(() => {
vi.stubEnv('ProgramFiles', 'D:\\Program Files')
vi.stubEnv('ProgramFiles(x86)', 'D:\\Program Files (x86)')
vi.stubEnv('SystemRoot', 'E:\\WinNT')
available()
where.mockReturnValue(whereResult('', 1))
})
afterEach(() => {
vi.resetAllMocks()
vi.unstubAllEnvs()
})
describe('runner shell selection', () => {
it.each(['linux', 'darwin'] as const)('uses /bin/sh on %s', (platform) => {
expect(resolveRunnerShell(platform)).toEqual({
executable: '/bin/sh',
commandTransport: 'shell',
})
expect(where).not.toHaveBeenCalled()
})
it.each([
[gitBash, [gitBash, gitBashX86]],
[gitBashX86, [gitBashX86]],
] as const)('prefers installed Git Bash at %s before PATH', (executable, paths) => {
available(...paths)
expect(resolveRunnerShell('win32')).toEqual({ executable, commandTransport: 'argv' })
expect(where).not.toHaveBeenCalled()
})
// Adapted from Pi's shell-path-exists.test.ts. See apps/cli/THIRD-PARTY-NOTICES.txt.
it('accepts a Windows Store Bash alias found on PATH via access(F_OK)', () => {
const alias = 'D:\\Users\\runner\\AppData\\Local\\Microsoft\\WindowsApps\\bash.exe'
available(alias)
where.mockReturnValue(whereResult(alias + '\r\n'))
expect(resolveRunnerShell('win32')).toEqual({ executable: alias, commandTransport: 'argv' })
expect(access).toHaveBeenCalledWith(alias, constants.F_OK)
expect(where).toHaveBeenCalledWith('E:\\WinNT\\System32\\where.exe', ['$PATH:bash.exe'], {
encoding: 'utf8',
timeout: 5000,
windowsHide: true,
})
})
it('skips relative and inaccessible PATH results', () => {
const executable = 'D:\\MSYS2\\usr\\bin\\bash.exe'
available(executable)
where.mockReturnValue(
whereResult(['bash.exe', 'D:\\missing\\bash.exe', executable].join('\r\n')),
)
expect(resolveRunnerShell('win32')).toEqual({ executable, commandTransport: 'argv' })
})
it.each([
['E:\\WinNT', 'E:\\WinNT\\System32\\cmd.exe'],
[undefined, 'C:\\Windows\\System32\\cmd.exe'],
] as const)(
'falls back to cmd.exe when no Bash is available (SystemRoot %s)',
(root, executable) => {
vi.stubEnv('SystemRoot', root)
expect(resolveRunnerShell('win32')).toEqual({ executable, commandTransport: 'shell' })
},
)
// Adapted from Pi's windows-shell-config.test.ts.
it.each([
['D:\\WinNT\\', 'd:/WINNT/System32/BASH.EXE', 'stdin'],
['D:\\WinNT', 'D:\\WinNT\\Sysnative\\bash.exe', 'stdin'],
[undefined, 'E:\\Windows\\System32\\bash.exe', 'stdin'],
['D:\\WinNT', 'D:\\Git\\bin\\bash.exe', 'argv'],
['D:\\WinNT', 'E:\\Windows\\System32\\bash.exe', 'argv'],
['WinNT', 'C:\\Windows\\System32\\bash.exe', 'argv'],
] as const)(
'uses the correct command transport for SystemRoot %s and %s',
(root, executable, commandTransport) => {
vi.stubEnv('SystemRoot', root)
available(executable)
where.mockReturnValue(whereResult(executable + '\r\n'))
expect(resolveRunnerShell('win32')).toEqual({ executable, commandTransport })
},
)
})
+64
View File
@@ -0,0 +1,64 @@
import { spawnSync } from 'node:child_process'
import { accessSync, constants } from 'node:fs'
import { win32 } from 'node:path'
export interface RunnerShellConfig {
executable: string
commandTransport: 'shell' | 'argv' | 'stdin'
}
// Windows Bash discovery and legacy WSL detection adapted from pi-mono.
// See apps/cli/THIRD-PARTY-NOTICES.txt.
function isLegacyWslBashPath(executable: string, systemRoot: string | undefined): boolean {
const normalized = win32.normalize(executable).toLowerCase()
if (systemRoot) {
if (!/^[a-z]:[\\/]/i.test(systemRoot)) return false
return ['System32', 'Sysnative'].some(
(directory) => normalized === win32.join(systemRoot, directory, 'bash.exe').toLowerCase(),
)
}
return /^[a-z]:\\windows\\(?:system32|sysnative)\\bash\.exe$/.test(normalized)
}
function shellPathExists(executable: string): boolean {
// access(F_OK) accepts runnable Windows Store aliases that fail an extra stat check.
try {
accessSync(executable, constants.F_OK)
return true
} catch {
return false
}
}
function findWindowsBash(system32: string): string | undefined {
for (const directory of [process.env.ProgramFiles, process.env['ProgramFiles(x86)']]) {
if (!directory || !win32.isAbsolute(directory)) continue
const executable = win32.join(directory, 'Git', 'bin', 'bash.exe')
if (shellPathExists(executable)) return executable
}
// Use the system tool and limit its search to PATH. The workspace may contain a bash.exe.
const found = spawnSync(win32.join(system32, 'where.exe'), ['$PATH:bash.exe'], {
encoding: 'utf8',
timeout: 5000,
windowsHide: true,
})
if (found.status !== 0 || !found.stdout) return
return found.stdout
.trim()
.split(/\r?\n/)
.find((executable) => win32.isAbsolute(executable) && shellPathExists(executable))
}
export function resolveRunnerShell(platform = process.platform): RunnerShellConfig {
if (platform !== 'win32') return { executable: '/bin/sh', commandTransport: 'shell' }
const systemRoot = process.env.SystemRoot
const system32 = win32.join(systemRoot ?? 'C:\\Windows', 'System32')
const executable = findWindowsBash(system32)
if (!executable) return { executable: win32.join(system32, 'cmd.exe'), commandTransport: 'shell' }
return {
executable,
commandTransport: isLegacyWslBashPath(executable, systemRoot) ? 'stdin' : 'argv',
}
}
+352
View File
@@ -0,0 +1,352 @@
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import path from 'node:path'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { SandboxManager } from '@anthropic-ai/sandbox-runtime'
import { RunnerFiles } from './files'
import { RunnerShell } from './shell'
describe('runner bash execution', () => {
let cwd: string
let shell: RunnerShell
beforeEach(async () => {
cwd = await mkdtemp(path.join(tmpdir(), 'shumai-shell-test-'))
shell = new RunnerShell(new RunnerFiles(cwd))
vi.spyOn(SandboxManager, 'initialize').mockResolvedValue()
vi.spyOn(SandboxManager, 'reset').mockResolvedValue()
vi.spyOn(SandboxManager, 'wrapWithSandboxArgv').mockImplementation(async (command) => ({
argv: ['/bin/sh', '-c', command],
env: process.env,
}))
})
afterEach(async () => {
await shell.close()
vi.restoreAllMocks()
vi.unstubAllEnvs()
await rm(cwd, { recursive: true, force: true })
})
const request = (command: string, executionId = 'test') => ({
type: 'exec.start' as const,
executionId,
command,
})
it('runs with no executables on PATH when SRT is not requested', async () => {
vi.stubEnv('PATH', '')
await shell.initialize()
let output = ''
const result = await shell.run(
request('printf no-extra-runtime'),
new AbortController().signal,
(_stream, data) => {
output += data
},
)
expect(result).toMatchObject({ exitCode: 0 })
expect(output).toBe('no-extra-runtime')
})
// Adapted from Pi's tools.test.ts. See apps/cli/THIRD-PARTY-NOTICES.txt.
it('sends the complete command over stdin for the legacy Bash launcher', async () => {
shell = new RunnerShell(new RunnerFiles(cwd), undefined, {
executable: '/bin/sh',
commandTransport: 'stdin',
})
const command = [
'name="Hello, 世界"',
'for choice in "one two" \'three "four"\'; do',
' printf \'%s / %s\\n\' "$name" "$choice"',
'done',
].join('\n')
let output = ''
const result = await shell.run(
request(command),
new AbortController().signal,
(_stream, data) => {
output += data
},
)
expect(result).toMatchObject({ exitCode: 0 })
expect(output).toBe('Hello, 世界 / one two\nHello, 世界 / three "four"\n')
})
it('passes a command with shell quoting directly to the selected executable', async () => {
shell = new RunnerShell(new RunnerFiles(cwd), undefined, {
executable: '/bin/sh',
commandTransport: 'argv',
})
const command = "printf '%s' '$literal \"quotes\"'"
let output = ''
const result = await shell.run(
request(command),
new AbortController().signal,
(_stream, data) => {
output += data
},
)
expect(result).toMatchObject({ exitCode: 0 })
expect(output).toBe('$literal "quotes"')
})
it('rejects the legacy WSL launcher before starting SRT', async () => {
shell = new RunnerShell(new RunnerFiles(cwd), path.join(cwd, 'sandbox.json'), {
executable: 'C:\\Windows\\System32\\bash.exe',
commandTransport: 'stdin',
})
await expect(shell.initialize()).rejects.toThrow('SRT cannot run the legacy WSL Bash launcher')
expect(SandboxManager.initialize).not.toHaveBeenCalled()
})
it('streams both outputs, returns the exit code, and saves a log in the runner workspace', async () => {
const output: string[] = []
const result = await shell.run(
request('printf "hello 世界"; printf "failure" >&2; exit 7'),
new AbortController().signal,
(stream, data) => output.push(`${stream}:${data}`),
)
expect(result).toMatchObject({
exitCode: 7,
signal: null,
timedOut: false,
fullOutputPath: '.pi/logs/bash-test.log',
})
expect(output.join('|')).toContain('stdout:hello 世界')
expect(output.join('|')).toContain('stderr:failure')
expect(await readFile(path.join(cwd, result.fullOutputPath!), 'utf8')).toContain('hello 世界')
})
it('passes skill env without inheriting runner or unrelated credentials', async () => {
vi.stubEnv('SHUMAI_RUNNER_KEY', 'runner-secret')
vi.stubEnv('AWS_SECRET_ACCESS_KEY', 'aws-secret')
let output = ''
await shell.run(
{
...request(
'printf "%s|%s|%s" "$SKILL_TOKEN" "$SHUMAI_RUNNER_KEY" "$AWS_SECRET_ACCESS_KEY"',
),
env: { SKILL_TOKEN: 'skill-secret' },
},
new AbortController().signal,
(_stream, data) => {
output += data
},
)
expect(output).toBe('skill-secret||')
})
it('times out and terminates background children', async () => {
const result = await shell.run(
{ ...request('sleep 30 & echo $!; wait'), timeoutSec: 0.1 },
new AbortController().signal,
() => {},
)
expect(result).toMatchObject({ timedOut: true, signal: 'SIGKILL' })
const pid = Number((await readFile(path.join(cwd, '.pi/logs/bash-test.log'), 'utf8')).trim())
expect(() => process.kill(pid, 0)).toThrow()
})
it('cancels a running command and never starts an already cancelled one', async () => {
const controller = new AbortController()
const running = shell.run(request('echo ready; sleep 30'), controller.signal, () =>
controller.abort(),
)
expect(await running).toMatchObject({ aborted: true, signal: 'SIGKILL' })
expect(
await shell.run(request('touch unwanted', 'cancelled'), controller.signal, () => {}),
).toMatchObject({ aborted: true })
await expect(readFile(path.join(cwd, 'unwanted'))).rejects.toThrow()
})
it('finishes draining after a timely root exit even if a background process holds the pipes', async () => {
const source = `const {spawn} = require('node:child_process'); const child = spawn(process.execPath, ['-e', 'setTimeout(() => {}, 30000)'], {detached: true, stdio: ['ignore', 1, 2]}); console.log(child.pid); child.unref()`
const quote = (value: string) => `'${value.replaceAll("'", "'\\''")}'`
const result = await shell.run(
{ ...request(`${quote(process.execPath)} -e ${quote(source)}`), timeoutSec: 0.5 },
new AbortController().signal,
() => {},
)
const pid = Number((await readFile(path.join(cwd, '.pi/logs/bash-test.log'), 'utf8')).trim())
try {
expect(result).toMatchObject({ exitCode: 0, timedOut: false, aborted: false })
expect(result.error).toBeUndefined()
} finally {
if (pid > 0) process.kill(pid, 'SIGKILL')
}
})
it('reports launch failure without including command text or skill secrets', async () => {
shell = new RunnerShell(new RunnerFiles(cwd), undefined, {
executable: path.join(cwd, 'missing-shell'),
commandTransport: 'shell',
})
const result = await shell.run(
request('echo private-command'),
new AbortController().signal,
() => {},
)
expect(result).toMatchObject({
exitCode: null,
signal: null,
error: 'Command execution failed (ENOENT).',
})
expect(result.error).not.toContain('private-command')
})
it('terminates a running process if the output consumer fails', async () => {
let pid = 0
const result = await shell.run(
request('printf "%s\\n" "$$"; sleep 30'),
new AbortController().signal,
(_stream, data) => {
pid = Number(data.trim())
throw new Error('Output consumer failed')
},
)
expect(result.error).toBe('Output consumer failed')
expect(pid).toBeGreaterThan(0)
expect(() => process.kill(pid, 0)).toThrow()
})
it('rejects invalid timeouts and caps runaway output', async () => {
expect(
await shell.run(
{ ...request('sleep 30'), timeoutSec: -1 },
new AbortController().signal,
() => {},
),
).toHaveProperty('error')
expect(
await shell.run(
request('head -c 70000000 /dev/zero', 'large'),
new AbortController().signal,
() => {},
),
).toMatchObject({ error: 'Command exceeded the 64 MiB output limit.' })
})
it('uses the exact SRT JSON snapshot and validates SRT before connecting', async () => {
const settings = path.join(cwd, 'sandbox.json')
const config = {
network: { allowedDomains: [], deniedDomains: [] },
filesystem: { denyRead: [], allowWrite: ['.pi'], denyWrite: [] },
}
const json = JSON.stringify(config)
await writeFile(settings, json)
vi.stubEnv('PATH', '')
shell = new RunnerShell(new RunnerFiles(cwd), settings)
await shell.initialize()
await writeFile(settings, '{}')
expect(
await shell.run(request('echo sandboxed'), new AbortController().signal, () => {}),
).toMatchObject({ exitCode: 0 })
expect(SandboxManager.initialize).toHaveBeenCalledExactlyOnceWith(
expect.objectContaining(config),
)
expect(SandboxManager.wrapWithSandboxArgv).toHaveBeenCalledWith(
expect.stringContaining('echo sandboxed'),
'/bin/sh',
undefined,
expect.any(AbortSignal),
cwd,
{ commandId: 'test', commandText: 'echo sandboxed' },
)
expect(SandboxManager.reset).not.toHaveBeenCalled()
})
it('keeps skill PATH and Node startup settings out of the SRT launcher', async () => {
const settings = path.join(cwd, 'sandbox.json')
const marker = path.join(cwd, 'injected')
const evil = path.join(cwd, 'evil.cjs')
await writeFile(
settings,
JSON.stringify({
network: { allowedDomains: [], deniedDomains: [] },
filesystem: { denyRead: [], allowWrite: ['.pi'], denyWrite: [] },
}),
)
await writeFile(evil, `require('node:fs').writeFileSync('${marker}', 'escaped')`)
shell = new RunnerShell(new RunnerFiles(cwd), settings)
await shell.initialize()
const result = await shell.run(
{
...request('printf secure'),
env: { PATH: '/evil-srt', NODE_OPTIONS: `--require=${evil}` },
},
new AbortController().signal,
() => {},
)
expect(result).toMatchObject({ exitCode: 0 })
await expect(readFile(marker)).rejects.toThrow()
})
it('initializes one sandbox for parallel commands and resets only at shutdown', async () => {
const settings = path.join(cwd, 'sandbox.json')
await writeFile(
settings,
JSON.stringify({
network: { allowedDomains: [], deniedDomains: [] },
filesystem: { denyRead: [], allowWrite: ['.pi'], denyWrite: [] },
}),
)
shell = new RunnerShell(new RunnerFiles(cwd), settings)
await shell.initialize()
const value = "quoted ' value\n$(echo should-not-expand)"
let output = ''
const results = await Promise.all(
['one', 'two'].map((id) =>
shell.run(
{ ...request('printf "%s" "$SKILL_TOKEN"', id), env: { SKILL_TOKEN: value } },
new AbortController().signal,
(_stream, data) => {
output += data
},
),
),
)
expect(results.every((result) => result.exitCode === 0)).toBe(true)
expect(output).toBe(value.repeat(2))
expect(SandboxManager.initialize).toHaveBeenCalledTimes(1)
expect(SandboxManager.reset).not.toHaveBeenCalled()
await shell.close()
expect(SandboxManager.reset).toHaveBeenCalledTimes(1)
})
it('fails startup and cleans up if sandbox initialization or execution fails', async () => {
const settings = path.join(cwd, 'sandbox.json')
await writeFile(
settings,
JSON.stringify({
network: { allowedDomains: [], deniedDomains: [] },
filesystem: { denyRead: [], allowWrite: ['.pi'], denyWrite: [] },
}),
)
shell = new RunnerShell(new RunnerFiles(cwd), settings)
vi.mocked(SandboxManager.initialize).mockRejectedValueOnce(
new Error('Run windows-install first'),
)
await expect(shell.initialize()).rejects.toThrow('windows-install')
expect(SandboxManager.reset).toHaveBeenCalledTimes(1)
expect(
await shell.run(request('echo unsafe'), new AbortController().signal, () => {}),
).toHaveProperty('error', 'SRT has not been initialized.')
vi.mocked(SandboxManager.wrapWithSandboxArgv).mockRejectedValueOnce(
new Error('sandbox unavailable'),
)
await expect(shell.initialize()).rejects.toThrow('SRT preflight failed: sandbox unavailable')
})
it('fails closed for missing or malformed SRT settings', async () => {
const settings = path.join(cwd, 'sandbox.json')
await expect(new RunnerShell(new RunnerFiles(cwd), settings).initialize()).rejects.toThrow()
await writeFile(settings, 'invalid json')
await expect(new RunnerShell(new RunnerFiles(cwd), settings).initialize()).rejects.toThrow()
await writeFile(settings, '[]')
await expect(new RunnerShell(new RunnerFiles(cwd), settings).initialize()).rejects.toThrow(
/object/,
)
await writeFile(settings, JSON.stringify({ network: { allowedDomains: 42 } }))
await expect(new RunnerShell(new RunnerFiles(cwd), settings).initialize()).rejects.toThrow()
expect(SandboxManager.initialize).not.toHaveBeenCalled()
})
})
+232
View File
@@ -0,0 +1,232 @@
import { randomUUID } from 'node:crypto'
import { execa } from 'execa'
import { SandboxManager } from '@anthropic-ai/sandbox-runtime'
import {
runnerEnvironmentSchema,
type RunnerEnvironment,
type RunnerExecResultMessage,
type RunnerExecStartMessage,
} from '@shumai/dtos'
import { commandEnv } from './config'
import { RunnerFiles } from './files'
import { resolveRunnerShell } from './shell-config'
import { loadSandbox } from './srt'
const MAX_OUTPUT_BYTES = 64 * 1024 * 1024
const MAX_TIMEOUT_SEC = 3600
export class RunnerShell {
readonly environment: RunnerEnvironment
private sandbox?: Awaited<ReturnType<typeof loadSandbox>>
private initialized = false
private active = 0
constructor(
private readonly files: RunnerFiles,
private readonly srtPath?: string,
private readonly config = resolveRunnerShell(),
) {
this.environment = runnerEnvironmentSchema.parse({
platform: process.platform,
shell: config.executable,
})
}
async initialize() {
if (!this.srtPath) return
if (this.config.commandTransport === 'stdin')
throw new Error(
'SRT cannot run the legacy WSL Bash launcher. Install Git for Windows to use Bash with SRT.',
)
// Read and validate once; commands cannot weaken the policy by editing the file.
this.sandbox = await loadSandbox(this.srtPath)
try {
await SandboxManager.initialize(this.sandbox.config)
this.initialized = true
let diagnostic = ''
const probe = await this.run(
{ type: 'exec.start', executionId: randomUUID(), command: 'exit 0', timeoutSec: 15 },
new AbortController().signal,
(_stream, data) => {
diagnostic = (diagnostic + data).slice(-4096)
},
)
if (probe.exitCode !== 0 || probe.error || probe.timedOut)
throw new Error(
`SRT preflight failed: ${probe.error || diagnostic.trim() || 'Check settings and local SRT prerequisites.'}`,
)
} catch (error) {
await this.close()
throw error
}
}
async close() {
if (!this.sandbox) return
try {
await SandboxManager.reset()
} finally {
await this.sandbox.cleanup()
this.sandbox = undefined
this.initialized = false
}
}
async run(
message: RunnerExecStartMessage,
signal: AbortSignal,
output: (stream: 'stdout' | 'stderr', data: string) => void,
): Promise<RunnerExecResultMessage> {
const result: RunnerExecResultMessage = {
type: 'exec.result',
executionId: message.executionId,
exitCode: null,
signal: null,
timedOut: false,
aborted: false,
}
if (this.srtPath) this.active++
try {
if (signal.aborted) return { ...result, aborted: true }
const timeout = message.timeoutSec ?? 120
if (!Number.isFinite(timeout) || timeout <= 0 || timeout > MAX_TIMEOUT_SEC)
throw new Error(`Command timeout must be between 0 and ${MAX_TIMEOUT_SEC} seconds.`)
const logPath = `.pi/logs/bash-${message.executionId}.log`
const log = await this.files.createWriter(logPath)
try {
const windows = process.platform === 'win32'
const { commandTransport } = this.config
const useShell = commandTransport === 'shell'
const sendCommandOnStdin = commandTransport === 'stdin'
let executable = useShell ? message.command : this.config.executable
let args = useShell ? [] : sendCommandOnStdin ? ['-s'] : ['-c', message.command]
let shell: string | false = useShell && this.config.executable
let env = commandEnv(message.env)
if (this.srtPath) {
if (!this.initialized) throw new Error('SRT has not been initialized.')
const skillEnv = commandEnv(message.env, false)
const command = windows
? message.command
: [
...Object.entries(skillEnv).map(
([key, value]) => `export ${key}='${value!.replaceAll("'", "'\\''")}'`,
),
message.command,
].join('\n')
const wrapped = await SandboxManager.wrapWithSandboxArgv(
command,
this.config.executable,
undefined,
signal,
this.files.cwd,
{ commandId: message.executionId, commandText: message.command },
)
;[executable, ...args] = wrapped.argv
shell = false
if (windows) {
// SRT's broker starts a fresh sandbox-user environment. Explicit skill
// values belong in its child overlay, never in the broker's startup env.
const separator = args.indexOf('--')
if (separator < 0) throw new Error('SRT returned an invalid Windows launch command.')
args.splice(
separator,
0,
...Object.entries(skillEnv).flatMap(([key, value]) => ['--env', `${key}=${value}`]),
)
env = wrapped.env
} else env = commandEnv()
}
if (signal.aborted) return { ...result, aborted: true }
const deadline = Date.now() + Math.ceil(timeout * 1000)
const child = execa(executable, args, {
cwd: this.files.cwd,
env,
extendEnv: false,
shell,
stdin: sendCommandOnStdin ? [Buffer.from(message.command)] : 'ignore',
buffer: false,
reject: false,
stripFinalNewline: false,
verbose: 'none',
windowsHide: true,
killDescendants: true,
killSignal: 'SIGKILL',
cancelSignal: signal,
timeout: Math.ceil(timeout * 1000),
})
let bytes = 0
let writes = Promise.resolve()
let drainTimer: ReturnType<typeof setTimeout> | undefined
let drainExpired = false
let exitedWithinDeadline = false
child.nodeChildProcess.once('exit', (exitCode) => {
exitedWithinDeadline = exitCode !== null && Date.now() < deadline
// Execa cleans up descendants where the OS still permits it. Bound
// draining if a background child keeps inherited output pipes open.
child.kill('SIGKILL')
drainTimer = setTimeout(() => {
drainExpired = true
child.stdout.destroy()
child.stderr.destroy()
}, 1000)
})
try {
const readers = (['stdout', 'stderr'] as const).map(async (stream) => {
child[stream].setEncoding('utf8')
try {
for await (const data of child[stream]) {
bytes += Buffer.byteLength(data)
if (bytes > MAX_OUTPUT_BYTES)
throw new Error('Command exceeded the 64 MiB output limit.')
await (writes = writes.then(async () => {
await log.write(data)
output(stream, data)
}))
}
} catch (error) {
if (
drainExpired &&
error instanceof Error &&
'code' in error &&
error.code === 'ERR_STREAM_PREMATURE_CLOSE'
)
return
result.error ??= error instanceof Error ? error.message : 'Output streaming failed.'
child.kill('SIGKILL')
}
})
const [completed] = await Promise.all([child, Promise.all(readers)])
await writes
result.exitCode = completed.exitCode ?? null
result.signal = completed.signal ?? null
// Execa's timeout includes stream draining. Report the command's
// runtime, so a timely exit does not become a timeout while draining.
result.timedOut = completed.timedOut && !exitedWithinDeadline
result.aborted = completed.isCanceled
if (
completed.failed &&
completed.exitCode === undefined &&
!result.timedOut &&
!result.aborted &&
!completed.signal
)
result.error = `Command execution failed (${completed.code ?? 'unknown error'}).`
result.fullOutputPath = logPath
} finally {
clearTimeout(drainTimer)
}
} finally {
await log.close()
}
} catch (error) {
result.aborted = signal.aborted
result.error = error instanceof Error ? error.message : 'Command execution failed.'
} finally {
// SRT's temporary mount resources are shared: clean up only after every
// concurrent command has finished, and keep the proxies until shutdown.
if (this.srtPath && --this.active === 0 && this.initialized)
SandboxManager.cleanupAfterCommand()
}
return result
}
}
+2
View File
@@ -0,0 +1,2 @@
// Release builds embed SRT's native helpers through scripts/srt-bundle.ts.
export const bundledSrtVendor: Record<string, string> | undefined = undefined
+59
View File
@@ -0,0 +1,59 @@
import { mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import path from 'node:path'
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { loadSandbox } from './srt'
describe('runner SRT configuration', () => {
let directory: string
let settings: string
const config = {
network: { allowedDomains: [], deniedDomains: [] },
filesystem: { denyRead: [], allowWrite: ['.pi'], denyWrite: [] },
}
beforeEach(async () => {
directory = await mkdtemp(path.join(tmpdir(), 'shumai-srt-settings-'))
settings = path.join(directory, 'sandbox.json')
})
afterEach(() => rm(directory, { recursive: true, force: true }))
it('validates upstream settings, supplies packaged helpers, and removes them on cleanup', async () => {
await writeFile(settings, JSON.stringify(config))
const sandbox = await loadSandbox(settings)
const helper = sandbox.config.javaAgentJarPath!
expect(sandbox.config).toMatchObject(config)
expect((await readFile(helper)).subarray(0, 2).toString()).toBe('PK')
if (process.platform === 'linux')
expect(await stat(sandbox.config.seccomp!.applyPath!)).toHaveProperty(
'size',
expect.any(Number),
)
if (process.platform === 'win32')
expect(await stat(sandbox.config.windows!.srtWin!.path!)).toHaveProperty(
'size',
expect.any(Number),
)
await sandbox.cleanup()
await expect(stat(helper)).rejects.toThrow()
})
it('preserves explicit upstream helper paths', async () => {
await writeFile(
settings,
JSON.stringify({
...config,
javaAgentJarPath: '/custom/proxy.jar',
seccomp: { applyPath: '/custom/seccomp' },
windows: { srtWin: { path: 'C:\\custom\\srt-win.exe' } },
}),
)
const sandbox = await loadSandbox(settings)
try {
expect(sandbox.config.javaAgentJarPath).toBe('/custom/proxy.jar')
expect(sandbox.config.seccomp?.applyPath).toBe('/custom/seccomp')
expect(sandbox.config.windows?.srtWin?.path).toBe('C:\\custom\\srt-win.exe')
} finally {
await sandbox.cleanup()
}
})
})
+59
View File
@@ -0,0 +1,59 @@
import { createRequire } from 'node:module'
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import path from 'node:path'
import {
SandboxRuntimeConfigSchema,
type SandboxRuntimeConfig,
} from '@anthropic-ai/sandbox-runtime'
import { bundledSrtVendor } from './srt-vendor'
/** Load the upstream JSON format and supply the helpers bundled with this CLI. */
export async function loadSandbox(configPath: string) {
const config = SandboxRuntimeConfigSchema.parse(JSON.parse(await readFile(configPath, 'utf8')))
const directory = await mkdtemp(path.join(tmpdir(), 'shumai-srt-'))
const cleanup = () => rm(directory, { recursive: true, force: true })
const asset = async (name: string) => {
const encoded = bundledSrtVendor?.[name]
const bytes =
encoded !== undefined
? Buffer.from(encoded, 'base64')
: await readFile(
path.join(
path.dirname(
createRequire(import.meta.url).resolve(
'@anthropic-ai/sandbox-runtime/package.json',
),
),
'vendor',
name,
),
)
const destination = path.join(directory, path.basename(name))
await writeFile(destination, bytes, { mode: 0o500 })
return destination
}
try {
if (process.platform === 'linux' && !config.seccomp?.applyPath) {
config.seccomp = {
...config.seccomp,
applyPath: await asset(`seccomp/${process.arch}/apply-seccomp`),
}
}
if (process.platform === 'win32' && !config.windows?.srtWin?.path) {
config.windows = {
...config.windows,
srtWin: {
...config.windows?.srtWin,
path: await asset(`srt-win/${process.arch}/srt-win.exe`),
},
}
}
if (!config.javaAgentJarPath)
config.javaAgentJarPath = await asset('java-proxy-agent/srt-proxy-agent.jar')
return { config: config satisfies SandboxRuntimeConfig, cleanup }
} catch (error) {
await cleanup()
throw error
}
}
+11
View File
@@ -81,6 +81,16 @@ if (regenCmdIndex !== -1) {
async function run() {
validateArchiveStorageConfig()
const runnerLease = await runnerService.startServerTopology()
runnerLease.addEventListener(
'abort',
() => {
console.error('Lost the runner server lease:', runnerLease.reason)
process.exit(1)
},
{ once: true },
)
runnerLease.throwIfAborted()
// Initialize workflows and activities for local executor mode
initAgentWorkflows()
initTranscodeWorkflows()
@@ -222,6 +232,7 @@ async function run() {
await workflowService.shutdownWorkers().catch(() => {})
assetService.stopCleanupJob()
server.stop(true)
await runnerService.stopServerTopology().catch(console.error)
process.exit(0)
}
+82 -10
View File
@@ -37,15 +37,25 @@
"shumai-cli": "./src/index.ts",
},
"dependencies": {
"@anthropic-ai/sandbox-runtime": "0.0.83",
"@shumai/api": "workspace:*",
"@shumai/dtos": "workspace:*",
"@shumai/operations": "workspace:*",
"@shumai/tableprinter": "workspace:*",
"commander": "15.0.0",
"execa": "10.2.1",
"hono": "4.13.5",
"ulid": "3.0.2",
"ws": "8.22.2",
"yauzl": "3.4.0",
"zod": "4.5.4",
},
"devDependencies": {
"@types/adm-zip": "0.5.8",
"@types/ws": "8.18.2",
"@types/yauzl": "3.4.0",
"adm-zip": "0.6.1",
},
},
"apps/premiere-uxp": {
"name": "@shumai/premiere-uxp",
@@ -474,6 +484,8 @@
"@alloc/quick-lru": ["@alloc/quick-lru@5.2.0", "", {}, "sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw=="],
"@anthropic-ai/sandbox-runtime": ["@anthropic-ai/sandbox-runtime@0.0.83", "", { "dependencies": { "@pondwader/socks5-server": "^1.0.10", "commander": "^12.1.0", "node-forge": "^1.4.0", "zod": "^3.25.0" }, "peerDependencies": { "@microsoft/mxc-sdk": "^1.0.1" }, "optionalPeers": ["@microsoft/mxc-sdk"], "bin": { "srt": "dist/cli.js" } }, "sha512-yEusORnVJrNseF/9fvjVYyk6tr79/aFbIB3T/X8p2puLZdaaiGNeYN50v0CE+1DT70v7XuzKOAnTivh1f5cksA=="],
"@anthropic-ai/sdk": ["@anthropic-ai/sdk@0.129.0", "", { "dependencies": { "json-schema-to-ts": "^3.1.1", "standardwebhooks": "^1.0.0" }, "peerDependencies": { "zod": "^3.25.0 || ^4.0.0" }, "optionalPeers": ["zod"], "bin": { "anthropic-ai-sdk": "bin/cli" } }, "sha512-MH7LB20kNpGLUpPTe2OG5XvL/KhX8HUO9XyBaFjgFk6TLS4Xjt0VPIuMKywe7POKKPslX+QxSlVok9e+8kG5Nw=="],
"@aws-sdk/checksums": ["@aws-sdk/checksums@3.1000.29", "", { "dependencies": { "@aws-sdk/core": "^3.977.9", "@aws-sdk/types": "^3.974.5", "@smithy/core": "^3.33.3", "@smithy/types": "^4.17.2", "tslib": "^2.6.2" } }, "sha512-Dtu0gr4dnATZAPwEYbpCsG+MpLM7OAliy2gTepEFQwl1vZ6DL3QMH2FveMa3HLvPsOdhJsPRB3KtxVhph9T75A=="],
@@ -886,6 +898,8 @@
"@playwright/test": ["@playwright/test@1.61.1", "", { "dependencies": { "playwright": "1.61.1" }, "bin": { "playwright": "cli.js" } }, "sha512-8nKv6+0RJSL9FE4jYOEGXnPeM/Hg12qZpmqzZjRh3qM0Y7c3z1mrOTfFLids72RDQYVh9WpLEfR5WdpNX4fkig=="],
"@pondwader/socks5-server": ["@pondwader/socks5-server@1.0.10", "", {}, "sha512-bQY06wzzR8D2+vVCUoBsr5QS2U6UgPUQRmErNwtsuI6vLcyRKkafjkr3KxbtGFf9aBBIV2mcvlsKD1UYaIV+sg=="],
"@preact/signals-core": ["@preact/signals-core@1.14.4", "", {}, "sha512-HNB6HYeYKhQbJ1aKl+YRjrS4+QWHLKX6qKoUsfS/m0vqzsVaEBiZiaKbG/e+NKk2ch5ALQr/ihWaMHxiCuuWHA=="],
"@prisma/adapter-pg": ["@prisma/adapter-pg@7.10.0", "", { "dependencies": { "@prisma/driver-adapter-utils": "7.10.0", "@types/pg": "^8.16.0", "pg": "^8.16.3", "postgres-array": "3.0.4" } }, "sha512-N7nwSor0HO1Kz6xBv0TPAjAPysKK0fac6p4fVN3ensLOuzc/83Fgmln5k92eK/cvzqdkSR/2kkAqlbcdwVrwpw=="],
@@ -1094,6 +1108,8 @@
"@rolldown/pluginutils": ["@rolldown/pluginutils@1.0.1", "", {}, "sha512-2j9bGt5Jh8hj+vPtgzPtl72j0yRxHAyumoo6TNfAjsLB04UtpSvPbPcDcBMxz7n+9CYB0c1GxQFxYRg2jimqGw=="],
"@sec-ant/readable-stream": ["@sec-ant/readable-stream@0.4.1", "", {}, "sha512-831qok9r2t8AlxLko40y2ebgSDhenenCatLVeW/uBtnHPyhHOvG0C7TvfgecV+wHzIm5KUICgzmVpWS+IMEAeg=="],
"@shumai/agent": ["@shumai/agent@workspace:packages/agent"],
"@shumai/api": ["@shumai/api@workspace:packages/api"],
@@ -1130,6 +1146,8 @@
"@sindresorhus/is": ["@sindresorhus/is@4.6.0", "", {}, "sha512-t09vSN3MdfsyCHoFcTRCH/iUtG7OJ0CsjzB8cjAmKc/va/kIgeDI/TxsigdncE/4be734m0cvIYwNaV4i2XqAw=="],
"@sindresorhus/merge-streams": ["@sindresorhus/merge-streams@4.0.0", "", {}, "sha512-tlqY9xq5ukxTUZBmoOp+m61cqwQD5pHJtFY3Mn8CA8ps6yghLH/Hw8UPdqg4OLmFW3IFlcXnQNmo/dh8HzXYIQ=="],
"@smithy/core": ["@smithy/core@3.33.3", "", { "dependencies": { "@smithy/types": "^4.17.2", "tslib": "^2.6.2" } }, "sha512-CsOeKq/9kA3y6VJHt+/+VTCtBaxJ4OTFpgrjIUhPpDIKxBci1k2bJaQASF2h/ELWrulGp+t97DZ0mevfAD8idg=="],
"@smithy/credential-provider-imds": ["@smithy/credential-provider-imds@4.5.2", "", { "dependencies": { "@smithy/core": "^3.33.2", "@smithy/types": "^4.17.2", "tslib": "^2.6.2" } }, "sha512-A9uSdn72ozbRUSit0eib0TW7nXuNPlaeM0zcGkJ+nE6tFcSDbnmtwoxbTCFBukVQcszDAyvsd7+rTduPTXpygg=="],
@@ -1514,9 +1532,9 @@
"@types/whatwg-mimetype": ["@types/whatwg-mimetype@3.0.2", "", {}, "sha512-c2AKvDT8ToxLIOUlN51gTiHXflsfIFisS4pO7pDPoKouJCESkhZnEy623gwP9laCy5lnLDAw1vAzu2vM2YLOrA=="],
"@types/ws": ["@types/ws@8.18.1", "", { "dependencies": { "@types/node": "*" } }, "sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg=="],
"@types/ws": ["@types/ws@8.18.2", "", { "dependencies": { "@types/node": "*" } }, "sha512-67MQl+fpWKVTT1NYdnmo3U4sc/xPo/zQBncVnI74qmQa0z/b+1g6iYqNmGCPbxO+zz2aklb08a0oHfegiVd0/w=="],
"@types/yauzl": ["@types/yauzl@2.10.3", "", { "dependencies": { "@types/node": "*" } }, "sha512-oJoftv0LSuaDZE3Le4DbKX+KS9G36NzOeSap90UIK0yMA/NhKJhqlSGtNDORNRaIbQfzjXDrQa0ytJ6mNRGz/Q=="],
"@types/yauzl": ["@types/yauzl@3.4.0", "", { "dependencies": { "@types/node": "*" } }, "sha512-NRPn5w6h8dhcnmx3YIRQcqMywY/+nND/uOkJessedcrowO3C0AssHp3tMJpxKAwOhFOo0OV1y9VtsC5hbKKBAw=="],
"@typescript-eslint/eslint-plugin": ["@typescript-eslint/eslint-plugin@8.68.0", "", { "dependencies": { "@eslint-community/regexpp": "^4.12.2", "@typescript-eslint/scope-manager": "8.68.0", "@typescript-eslint/type-utils": "8.68.0", "@typescript-eslint/utils": "8.68.0", "@typescript-eslint/visitor-keys": "8.68.0", "ignore": "^7.0.5", "natural-compare": "^1.4.0", "ts-api-utils": "^2.5.0" }, "peerDependencies": { "@typescript-eslint/parser": "^8.68.0", "eslint": "^8.57.0 || ^9.0.0 || ^10.0.0", "typescript": ">=4.8.4 <6.1.0" } }, "sha512-WASHDpCm6qO5jj9g1a+8NiW5+GCkAyLReR56/4VruYmNgfUmqpxOfZ2Yfb8xGfJPWv5Qi6LSD8sXdces3vbp/Q=="],
@@ -1974,6 +1992,8 @@
"eventsource-parser": ["eventsource-parser@3.1.1", "", {}, "sha512-EKN1vKAMcZ8MlYMpaNuxN6R9yakzH6uajHcHVTqWJzvu5pWw9DyhbP35HH8MVBQ+dZjAfDxk+A8NiR9KWaXiyQ=="],
"execa": ["execa@10.2.1", "", { "dependencies": { "@sindresorhus/merge-streams": "^4.0.0", "figures": "^6.1.0", "get-stream": "^9.0.1", "human-signals": "^8.0.1", "is-plain-obj": "^4.1.0", "is-stream": "^4.0.1", "npm-run-path": "^6.0.0", "pretty-ms": "^9.3.1", "signal-exit": "^4.1.0", "strip-final-newline": "^4.0.0", "which-command": "^0.1.0", "yoctocolors": "^2.2.0" } }, "sha512-/Bofl3ZBh2Q4/PG0W63qFzP5vDAypi8Zc90UWTBzaDNIYiX7apPOd0N6L8hG8CojUdq4V51adu8BObP+iVsSTw=="],
"exiftool-vendored": ["exiftool-vendored@38.1.0", "", { "dependencies": { "@photostructure/tz-lookup": "^11.7.0", "@types/luxon": "^3.7.5", "batch-cluster": "^19.2.0", "he": "^1.2.0", "luxon": "^3.7.2" }, "optionalDependencies": { "exiftool-vendored.exe": "13.59.2", "exiftool-vendored.pl": "13.59.2" } }, "sha512-LIERqM/Qb0oqJlSswLFKWsxGVH81skp7omsX2BPKA/dEzfLP5syNO5KL7k5UlMCf+m0jqEXrKTQr/xV5vScPow=="],
"exiftool-vendored.exe": ["exiftool-vendored.exe@13.59.2", "", { "os": "win32" }, "sha512-U0l0CApIGO+AjpFYCiNCApQsMdlf6sqCyEl4wlKKqvwVAEVcvMRw91k7zj32OPFZE+/+XlLFsc6876MpOqBLiw=="],
@@ -2018,6 +2038,8 @@
"fflate": ["fflate@0.8.3", "", {}, "sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA=="],
"figures": ["figures@6.1.0", "", { "dependencies": { "is-unicode-supported": "^2.0.0" } }, "sha512-d+l3qxjSesT4V7v2fh+QnmFnUWv9lSpjarhShNTgBOfA0ttejbQUAlHLitbjkoRiDulW0OPoQPYIGhIC8ohejg=="],
"file-entry-cache": ["file-entry-cache@8.0.0", "", { "dependencies": { "flat-cache": "^4.0.0" } }, "sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ=="],
"find-my-way": ["find-my-way@9.7.0", "", { "dependencies": { "fast-deep-equal": "^3.1.3", "fast-querystring": "^1.0.0", "safe-regex2": "^5.0.0" } }, "sha512-f2JHn75x2JlwUwLenZypgczR7YWMb/uO9BvUXtus+JMgkbIkLADd38cI4EiV+OQqrGo1Zlq6V8wnqMJ8e62wUQ=="],
@@ -2074,7 +2096,7 @@
"get-proto": ["get-proto@1.0.1", "", { "dependencies": { "dunder-proto": "^1.0.1", "es-object-atoms": "^1.0.0" } }, "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g=="],
"get-stream": ["get-stream@5.2.0", "", { "dependencies": { "pump": "^3.0.0" } }, "sha512-nBF+F1rAZVCu/p7rjzgA+Yb4lfYXrpl7a6VmJrU8wF9I1CKvP/QwPNZHnOlwbTkY6dvtFIzFMSyQXbLoTQPRpA=="],
"get-stream": ["get-stream@9.0.1", "", { "dependencies": { "@sec-ant/readable-stream": "^0.4.1", "is-stream": "^4.0.1" } }, "sha512-kVCxPF3vQM/N0B1PmoqVUqgHP+EeVjmZSQn+1oCRPxd2P21P2F19lIgbR3HBosbB1PUhOAoctJnfEn2GbN2eZA=="],
"get-symbol-description": ["get-symbol-description@1.1.0", "", { "dependencies": { "call-bound": "^1.0.3", "es-errors": "^1.3.0", "get-intrinsic": "^1.2.6" } }, "sha512-w9UMqWwJxHNOvoNzSJ2oPF5wvYcvP7jUvYzhp67yEhTi17ZDBBC1z9pTdGuzjD+EFIqLSYRweZjqfiPzQ06Ebg=="],
@@ -2134,6 +2156,8 @@
"https-proxy-agent": ["https-proxy-agent@9.1.0", "", { "dependencies": { "agent-base": "9.0.0", "debug": "^4.3.4", "proxy-agent-negotiate": "1.1.0" } }, "sha512-ag87y7cJJ9/3+GxFr8Oy4O5faDsGRGnBGsJj/YjOSsSx/5eadKLYTMPlzuR6obgoCDDm0abAAZitXXQkMOPSpA=="],
"human-signals": ["human-signals@8.0.1", "", {}, "sha512-eKCa6bwnJhvxj14kZk5NCPc6Hb6BdsU9DZcOnmQKSnO1VKrfV0zCvtttPZUsBvjmNDn8rpcJfpwSYnHBjc95MQ=="],
"iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="],
"ieee754": ["ieee754@1.2.1", "", {}, "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA=="],
@@ -2208,7 +2232,7 @@
"is-shared-array-buffer": ["is-shared-array-buffer@1.0.4", "", { "dependencies": { "call-bound": "^1.0.3" } }, "sha512-ISWac8drv4ZGfwKl5slpHG9OwPNty4jOWPRIhBpxOoD+hqITiwuipOQ2bNthAzwA3B4fIjO4Nln74N0S9byq8A=="],
"is-stream": ["is-stream@2.0.1", "", {}, "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg=="],
"is-stream": ["is-stream@4.0.1", "", {}, "sha512-Dnz92NInDqYckGEUJv689RbRiTSEHCQ7wOVeALbkOz999YpqT46yMRIGtSNl2iCL1waAZSx40+h59NV/EwzV/A=="],
"is-string": ["is-string@1.1.1", "", { "dependencies": { "call-bound": "^1.0.3", "has-tostringtag": "^1.0.2" } }, "sha512-BtEeSsoaQjlSPBemMQIrY1MY0uM6vnS1g5fmufYOtnxLGUZM2178PKbhsk7Ffv58IX+ZtcvoGwccYsh0PglkAA=="],
@@ -2216,6 +2240,8 @@
"is-typed-array": ["is-typed-array@1.1.15", "", { "dependencies": { "which-typed-array": "^1.1.16" } }, "sha512-p3EcsicXjit7SaskXHs1hA91QxgTw46Fv6EFKKGS5DRFLD8yKnohjF3hxoju94b/OcMZoQukzpPpBE9uLVKzgQ=="],
"is-unicode-supported": ["is-unicode-supported@2.1.0", "", {}, "sha512-mE00Gnza5EEB3Ds0HfMyllZzbBrmLOX3vfWoj9A9PEnTfratQ/BcaJOuMhnkhjXvb2+FkY3VuHqtAGpTPmglFQ=="],
"is-weakmap": ["is-weakmap@2.0.2", "", {}, "sha512-K5pXYOm9wqY1RgjpL3YTkF39tni1XajUIkawTLUo9EZEVUFga5gSQJF8nNS7ZwJQ02y+1YCNYcMh+HIf1ZqE+w=="],
"is-weakref": ["is-weakref@1.1.1", "", { "dependencies": { "call-bound": "^1.0.3" } }, "sha512-6i9mGWSlqzNMEqpCp93KwRS1uUOodk2OJ6b+sq7ZPDSy2WuI5NFIxp/254TytR8ftefexkWn5xNiHUNpPOfSew=="],
@@ -2440,12 +2466,16 @@
"node-fetch": ["node-fetch@3.3.2", "", { "dependencies": { "data-uri-to-buffer": "^4.0.0", "fetch-blob": "^3.1.4", "formdata-polyfill": "^4.0.10" } }, "sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA=="],
"node-forge": ["node-forge@1.4.3", "", {}, "sha512-P6wzfh6t70avmV8FrVEAd/36sOPJQ+hkaJVX1Laap9J/77m7M0eyozDvc8SwnDf4P7MP94gwbAAo0rg3YVXT+g=="],
"node-releases": ["node-releases@2.0.54", "", {}, "sha512-YHs7BmmcsdAI5Ozuf8JZo6PT0mv2GIWC9vMfvUC3dp65M8hn7Ux8CPL+2oBI7juNuj9d0ndhTcznq2ODBps9cQ=="],
"nodemailer": ["nodemailer@10.0.10", "", {}, "sha512-He9XskOFms62SyAKkLu8CcGcYHAo+BSHSsORI8s4PJH8qZgZY4L4lDfvyN0twvmbpyG+eGrxpyBlskj9d9rJ8A=="],
"normalize-path": ["normalize-path@3.0.0", "", {}, "sha512-6eZs5Ls3WtCisHWp9S2GUy8dqkpGi4BVSz3GaqiE6ezub0512ESztXUwUB6C6IKbQkY2Pnb/mD4WYojCRwcwLA=="],
"npm-run-path": ["npm-run-path@6.0.0", "", { "dependencies": { "path-key": "^4.0.0", "unicorn-magic": "^0.3.0" } }, "sha512-9qny7Z9DsQU8Ou39ERsPU4OZQlSTP47ShQzuKZ6PRXpYLtIFgl/DEBYEXKlvcEa+9tHVcK8CF81Y2V72qaZhWA=="],
"object-assign": ["object-assign@4.1.1", "", {}, "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg=="],
"object-inspect": ["object-inspect@1.13.4", "", {}, "sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew=="],
@@ -2490,6 +2520,8 @@
"parse-entities": ["parse-entities@4.0.2", "", { "dependencies": { "@types/unist": "^2.0.0", "character-entities-legacy": "^3.0.0", "character-reference-invalid": "^2.0.0", "decode-named-character-reference": "^1.0.0", "is-alphanumerical": "^2.0.0", "is-decimal": "^2.0.0", "is-hexadecimal": "^2.0.0" } }, "sha512-GG2AQYWoLgL877gQIKeRPGO1xF9+eG1ujIb5soS5gPvLQ1y2o8FL90w2QWNdf9I361Mpp7726c+lj3U0qK1uGw=="],
"parse-ms": ["parse-ms@4.0.0", "", {}, "sha512-TXfryirbmq34y8QBwgqCVLi+8oA3oWx2eAnSn62ITyEhEYaWRlVZ2DvMM9eZbMs/RfxPu/PK/aBLyGj4IrqMHw=="],
"partial-json": ["partial-json@0.1.7", "", {}, "sha512-Njv/59hHaokb/hRUjce3Hdv12wd60MtM9Z5Olmn+nehe0QDAsRtRbJPvJ0Z91TusF0SuZRIvnM+S4l6EIP8leA=="],
"path-exists": ["path-exists@4.0.0", "", {}, "sha512-ak9Qy5Q7jYb2Wwcey5Fpvg2KoAc/ZIhLSLOSBmRmygPsGwkVVt0fZa0qrtMz+m6tJTAHfZQ8FnmB4MG4LWy7/w=="],
@@ -2572,6 +2604,8 @@
"pretty-format": ["pretty-format@27.5.1", "", { "dependencies": { "ansi-regex": "^5.0.1", "ansi-styles": "^5.0.0", "react-is": "^17.0.1" } }, "sha512-Qb1gy5OrP5+zDf2Bvnzdl3jsTf1qXVMazbvCoKhtKqVs4/YK4ozX4gKQJJVyNe+cajNPn0KoC0MC3FUmaHWEmQ=="],
"pretty-ms": ["pretty-ms@9.3.1", "", { "dependencies": { "parse-ms": "^4.0.0" } }, "sha512-HzMy3Geq23nVALD/M2LliU+F+M+gVNsvkQWWqeBZ8HDiCgzo6YPJ/Omrmtq24EFrIsk0a3EkQGEd7bDOo+IhGA=="],
"prisma": ["prisma@7.10.0", "", { "dependencies": { "@prisma/config": "7.10.0", "@prisma/dev": "0.24.17", "@prisma/engines": "7.10.0", "@prisma/studio-core": "0.33.0", "mysql2": "3.15.3", "postgres": "3.4.7" }, "peerDependencies": { "better-sqlite3": ">=9.0.0", "typescript": ">=5.4.0" }, "optionalPeers": ["better-sqlite3", "typescript"], "bin": { "prisma": "build/index.js" } }, "sha512-o0ornyJOWgygVAzGCpr8PdXV8EJLHyVGDDUr/voBQt8Azzw8cYTByzzPGcA/m4tCkPcnJA8raEOv2CslsKhPEw=="],
"prisma-json-types-generator": ["prisma-json-types-generator@5.1.1", "", { "dependencies": { "@prisma/generator-helper": "^7.8.0", "semver": "^7.8.5", "try": "^1.0.3", "tslib": "^2.8.1", "typescript": "npm:typescript@^6.0.3" }, "peerDependencies": { "@prisma/client": "^7.8.0", "prisma": "^7.8.0" }, "bin": { "prisma-json-types-generator": "index.js" } }, "sha512-vtj68YkCla7bEnblC54xzKU3w+H/1uoKxtLU3F2eC9LL8j1Dz8GadVagKhQqQd03KGfHmO7o83Tf3yXxCKQwzQ=="],
@@ -2734,7 +2768,7 @@
"siginfo": ["siginfo@2.0.0", "", {}, "sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g=="],
"signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="],
"signal-exit": ["signal-exit@4.1.0", "", {}, "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw=="],
"skin-tone": ["skin-tone@2.0.0", "", { "dependencies": { "unicode-emoji-modifier-base": "^1.0.0" } }, "sha512-kUMbT1oBJCpgrnKoSr0o6wPtvRWT9W9UKvGLwfJYO2WuahZRHOpEyL1ckyMGgMWh0UdpmaoFqKKD29WTomNEGA=="],
@@ -2790,6 +2824,8 @@
"strip-ansi-cjs": ["strip-ansi@6.0.1", "", { "dependencies": { "ansi-regex": "^5.0.1" } }, "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A=="],
"strip-final-newline": ["strip-final-newline@4.0.0", "", {}, "sha512-aulFJcD6YK8V1G7iRB5tigAP4TsHBZZrOV8pjV++zdUwmeV8uzbY7yn6h9MswN62adStNZFuCIx4haBnRuMDaw=="],
"strip-json-comments": ["strip-json-comments@5.0.3", "", {}, "sha512-1tB5mhVo7U+ETBKNf92xT4hrQa3pm0MZ0PQvuDnWgAAGHDsfp4lPSpiS6psrSiet87wyGPh9ft6wmhOMQ0hDiw=="],
"style-to-js": ["style-to-js@1.1.21", "", { "dependencies": { "style-to-object": "1.0.14" } }, "sha512-RjQetxJrrUJLQPHbLku6U/ocGtzyjbJMP9lCNK7Ag0CNh690nSH8woqWH9u16nMjYBAok+i7JO1NP2pOy8IsPQ=="],
@@ -2888,6 +2924,8 @@
"unicode-trie": ["unicode-trie@2.0.0", "", { "dependencies": { "pako": "^0.2.5", "tiny-inflate": "^1.0.0" } }, "sha512-x7bc76x0bm4prf1VLg79uhAzKw8DVboClSN5VxJuQ+LKDOVEW9CdH+VY7SP+vX7xCYQqzzgQpFqz15zeLvAtZQ=="],
"unicorn-magic": ["unicorn-magic@0.3.0", "", {}, "sha512-+QBBXBCvifc56fsbuxZQ6Sic3wqqc3WWaqxs58gvJrcOuN83HGTCwz3oS5phzU9LthRNE9VrJCFCLUgHeeFnfA=="],
"unified": ["unified@11.0.5", "", { "dependencies": { "@types/unist": "^3.0.0", "bail": "^2.0.0", "devlop": "^1.0.0", "extend": "^3.0.0", "is-plain-obj": "^4.0.0", "trough": "^2.0.0", "vfile": "^6.0.0" } }, "sha512-xKvGhPWw3k84Qjh8bI3ZeJjqnyadK+GEFtazSfZv/rKeTkTjOJho6mFqh2SM96iIcZokxiOpg78GazTSg8+KHA=="],
"unist-util-is": ["unist-util-is@6.0.1", "", { "dependencies": { "@types/unist": "^3.0.0" } }, "sha512-LsiILbtBETkDz8I9p1dQ0uyRUWuaQzd/cuEeS1hoRSyW5E5XGmTzlwY1OrNzzakGowI9Dr/I8HVaw4hTtnxy8g=="],
@@ -2950,6 +2988,8 @@
"which-collection": ["which-collection@1.0.2", "", { "dependencies": { "is-map": "^2.0.3", "is-set": "^2.0.3", "is-weakmap": "^2.0.2", "is-weakset": "^2.0.3" } }, "sha512-K4jVyjnBdgvc86Y6BkaLZEN933SwYOuBFkdmBu9ZfkcAbdVbpITnDmjvZ/aQjRXQrv5EPkTnD1s39GiiqbngCw=="],
"which-command": ["which-command@0.1.0", "", { "bin": { "which-command": "cli.js" } }, "sha512-XZyoF5/5hZtXitIwzrU4NKK+Wtbb9aB9CezUEw2Q0wlYK8NUYQxC1rRXgNueYLtBAJwXIb+/tFVk4dozciNJMA=="],
"which-typed-array": ["which-typed-array@1.1.22", "", { "dependencies": { "available-typed-arrays": "^1.0.7", "call-bind": "^1.0.9", "call-bound": "^1.0.4", "for-each": "^0.3.5", "get-proto": "^1.0.1", "gopd": "^1.2.0", "has-tostringtag": "^1.0.2" } }, "sha512-fvO4ExWMFsqyhG3AiPAObMuY1lxaqgYcxbc49CNdWDDECOJNgQyvsOWVwbZc+qf3rzRtxojBK+CMEv0Ld5CYpw=="],
"why-is-node-running": ["why-is-node-running@2.3.0", "", { "dependencies": { "siginfo": "^2.0.0", "stackback": "0.0.2" }, "bin": { "why-is-node-running": "cli.js" } }, "sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w=="],
@@ -2964,7 +3004,7 @@
"wrappy": ["wrappy@1.0.2", "", {}, "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ=="],
"ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
"ws": ["ws@8.22.2", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-/rEZ2ejown2N5gxVTHUu9bVVC7ttu0/Tgzf5hbTOixR3Fs7mbbXgn2DbdbcKgJOmpQMtrIUWivtYuVMnp/W/yA=="],
"xtend": ["xtend@4.0.2", "", {}, "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ=="],
@@ -2978,12 +3018,14 @@
"yargs-parser": ["yargs-parser@21.1.1", "", {}, "sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw=="],
"yauzl": ["yauzl@2.10.0", "", { "dependencies": { "buffer-crc32": "~0.2.3", "fd-slicer": "~1.1.0" } }, "sha512-p4a9I6X6nu6IhoGmBqAcbJy1mlC4j27vEPZX9F4L4/vZT3Lyq1VkFHw/V/PUcB9Buo+DG3iHkT0x3Qya58zc3g=="],
"yauzl": ["yauzl@3.4.0", "", { "dependencies": { "pend": "~1.2.0" } }, "sha512-jIH9yLR9wqr0wOS0TpBvo/g/2UgZH5qePVbjgRliiF0BYvOZyaBknKsF+x9Iht0O6sqgnB93rCICdOZFecJuDw=="],
"yjs": ["yjs@13.6.32", "", { "dependencies": { "lib0": "^0.2.99" } }, "sha512-lfiJIIC4Xayt5ItynE407ehlE03pCjeOc4hkR4yxxvvNJ4kuiN25B0g+Qp8XagYz361LLL7DCzR5bvFJ81QKtQ=="],
"yocto-queue": ["yocto-queue@0.1.0", "", {}, "sha512-rVksvsnNCdJ/ohGc6xgPwyN8eheCxsiLM8mxuE/t/mOVqJewPuO1miLpTHQiRgTKCLexL4MeAFVagts7HmNZ2Q=="],
"yoctocolors": ["yoctocolors@2.2.0", "", {}, "sha512-xYqdZFUK/VYazNl/oCDYN+3WloWQwMfZxBoiNt6qNyk+xfOdi598muWE42rNZFp1kNOiqW936q5RhUdnpqElSg=="],
"zeptomatch": ["zeptomatch@2.1.0", "", { "dependencies": { "grammex": "^3.1.11", "graphmatch": "^1.1.0" } }, "sha512-KiGErG2J0G82LSpniV0CtIzjlJ10E04j02VOudJsPyPwNZgGnRKQy7I1R7GMyg/QswnE4l7ohSGrQbQbjXPPDA=="],
"zip-stream": ["zip-stream@6.0.1", "", { "dependencies": { "archiver-utils": "^5.0.0", "compress-commons": "^6.0.2", "readable-stream": "^4.0.0" } }, "sha512-zK7YHHz4ZXpW89AHXUPbQVGKI7uvkd3hzusTdotCg1UxyaVtg0zFJSTfW/Dq5f7OBBVnq6cZIaC8Ti4hb6dtCA=="],
@@ -2996,6 +3038,10 @@
"@ai-sdk/provider-utils/undici": ["undici@7.29.0", "", {}, "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw=="],
"@anthropic-ai/sandbox-runtime/commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="],
"@anthropic-ai/sandbox-runtime/zod": ["zod@3.25.76", "", {}, "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ=="],
"@aws-sdk/client-bedrock-runtime/@aws-sdk/credential-provider-node": ["@aws-sdk/credential-provider-node@3.972.92", "", { "dependencies": { "@aws-sdk/credential-provider-env": "^3.972.80", "@aws-sdk/credential-provider-http": "^3.972.82", "@aws-sdk/credential-provider-ini": "^3.973.25", "@aws-sdk/credential-provider-process": "^3.972.80", "@aws-sdk/credential-provider-sso": "^3.973.24", "@aws-sdk/credential-provider-web-identity": "^3.972.86", "@aws-sdk/types": "^3.975.0", "@smithy/core": "^3.38.0", "@smithy/credential-provider-imds": "^4.5.2", "@smithy/types": "^4.19.0", "tslib": "^2.6.2" } }, "sha512-jC3zmOo7Qxlk+oBivvSsS1Ph7sFGm0QfuhqDGbqgRXi4cxUYMG+dWtBwS2XYYXgc8TIVXJiwgQ65sbNpbYupTA=="],
"@aws-sdk/client-bedrock-runtime/@smithy/node-http-handler": ["@smithy/node-http-handler@4.11.3", "", { "dependencies": { "@smithy/core": "^3.33.3", "@smithy/types": "^4.17.2", "tslib": "^2.6.2" } }, "sha512-2jY1tSpERfPfWqyBV2pH+iGFaghVsIJszJNsT7hxtQYhVJpWDyc0LqOWI+nXOxOAHaEfZ4PXXtp1wW1TGpHhkA=="],
@@ -3012,10 +3058,16 @@
"@dbos-inc/dbos-sdk/commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="],
"@dbos-inc/dbos-sdk/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
"@earendil-works/chord/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
"@earendil-works/pi-ai/@google/genai": ["@google/genai@2.21.0", "", { "dependencies": { "google-auth-library": "^10.3.0", "p-retry": "^4.6.2", "protobufjs": "^7.5.4", "ws": "^8.18.0" }, "peerDependencies": { "@modelcontextprotocol/sdk": "^1.25.2" }, "optionalPeers": ["@modelcontextprotocol/sdk"], "hasInstallScript": true }, "sha512-+PDtco2/Z0ONdzCGekCoCT+O1VJS9xJQNN4XzQpXG/t3El/SWWMkCWlFRO1KmivOHPa4Q0VjUYu1HBKCZ/v33Q=="],
"@eslint-community/eslint-utils/eslint-visitor-keys": ["eslint-visitor-keys@3.4.3", "", {}, "sha512-wpc+LXeiyiisxPlEkUzU6svyS1frIO3Mgxj1fdy7Pm8Ygzguax2N3Fa/D/ag1WqbOprdI+uY6wMUl8/a2G+iag=="],
"@google/genai/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
"@grpc/grpc-js/@grpc/proto-loader": ["@grpc/proto-loader@0.8.1", "", { "dependencies": { "lodash.camelcase": "^4.3.0", "long": "^5.0.0", "protobufjs": "^7.5.5", "yargs": "^17.7.2" }, "bin": { "proto-loader-gen-types": "build/bin/proto-loader-gen-types.js" } }, "sha512-wtF6h+DY6M3YaDBPAmvuuA6jV8Sif9MjtOI5euKFWRgCDl5PeDpPsHR9u2l6St5ceY8AZgoNDww5+HvEsXFsGg=="],
"@inlang/paraglide-js/commander": ["commander@11.1.0", "", {}, "sha512-yPVavfyCcRhmorC7rWlkHn15b4wDVgVmBA7kV4QVBsF7kv/9TKJAbAXVTxvTnwP8HHKjRCJDClKbciiYS7p0DQ=="],
@@ -3042,6 +3094,8 @@
"@reduxjs/toolkit/reselect": ["reselect@5.3.0", "", {}, "sha512-XGoLeRAVzUTcJ1qkxPQhDJyIZ5d6zzZD9nT7AEZOaaU9UbWclhycElmhO+VD5bFeLuzhPBaOV2oXC8uG35ZSpg=="],
"@shumai/cli/adm-zip": ["adm-zip@0.6.1", "", {}, "sha512-Xwrja8nx9e5o2N1my4DsKCeKpdrnACyr1wtbPxBDgGzKzKyE9kRtBFA8mWldI+RVlD7CBZNWY/wQ2+ydwOR6kQ=="],
"@shumai/webui/@types/react-dom": ["@types/react-dom@19.2.5", "", { "peerDependencies": { "@types/react": "^19.2.0" } }, "sha512-fMPwH9v7r/pp43yUd2/Mbiex5KouJwwR3dzHkhLREUC6764VyDsqxhAxv6OFEYR1RhjOyD1naqba8ECDBe7ZQg=="],
"@smithy/node-http-handler/@smithy/types": ["@smithy/types@4.19.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-r7jh49VJxGerfAcTQA6gXcKc+98zOp/tqRwzYjgOE+iSQsP6cEU1hq2QzbuipmP68QtYdY9wKEhiCQZIzHgZ4Q=="],
@@ -3188,6 +3242,8 @@
"@visx/vendor/d3-format": ["d3-format@3.1.0", "", {}, "sha512-YyUI6AEuY/Wpt8KWLgZHsIU86atmikuoOmCfommt0LYHiQSPjvX2AcFc38PX0CBpr2RCyZhjex+NS/LPOv6YqA=="],
"archiver-utils/is-stream": ["is-stream@2.0.1", "", {}, "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg=="],
"bcrypt-pbkdf/tweetnacl": ["tweetnacl@0.14.5", "", {}, "sha512-KXXFFdAbFXY4geFIwoyNK+f5Z1b7swfXABfL7HXCmoIWMKU3dmS26672A4EeQtDzLKy7SXmfBu51JolvEKwtGA=="],
"better-call/@better-fetch/fetch": ["@better-fetch/fetch@1.3.1", "", {}, "sha512-ABkD1WhyfPZprKRQI3bhATjeiFuNWC9PXhfGWqL+sg/gKrM977oFrYkdb4msM3hgUGonr7KlOsOFT5TU2rht9g=="],
@@ -3196,13 +3252,19 @@
"bl/readable-stream": ["readable-stream@3.6.2", "", { "dependencies": { "inherits": "^2.0.3", "string_decoder": "^1.1.1", "util-deprecate": "^1.0.1" } }, "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA=="],
"compress-commons/is-stream": ["is-stream@2.0.1", "", {}, "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg=="],
"docker-modem/readable-stream": ["readable-stream@3.6.2", "", { "dependencies": { "inherits": "^2.0.3", "string_decoder": "^1.1.1", "util-deprecate": "^1.0.1" } }, "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA=="],
"dockerode/tar-fs": ["tar-fs@2.1.5", "", { "dependencies": { "chownr": "^1.1.1", "mkdirp-classic": "^0.5.2", "pump": "^3.0.0", "tar-stream": "^2.1.4" } }, "sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw=="],
"eslint-plugin-react/minimatch": ["minimatch@3.1.5", "", { "dependencies": { "brace-expansion": "^1.1.7" } }, "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w=="],
"foreground-child/signal-exit": ["signal-exit@4.1.0", "", {}, "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw=="],
"extract-zip/@types/yauzl": ["@types/yauzl@2.10.3", "", { "dependencies": { "@types/node": "*" } }, "sha512-oJoftv0LSuaDZE3Le4DbKX+KS9G36NzOeSap90UIK0yMA/NhKJhqlSGtNDORNRaIbQfzjXDrQa0ytJ6mNRGz/Q=="],
"extract-zip/get-stream": ["get-stream@5.2.0", "", { "dependencies": { "pump": "^3.0.0" } }, "sha512-nBF+F1rAZVCu/p7rjzgA+Yb4lfYXrpl7a6VmJrU8wF9I1CKvP/QwPNZHnOlwbTkY6dvtFIzFMSyQXbLoTQPRpA=="],
"extract-zip/yauzl": ["yauzl@2.10.0", "", { "dependencies": { "buffer-crc32": "~0.2.3", "fd-slicer": "~1.1.0" } }, "sha512-p4a9I6X6nu6IhoGmBqAcbJy1mlC4j27vEPZX9F4L4/vZT3Lyq1VkFHw/V/PUcB9Buo+DG3iHkT0x3Qya58zc3g=="],
"gaxios/https-proxy-agent": ["https-proxy-agent@7.0.6", "", { "dependencies": { "agent-base": "^7.1.2", "debug": "4" } }, "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw=="],
@@ -3210,6 +3272,10 @@
"glob/minimatch": ["minimatch@9.0.9", "", { "dependencies": { "brace-expansion": "^2.0.2" } }, "sha512-OBwBN9AL4dqmETlpS2zasx+vTeWclWzkblfZk7KTA5j3jeOONz/tRCnZomUyvNg83wL5Zv9Ss6HMJXAgL8R2Yg=="],
"happy-dom/@types/ws": ["@types/ws@8.18.1", "", { "dependencies": { "@types/node": "*" } }, "sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg=="],
"happy-dom/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
"its-fine/@types/react-reconciler": ["@types/react-reconciler@0.28.9", "", { "peerDependencies": { "@types/react": "*" } }, "sha512-HHM3nxyUZ3zAylX8ZEyrDNd2XZOnQ0D5XfunJF5FLQnZbHHYq4UWvW1QfelQNXv1ICNkwYhfxjwfnqivYB6bFg=="],
"lazystream/readable-stream": ["readable-stream@2.3.8", "", { "dependencies": { "core-util-is": "~1.0.0", "inherits": "~2.0.3", "isarray": "~1.0.0", "process-nextick-args": "~2.0.0", "safe-buffer": "~5.1.1", "string_decoder": "~1.1.1", "util-deprecate": "~1.0.1" } }, "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA=="],
@@ -3220,6 +3286,8 @@
"lit-element/@lit/reactive-element": ["@lit/reactive-element@1.6.3", "", { "dependencies": { "@lit-labs/ssr-dom-shim": "^1.0.0" } }, "sha512-QuTgnG52Poic7uM1AN5yJ09QMe0O28e10XzSvWDz02TJiiKee4stsiownEIadWm8nYzyDAyT+gKzUoZmiWQtsQ=="],
"npm-run-path/path-key": ["path-key@4.0.0", "", {}, "sha512-haREypq7xkM7ErfgIyA0z+Bj4AGKlMSdlQE2jvJo6huWD1EdkKYV+G/T4nq0YEF2vgTT8kqMFKo1uHn950r4SQ=="],
"p-retry/retry": ["retry@0.13.1", "", {}, "sha512-XQBQ3I8W1Cge0Seh+6gjj03LbmRFWuoszgK9ooCpwYIrhhoO80pfq4cUkU5DkknwfOfFteRwlZ56PYOGYyFWdg=="],
"parse-entities/@types/unist": ["@types/unist@2.0.11", "", {}, "sha512-CmBKiL6NNo/OqgmMn95Fk9Whlp2mtvIv+KNpQKN2F4SjvrEesubTRWGYSg+BnWZOnlCaSTU1sMpsBOzgbYhnsA=="],
@@ -3238,6 +3306,8 @@
"prop-types/react-is": ["react-is@16.13.1", "", {}, "sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ=="],
"proper-lockfile/signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="],
"readdir-glob/minimatch": ["minimatch@5.1.9", "", { "dependencies": { "brace-expansion": "^2.0.1" } }, "sha512-7o1wEA2RyMP7Iu7GNba9vc0RWWGACJOCZBJX2GJWip0ikV+wcOsgVuY9uE8CPiyQhkGFSlhuSkZPavN7u1c2Fw=="],
"sharp/semver": ["semver@7.8.5", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA=="],
@@ -3250,8 +3320,6 @@
"wrap-ansi-cjs/ansi-styles": ["ansi-styles@4.3.0", "", { "dependencies": { "color-convert": "^2.0.1" } }, "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg=="],
"yauzl/buffer-crc32": ["buffer-crc32@0.2.13", "", {}, "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ=="],
"@aws-sdk/client-bedrock-runtime/@aws-sdk/credential-provider-node/@aws-sdk/credential-provider-env": ["@aws-sdk/credential-provider-env@3.972.80", "", { "dependencies": { "@aws-sdk/core": "^3.979.1", "@aws-sdk/types": "^3.975.0", "@smithy/core": "^3.38.0", "@smithy/types": "^4.19.0", "tslib": "^2.6.2" } }, "sha512-uO5TiSvb6j1w9RTe5z28OaOmsrOxVQUPaC5+wZl50RfOXcrYD/KgnS8I9lfyGXc7sGbkCopK6VnhNIAoLqfewQ=="],
"@aws-sdk/client-bedrock-runtime/@aws-sdk/credential-provider-node/@aws-sdk/credential-provider-http": ["@aws-sdk/credential-provider-http@3.972.82", "", { "dependencies": { "@aws-sdk/core": "^3.979.1", "@aws-sdk/types": "^3.975.0", "@smithy/core": "^3.38.0", "@smithy/fetch-http-handler": "^5.8.0", "@smithy/node-http-handler": "^4.14.0", "@smithy/types": "^4.19.0", "tslib": "^2.6.2" } }, "sha512-vQzlDHlC24Yz42p+rT4NBoP/5VdetkBoujbOLq9h6QYWC8THuZSPYnXIR9OP3XcOrdMgPBBipOrdssfxnIiWUg=="],
@@ -3270,6 +3338,8 @@
"@aws-sdk/client-bedrock-runtime/@aws-sdk/credential-provider-node/@smithy/types": ["@smithy/types@4.19.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-r7jh49VJxGerfAcTQA6gXcKc+98zOp/tqRwzYjgOE+iSQsP6cEU1hq2QzbuipmP68QtYdY9wKEhiCQZIzHgZ4Q=="],
"@earendil-works/pi-ai/@google/genai/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
"@isaacs/cliui/string-width/emoji-regex": ["emoji-regex@9.2.2", "", {}, "sha512-L18DaJsXSUk2+42pv8mLs5jJT2hqFkFE4j21wOmgbUqsZ2hL72NsUU785g9RXgo3s0ZNgVl42TiHp3ZtOv/Vyg=="],
"@isaacs/cliui/strip-ansi/ansi-regex": ["ansi-regex@6.3.0", "", {}, "sha512-WpDfL7NO6j7tH88IDBNVdUJxDh9nmCteAVW9dsep846XdwF4naCBK+/tGLX3KJgcpgMRXCFlTM2hKGoK9FsdrQ=="],
@@ -3372,6 +3442,8 @@
"eslint-plugin-react/minimatch/brace-expansion": ["brace-expansion@1.1.18", "", { "dependencies": { "balanced-match": "^1.0.0", "concat-map": "0.0.1" } }, "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw=="],
"extract-zip/yauzl/buffer-crc32": ["buffer-crc32@0.2.13", "", {}, "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ=="],
"gaxios/https-proxy-agent/agent-base": ["agent-base@7.1.4", "", {}, "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ=="],
"glob/minimatch/brace-expansion": ["brace-expansion@2.1.4", "", { "dependencies": { "balanced-match": "^1.0.0" } }, "sha512-hGfVzPxthbf3+2yjg/RBs60cB0FhqBS/zvdV/4wn4/BmN0bNMMHPc4V/BbFieqf1TKAGGAHnY4eSjajCl0f2Xg=="],
+2
View File
@@ -72,6 +72,8 @@ When running multiple replicas of the transcode workers or main servers across m
- **Custom Worker ID**: Set `SHUMAI_WORKERS_ID` (or `SHUMAI_WORKER_ID`) if you wish to assign an explicit identifier.
- **Kubernetes**: In Kubernetes Pods, set `POD_NAME` (via Downward API `fieldRef: { fieldPath: metadata.name }`) to provide stable, unique per-pod identity across pod restarts.
Agent runners require a single main server per database. With runners configured, stop the old main server before starting its replacement. For Kubernetes, use a single main server replica with the Recreate deployment strategy. Transcode workers can still scale independently.
---
## Scalable Deployment on Kubernetes & AWS EKS
+81 -25
View File
@@ -1,45 +1,101 @@
---
title: Agent Sandbox
description: Secure execution environment and network domain restrictions for AI agent scripts.
title: Agent Runners
description: Connect an optional command runner and configure sandboxing on its machine.
---
Running automated scripts poses security risks. Shumai resolves this by executing all agent skills inside a secure, isolated **Sandbox** environment.
Agents use Shumai's built-in tools without a runner. To execute shell commands or download assets into a shell workspace, connect a **runner** on a machine you control. The runner opens an authenticated outbound WebSocket connection; no inbound port is needed.
---
Agent runners require one main Shumai server per database. Multiple main servers can share a database while it has no configured runners. Stop the other main servers before creating a runner. Transcode workers can run separately.
## Sandbox Security
## Connect a runner
The sandbox restricts the execution process from accessing the host filesystem, system processes, and internal networks:
1. In **Settings → Agent Runners**, create a runner and copy its one-time key. Only team owners can manage runners and rotate their keys.
2. Install [Shumai CLI](https://github.com/shumaiOne/shumai/tree/main/apps/cli#installation) on macOS, Linux, or Windows.
3. Start it in a dedicated, existing workspace:
- **Isolation**: Scripts run in a virtualized workspace container that prevents them from modifying critical system files or reading other project data.
- **Network Bridging**: External network calls from inside the sandbox are strictly monitored.
```bash
export SHUMAI_API_SERVER="https://shumai.example.com"
export SHUMAI_RUNNER_KEY="your-runner-key"
shumai-cli runner start --cwd /path/to/workspace
```
### File System Restrictions
On Windows, set the variables in PowerShell:
To protect your workspace environments and sensitive data, the sandbox enforces strict file-level access controls:
```powershell
$env:SHUMAI_API_SERVER = "https://shumai.example.com"
$env:SHUMAI_RUNNER_KEY = "your-runner-key"
shumai-cli runner start --cwd C:\workspace
```
- **Read Access**: The sandbox is allowed to read files within the workspace, with a strict exception for sensitive environment and certificate files. It cannot read: `['.env', '.env.*', '*.pem', '*.key']`.
- **Write Access**: The sandbox is heavily restricted and may only write to the `.pi` folder located in the current working directory (cwd). Even within this allowed directory, the sandbox is explicitly blocked from writing or creating sensitive files matching: `['.env', '.env.*', '*.pem', '*.key']`.
Use a runner key, not a personal API token. Prefer the environment variable over `--key` to keep the key out of shell history. The runner stays in the foreground; Ctrl+C stops it and its active commands. Temporary connection losses trigger reconnection. Revoked or replaced connections stop instead of reconnecting.
---
The first runner becomes the team default. Each agent can use that default or a dedicated runner selected in its settings. Without a configured runner, `bash` and `download_asset` are hidden. With one configured, authorized editors and reviewers can run commands immediately without reading a skill first. Disabled-tool policies still apply. An offline runner produces an execution error.
## Domain Whitelisting
The `bash` tool uses `/bin/sh` on macOS and Linux. On Windows, the runner looks for Git Bash in the standard installation directories, then for `bash.exe` on PATH, and falls back to `cmd.exe`. The runner prints its selected shell at startup and reports it to the agent so it can use compatible syntax. For Bash with SRT on Windows, use Git Bash in place of the legacy WSL Bash launcher.
By default, Network Sandbox is **disabled** (allowing all outbound network requests). Team Owners can toggle Network Sandbox **ON** in Sandbox Settings to enforce strict domain whitelisting:
Commands start in the workspace directory. Skills, downloaded assets, generated files, and output logs live under `.pi`. Commands inherit baseline OS settings and explicit skill variables, rather than the runner's full environment, including its runner key and personal API token.
### Allowed Domains
Everyone with shell access to the same runner can read files left in its workspace, including assets and logs from other projects. Shumai project permissions apply when assets are downloaded. Share a runner only among users who may read all files in that workspace. For separate trust groups, use separate runner accounts or virtual machines and restrict access to the agents bound to each runner.
A list of domains that sandboxed scripts are permitted to access.
## Optional SRT sandbox
- Default whitelisted domains include standard package registries (like npm or python package hosts) and repository hosts (like `github.com`).
- If a script needs to call an external API (e.g. `api.slack.com`), the administrator must add it to the allowed list.
Without `--srt`, commands have the runner operating-system user's permissions. Connect only to a trusted Shumai server; use a dedicated container, VM, or account when appropriate.
### Pending Domains
[Anthropic Sandbox Runtime](https://github.com/anthropics/sandbox-runtime) is bundled as a library in Shumai CLI. Install its platform prerequisites on the runner machine (Linux: bubblewrap, socat, and ripgrep; macOS: ripgrep), then start with its JSON settings:
When an agent script attempts to reach a domain that is not currently authorized:
```bash
shumai-cli runner start --cwd /path/to/workspace --srt sandbox.json
```
1. The network connection is blocked immediately.
2. The target domain is logged and added to the **Pending Domains** list.
3. Admins can review this list and click **Approve** to authorize the domain, or **Reject** to keep it blocked.
Windows uses SRT's alpha Windows support. Run the one-time setup before starting a sandboxed runner; it requests administrator permission:
![Agent Sandbox Settings](../sandbox.webp)
```powershell
npx @anthropic-ai/sandbox-runtime windows-install
```
SRT verifies the setup at startup and reports an error if it is missing. A runner without `--srt` does not need this setup.
The JSON uses upstream SRT settings directly. For example:
```json
{
"filesystem": {
"denyRead": ["~/.ssh", "~/.aws", ".env", ".env.*"],
"allowWrite": [".pi"],
"denyWrite": []
},
"network": {
"allowedDomains": [],
"deniedDomains": []
}
}
```
This example allows writes only in `.pi`, denies reads of the listed credential locations, and blocks command network access. Review read permissions and network rules for your environment using SRT's documentation. Relative filesystem paths use the runner workspace. Settings are snapshotted at startup; restart after changing them. Missing or invalid settings, missing SRT prerequisites, and sandbox initialization failures stop startup. Commands never fall back to unrestricted execution.
SRT wraps shell commands and their child processes. One shared SRT runtime handles concurrent commands and is reset when the runner stops. The runner's WebSocket and server-directed file transfers operate outside SRT. **The Shumai server address is not automatically added to allowed domains.** If a script itself needs network access, explicitly allow its destination in the JSON.
Shumai has no server-side sandbox or domain settings. SRT is initialized only on runners that opt into it. SRT in Docker may require namespace permissions on that dedicated runner container; these permissions are not needed by the Shumai server.
## Runner in Docker
An ordinary runner can run in its own container without extra capabilities or an exposed port:
```bash
docker run --rm --init --name shumai-runner \
--cap-drop ALL --security-opt no-new-privileges \
-e SHUMAI_API_SERVER -e SHUMAI_RUNNER_KEY \
-v shumai-runner-workspace:/workspace -w /workspace \
node:22-bookworm sh -c \
'npm install -g @shumai-one/shumai-cli && exec shumai-cli runner start'
```
For development, HTTP is accepted for `localhost`, `127.0.0.1`, `[::1]`, and Docker's `host.docker.internal`. Remote servers require HTTPS. Avoid mounting sensitive host directories or the Docker socket into the runner workspace.
## Limits and lifecycle
Commands default to 120 seconds and accept a maximum timeout of one hour. Timeout and cancellation attempt to terminate the command and its process tree. Each command saves up to 64 MiB of output in `.pi/logs`; manage retained logs as part of workspace maintenance. Transfers and extracted archives are limited to 256 MiB. Protocol file operations reject paths outside `.pi`, existing symlinks, and unsafe archive entries. V1 assumes commands do not deliberately change workspace paths while transfers are running.
Temporary reconnects reconcile active and recently completed commands without deliberately executing them again. Restarting the runner process does not resume commands from the previous process.
Up to eight commands and file transfers can run concurrently in the shared workspace. Additional requests return a busy error. On Windows without SRT, background processes may survive after the command's shell exits; remaining output pipes are closed after one second. Use foreground commands for predictable timeout and cancellation. SRT manages the sandboxed Windows process tree.
+2
View File
@@ -53,6 +53,8 @@ Worker 根据可配置的并发上限从 PostgreSQL 队列中拉取任务:
- **自定义 Worker ID**:若需显式指定标识符,可设置 `SHUMAI_WORKERS_ID`(或 `SHUMAI_WORKER_ID`)。
- **Kubernetes 环境**:在 Kubernetes Pod 中,可通过 Downward API(`fieldRef: { fieldPath: metadata.name }`)设置 `POD_NAME`,以在 Pod 重启期间保持稳定且唯一的单 Pod 身份标识。
使用智能体运行器时,每个数据库只允许运行一个主服务器。已配置运行器时,请先停止旧主服务器,再启动替换实例。在 Kubernetes 中,主服务器应使用单副本和 Recreate 部署策略。转码 Worker 仍可独立扩展。
---
## 基于 Kubernetes 与 AWS EKS 的弹性扩展部署
+79 -25
View File
@@ -1,45 +1,99 @@
---
title: 智能体沙箱
description: AI 智能体脚本的安全执行环境与网络域名限制规范。
title: 智能体运行器
description: 连接可选的命令运行器,并在运行器机器上配置沙箱。
---
运行自动化脚本存在潜在的安全风险。Shumai 通过在安全且隔离的**沙箱(Sandbox)**环境中执行所有智能体技能来化解这一隐患。
智能体无需运行器即可使用内置工具。如需执行命令或将素材下载到命令工作区,请在自己管理的机器上连接**运行器**。运行器主动通过经过身份验证的 WebSocket 连接 Shumai,无需开放入站端口。
---
每个数据库只支持一台主 Shumai 服务器使用智能体运行器。未配置运行器时,多台主服务器可以共享数据库。创建运行器前,请停止其他主服务器副本。转码 Worker 可以独立运行。
## 沙箱安全机制
## 连接运行器
沙箱严格限制执行进程访问宿主机文件系统、系统进程及内部网络:
1. 在 **设置 → 智能体运行器** 中创建运行器,复制仅显示一次的密钥。只有团队所有者可以管理运行器和轮换密钥。
2. 在 macOS、Linux 或 Windows 机器上安装 [Shumai CLI](https://github.com/shumaiOne/shumai/tree/main/apps/cli#installation)。
3. 在已有的专用工作目录中启动:
- **环境隔离**:脚本在虚拟化的工作区容器中运行,无法篡改系统核心文件或读取其他项目数据。
- **网络桥接管控**:来自沙箱内部的外部网络请求会受到严格监控。
```bash
export SHUMAI_API_SERVER="https://shumai.example.com"
export SHUMAI_RUNNER_KEY="your-runner-key"
shumai-cli runner start --cwd /path/to/workspace
```
### 文件系统访问限制
Windows 用户可在 PowerShell 中设置变量:
为保障工作区环境与敏感数据安全,沙箱实施了严密的文件级访问控制策略:
```powershell
$env:SHUMAI_API_SERVER = "https://shumai.example.com"
$env:SHUMAI_RUNNER_KEY = "your-runner-key"
shumai-cli runner start --cwd C:\workspace
```
- **读取权限**:沙箱允许读取工作区内的普通文件,但严格禁止访问敏感环境配置与证书密钥文件。禁止读取的文件模式包括:`['.env', '.env.*', '*.pem', '*.key']`。
- **写入权限**:沙箱的写入权限受到极为严苛的限制,仅允许写入当前工作目录(cwd)下的 `.pi` 文件夹。即便在该受信任的目录内部,沙箱也严禁写入或创建匹配以下模式的敏感文件:`['.env', '.env.*', '*.pem', '*.key']`。
使用运行器密钥,而非个人 API 令牌。建议通过环境变量传入,避免 `--key` 将密钥留在命令历史中。运行器在前台执行,Ctrl+C 会停止运行器及其活动命令。临时断线后自动重连;密钥撤销或连接被替换时退出。
---
第一个运行器是团队默认运行器,也可在智能体设置中指定专用运行器。未配置运行器时隐藏 `bash` 和 `download_asset`。配置后,具有智能体使用权限的编辑者和审阅者可立即使用 Bash,无需先读取技能;禁用工具设置仍然生效。运行器离线时执行命令会返回错误。
## 域名白名单
命令从工作目录启动,技能、下载素材、生成文件和日志位于 `.pi` 中。子进程只继承基础系统环境及明确配置的技能变量,不自动继承运行器密钥、个人 API 令牌或其他任意环境变量。
默认情况下,网络沙箱处于**关闭**状态(允许所有出站网络请求)。团队所有者可在沙箱设置中开启网络沙箱,以强制推行严格的域名白名单管控:
所有有权在同一运行器执行命令的用户都可以读取工作区中保留的文件,包括其他项目的素材和日志。Shumai 下载素材时会检查项目权限。请仅在有权读取工作区内所有文件的用户之间共享运行器。不同信任范围应使用独立的运行器账号或虚拟机,并限制对应智能体的访问权限。
### 允许的域名 (Allowed Domains)
## 可选 SRT 沙箱
允许沙箱内脚本正常访问的域名清单。
未使用 `--srt` 时,命令具有运行器系统用户的权限。只连接可信 Shumai 服务器,可按需使用独立容器、虚拟机或账号。
- 默认白名单包含标准软件包注册表(如 npm、Python 官方源等)以及代码托管平台(如 `github.com`)。
- 若脚本需要调用外部 API(如 `api.slack.com`),管理员须预先将其添加至白名单中。
[Anthropic Sandbox Runtime](https://github.com/anthropics/sandbox-runtime) 已作为库内置于 Shumai CLI。安装平台依赖(Linux:bubblewrap、socat 和 ripgrep;macOS:ripgrep)后传入 JSON 配置:
### 待审批域名 (Pending Domains)
```bash
shumai-cli runner start --cwd /path/to/workspace --srt sandbox.json
```
当智能体脚本尝试访问未经授权的域名时:
Windows 使用 SRT 的实验性 Windows 支持。启用沙箱前执行一次初始化;此命令会请求管理员权限:
1. 网络连接将被即刻拦截。
2. 系统会记录目标域名并将其加入**待审批域名**列表。
3. 管理员可审阅该列表,点击**批准**将其加白,或点击**拒绝**保持拦截。
```powershell
npx @anthropic-ai/sandbox-runtime windows-install
```
![智能体沙箱设置](/sandbox.webp)
SRT 会在启动时检查安装状态,缺少初始化时返回错误。未使用 `--srt` 的运行器不需要此步骤。`bash` 工具在 macOS 和 Linux 上使用 `/bin/sh`。在 Windows 上,运行器先在标准安装目录查找 Git Bash,再在 PATH 中查找 `bash.exe`,找不到时使用 `cmd.exe`。运行器在启动时显示所选 Shell,并将其报告给智能体,让智能体使用匹配的命令语法。在 Windows 上搭配 SRT 使用 Bash 时,请用 Git Bash 代替旧版 WSL Bash 启动器。
直接使用上游 SRT JSON 格式,例如:
```json
{
"filesystem": {
"denyRead": ["~/.ssh", "~/.aws", ".env", ".env.*"],
"allowWrite": [".pi"],
"denyWrite": []
},
"network": {
"allowedDomains": [],
"deniedDomains": []
}
}
```
此例只允许写入 `.pi`,禁止读取所列凭证位置,并禁止命令访问网络。请参考 SRT 文档审查适合自己环境的读取权限和域名规则。相对路径以运行器工作目录为基准。配置在启动时读取并保存快照,修改后需要重启。配置缺失或无效、SRT 依赖缺失、沙箱初始化失败都会阻止启动,不会回退到无沙箱执行。
SRT 只包裹 Shell 命令及其子进程。并发命令共享同一个 SRT 运行时,运行器停止时统一清理。运行器 WebSocket 和服务器发起的文件传输在 SRT 外运行。**不会自动将 Shumai 服务器地址添加到允许域名中。** 如果脚本自身需要网络,请在 JSON 中明确允许目标域名。
Shumai 服务器不管理沙箱和域名规则。只有启用沙箱的运行器才初始化 SRT。在 Docker 中使用 SRT 可能需要为专用运行器容器增加命名空间权限,Shumai 服务器不需要这些权限。
## Docker 运行器
普通运行器可在独立容器中执行,无需额外权限或暴露端口:
```bash
docker run --rm --init --name shumai-runner \
--cap-drop ALL --security-opt no-new-privileges \
-e SHUMAI_API_SERVER -e SHUMAI_RUNNER_KEY \
-v shumai-runner-workspace:/workspace -w /workspace \
node:22-bookworm sh -c \
'npm install -g @shumai-one/shumai-cli && exec shumai-cli runner start'
```
本地开发允许 `localhost`、`127.0.0.1`、`[::1]`、Docker 的 `host.docker.internal` HTTP 地址;远程连接必须使用 HTTPS。不要将敏感宿主机目录或 Docker socket 挂载到运行器工作区。
## 限制与生命周期
默认超时 120 秒,最长一小时。超时或取消会尝试终止命令及其子进程。单条命令最多在 `.pi/logs` 保存 64 MiB 输出,请按工作区维护需求清理日志。文件传输和技能解压限制为 256 MiB。协议文件操作拒绝 `.pi` 外路径、已有符号链接及不安全的压缩包条目。V1 假定命令不会在文件传输期间故意修改工作区路径。
临时重连会同步执行中及近期完成的命令,避免主动重复执行。重启运行器进程不会恢复旧进程中的命令。
共享工作区最多同时执行八个命令或文件传输,超出时返回繁忙错误。Windows 未启用 SRT 时,后台进程可能在命令 Shell 退出后继续运行,剩余输出管道会在一秒后关闭。请使用前台命令,以确保超时和取消可预测。启用 SRT 时由 SRT 管理 Windows 沙箱进程树。
@@ -5,7 +5,7 @@ import { setupTestDbHooks } from '@shumai/db/test'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { createAgentSession, type DbProviderInfo } from './index'
import { closeAgentSessions } from './runtime'
import { createSandboxedBashTool } from './tools/sandboxed-bash'
import { createRunnerBashTool } from './tools/runner-bash'
const mockProviders: DbProviderInfo[] = [
{
@@ -43,6 +43,19 @@ function createMockWs() {
}
}
function reconcile(runnerId: string, ws: RunnerSocket) {
runnerService.handleMessage(
runnerId,
ws,
JSON.stringify({
type: 'reconcile',
environment: { platform: 'linux', shell: '/bin/sh' },
activeExecutions: [],
completedExecutions: [],
}),
)
}
describe('Concurrent Runner Sessions', () => {
setupTestDbHooks()
@@ -64,6 +77,7 @@ describe('Concurrent Runner Sessions', () => {
{ id: runner.id, teamId: team.id, name: runner.name, keyHash: hashRunnerKey(key) },
conn.ws,
)
reconcile(runner.id, conn.ws)
await prisma.user.create({
data: {
@@ -92,19 +106,17 @@ describe('Concurrent Runner Sessions', () => {
providers: mockProviders,
})
const tool = createSandboxedBashTool('/mock/cwd', {}, { teamId: team.id })
const tool = createRunnerBashTool({}, { teamId: team.id })
// Note: registerConnection sends a 'welcome' message first (conn.sent[0])
const execA = invokeTool(tool, 'call-a', {
command: 'echo session-a',
source: 'skill',
})
await vi.waitFor(() => expect(conn.sent.length).toBe(2))
const msgA = JSON.parse(conn.sent[1])
const execB = invokeTool(tool, 'call-b', {
command: 'echo session-b',
source: 'skill',
})
await vi.waitFor(() => expect(conn.sent.length).toBe(3))
const msgB = JSON.parse(conn.sent[2])
@@ -160,10 +172,12 @@ describe('Concurrent Runner Sessions', () => {
)
const [resA, resB] = await Promise.all([execA, execB])
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- ToolResponse content blocks
expect((resA as any).content[0].text).toContain('out-a')
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- ToolResponse content blocks
expect((resB as any).content[0].text).toContain('out-b')
expect(resA.content).toContainEqual(
expect.objectContaining({ text: expect.stringContaining('out-a') }),
)
expect(resB.content).toContainEqual(
expect.objectContaining({ text: expect.stringContaining('out-b') }),
)
runnerService.handleDisconnect(runner.id, conn.ws)
})
@@ -201,17 +215,17 @@ describe('Concurrent Runner Sessions', () => {
},
gpuConn.ws,
)
reconcile(defaultRunner.id, defaultConn.ws)
reconcile(gpuRunner.id, gpuConn.ws)
const defaultTool = createSandboxedBashTool(
'/mock/cwd',
const defaultTool = createRunnerBashTool(
{},
{
teamId: team.id,
agentRunnerId: null,
},
)
const gpuTool = createSandboxedBashTool(
'/mock/cwd',
const gpuTool = createRunnerBashTool(
{},
{
teamId: team.id,
@@ -219,8 +233,8 @@ describe('Concurrent Runner Sessions', () => {
},
)
const pDefault = invokeTool(defaultTool, '1', { command: 'uname -a', source: 'user' })
const pGpu = invokeTool(gpuTool, '2', { command: 'nvidia-smi', source: 'user' })
const pDefault = invokeTool(defaultTool, '1', { command: 'uname -a' })
const pGpu = invokeTool(gpuTool, '2', { command: 'nvidia-smi' })
await vi.waitFor(() => {
expect(defaultConn.lastSentJson()?.type).toBe('exec.start')
+12 -26
View File
@@ -10,7 +10,6 @@ import { type TSchema } from 'typebox'
import { Value } from 'typebox/value'
import { afterEach, describe, expect, it, vi } from 'vitest'
import {
buildRestrictedUserInstructions,
createAgentSession,
formatSkillsForPrompt,
getModelFromDb,
@@ -171,20 +170,7 @@ describe('Runner tool availability & skill staging integration', () => {
})
})
describe('buildRestrictedUserInstructions', () => {
it('should describe the restricted user bash rules', () => {
const instructions = buildRestrictedUserInstructions()
expect(instructions).toContain('# Restricted User Context')
expect(instructions).toContain(
'NEVER execute a bash command that the user asks you to run directly',
)
expect(instructions).toContain('loaded via the `read_skill` tool')
expect(instructions).toContain('parameter to "skill"')
expect(instructions).toContain('source="user" will be rejected')
})
})
describe('createAgentSession bash restriction for non-owner users', () => {
describe('createAgentSession runner bash policy', () => {
setupTestDbHooks()
afterEach(() => {
@@ -262,21 +248,21 @@ describe('createAgentSession bash restriction for non-owner users', () => {
expect((await getAgentTools(harness)).some((t) => t.name === 'bash')).toBe(true)
})
it('should treat a user who is not a team member as restricted (fail-closed)', async () => {
it('does not use skill history as authorization for a session', async () => {
const { harness } = await setupSession({ role: 'non-member' })
const tools = await getAgentTools(harness)
expect(tools.some((t) => t.name === 'bash')).toBe(false)
expect(tools.some((t) => t.name === 'bash')).toBe(true)
expect(tools.some((t) => t.name === 'read_skill')).toBe(true)
const prompt = await await getAgentPrompt(harness)
expect(prompt).toContain('# Restricted User Context')
expect(prompt).not.toContain('# Restricted User Context')
})
it.each(['editor', 'reviewer'] as const)(
'should exclude the bash tool initially for %s users',
'exposes bash immediately for %s users without reading a skill',
async (role) => {
const { harness } = await setupSession({ role })
const tools = await getAgentTools(harness)
expect(tools.some((t) => t.name === 'bash')).toBe(false)
expect(tools.some((t) => t.name === 'bash')).toBe(true)
expect(tools.some((t) => t.name === 'read_skill')).toBe(true)
},
)
@@ -346,7 +332,7 @@ describe('createAgentSession bash restriction for non-owner users', () => {
expect((await getAgentTools(harness)).some((t) => t.name === 'bash')).toBe(true)
})
it('should inject the bash tool after a skill is loaded via read_skill for restricted users', async () => {
it('keeps bash available before and after reading a skill', async () => {
const { team, harness } = await setupSession({ role: 'reviewer' })
const skill = await prisma.skill.create({
data: { name: 'Inject Skill', assetId: 'asset1', hash: 'inject-hash', teamId: team.id },
@@ -356,14 +342,14 @@ describe('createAgentSession bash restriction for non-owner users', () => {
const readSkillTool = (await getAgentTools(harness)).find((t) => t.name === 'read_skill')
expect(readSkillTool).toBeDefined()
expect((await getAgentTools(harness)).some((t) => t.name === 'bash')).toBe(false)
expect((await getAgentTools(harness)).some((t) => t.name === 'bash')).toBe(true)
await invokeTool(readSkillTool!, '1', { skillId: skill.id }, undefined, undefined)
expect((await getAgentTools(harness)).some((t) => t.name === 'bash')).toBe(true)
})
it('should not inject the bash tool for restricted users when bash is in deniedTools', async () => {
it('keeps bash disabled by agent policy after reading a skill', async () => {
const { team, harness } = await setupSession({ role: 'editor', deniedTools: ['bash'] })
const skill = await prisma.skill.create({
data: { name: 'Denied Skill', assetId: 'asset1', hash: 'denied-hash', teamId: team.id },
@@ -379,11 +365,11 @@ describe('createAgentSession bash restriction for non-owner users', () => {
expect((await getAgentTools(harness)).some((t) => t.name === 'bash')).toBe(false)
})
it('should include restricted user instructions in the system prompt for non-owners', async () => {
it('omits skill-first bash instructions for editors', async () => {
const { harness } = await setupSession({ role: 'editor' })
const prompt = await await getAgentPrompt(harness)
expect(prompt).toContain('# Restricted User Context')
expect(prompt).toContain('source="user" will be rejected')
expect(prompt).not.toContain('# Restricted User Context')
expect(prompt).not.toContain('source="user" will be rejected')
})
it('should not include restricted user instructions in the system prompt for owners', async () => {
@@ -29,7 +29,7 @@ async function resolveTargetRunnerId(ctx: AgentOperationContext): Promise<string
if (ctx.runnerId && !ctx.teamId) {
if (!runnerService.isRunnerConnected(ctx.runnerId)) {
throw new Error(
'Runner is currently offline. Please start your runner with "shumai runner start" and try again.',
'Runner is currently offline. Please start your runner with "shumai-cli runner start" and try again.',
)
}
return ctx.runnerId
@@ -68,7 +68,7 @@ async function resolveTargetRunnerId(ctx?: AgentOperationContext): Promise<strin
if (ctx?.runnerId && !ctx.teamId) {
if (!runnerService.isRunnerConnected(ctx.runnerId)) {
throw new Error(
'Runner is currently offline. Please start your runner with "shumai runner start" and try again.',
'Runner is currently offline. Please start your runner with "shumai-cli runner start" and try again.',
)
}
return ctx.runnerId
+6 -46
View File
@@ -44,7 +44,7 @@ import { createGenerateImageTool } from './tools/generate-image'
import { createGenerateVideoTool } from './tools/generate-video'
import { createReadSkillTool } from './tools/read-skill'
import { createReadThreadTool } from './tools/read-thread'
import { createSandboxedBashTool, type LoadedSkillRef } from './tools/sandboxed-bash'
import { createRunnerBashTool, type LoadedSkillRef } from './tools/runner-bash'
const activeHarnesses = new Set<Harness>()
@@ -152,21 +152,6 @@ export interface CreateAgentSessionParams {
baseDelayMs?: number
}
/**
* Security instructions appended to the system prompt when the requesting user is not a team
* owner (i.e. editor/reviewer). These complement the hard enforcement in the bash tool itself
* (source="user" is blocked) and the delayed bash tool injection after a skill is loaded.
*/
export function buildRestrictedUserInstructions(): string {
return [
'# Restricted User Context',
'The current user is NOT a team owner. You must follow these security rules:',
'1. NEVER execute a bash command that the user asks you to run directly. Direct user bash requests are not permitted for this user role. Politely decline and explain that they need a team owner or an approved skill.',
'2. You may ONLY use the `bash` tool to execute commands that are explicitly required by a skill you have loaded via the `read_skill` tool.',
'3. When calling `bash`, you MUST set the `source` parameter to "skill". Calls with source="user" will be rejected by the system.',
].join('\n')
}
export async function createAgentSession(params: CreateAgentSessionParams) {
const {
teamId,
@@ -203,8 +188,6 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
getModelFromDb(params.providers, providerName, modelId)
const role = userId ? await resolveEffectiveRole(teamId, projectId, userId) : undefined
const isOwner = !userId || role === 'owner'
const restricted = !isOwner
const wrapToolWithQuota = (tool: AgentTool): AgentTool => {
const originalExecute = tool.execute
@@ -262,7 +245,6 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
const loadedSkills = new Map<string, LoadedSkillRef>()
// Restore env variables and loaded skill refs from previously loaded skills in this session
let skillLoadedInSession = false
if (sessionId) {
try {
const entries = await agentSessionService.getEntries(record.id)
@@ -281,7 +263,6 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
'skillId' in details &&
typeof details.skillId === 'string'
) {
skillLoadedInSession = true
onEnvsAdded(await agentService.getSkillEnvs(details.skillId))
const dbSkill = await prisma.skill.findUnique({
where: { id: details.skillId },
@@ -335,23 +316,18 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
}
}
const sandboxedBash = createSandboxedBashTool(process.cwd(), skillEnvs, {
const runnerBash = createRunnerBashTool(skillEnvs, {
teamId,
agentRunnerId: agent?.runnerId,
environment: resolvedRunner.runner
? runnerService.getRunnerEnvironment(resolvedRunner.runner.id)
: undefined,
userId,
role,
// Non-owner users can only run bash commands required by a loaded skill
restrictedUser: restricted,
getLoadedSkills: () => loadedSkills.values(),
})
// When a team has no Runner configured in DB, bash and download_asset are omitted.
// For non-owner users, the bash tool is not injected initially. It becomes available
// only after a skill is successfully loaded via the read_skill tool (or immediately when
// a skill was already loaded earlier in this session).
const registry = createRegistry()
const bashIncludedFromStart = hasRunner && (isOwner || skillLoadedInSession)
let bashInjected = bashIncludedFromStart
const onSkillLoaded = async (skill: LoadedSkillRef) => {
loadedSkills.set(skill.id, skill)
if (resolvedRunner.runner && runnerService.isRunnerConnected(resolvedRunner.runner.id)) {
@@ -362,18 +338,6 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
)
})
}
if (!restricted || !hasRunner || bashInjected || deniedTools.includes('bash')) return
bashInjected = true
registry.install(
defineExtension({ name: 'shumai.bash', tools: [wrapToolWithQuota(sandboxedBash)] }),
)
await session?.configure(
{
tools: [...enabledTools, wrapToolWithQuota(sandboxedBash)],
modelTools: { add: [sandboxedBash] },
},
piContext,
)
}
const readSkill = createReadSkillTool(
userId,
@@ -404,7 +368,7 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
...mediaTools,
readSkill,
readThread,
...(bashIncludedFromStart ? [sandboxedBash] : []),
...(hasRunner ? [runnerBash] : []),
...systemTools,
...customTools,
]
@@ -472,10 +436,6 @@ export async function createAgentSession(params: CreateAgentSessionParams) {
return prompt
}
if (restricted) {
prompt += `\n\n${buildRestrictedUserInstructions()}`
}
if (hasRunner && !deniedTools.includes('bash')) {
prompt +=
'\n\n' +
+1 -1
View File
@@ -67,7 +67,7 @@ export const createReadSkillTool = (
// Retrieve skill content using agentService
const skillMdContent = await agentService.getSkillContent(params.skillId)
// Notify that a skill was successfully loaded (e.g. to enable restricted tools and stage skill on runner)
// Notify that a skill was successfully loaded (e.g. to stage the skill on the runner)
await onSkillLoaded?.({ id: skill.id, assetId: skill.assetId, hash: skill.hash })
return {
@@ -1,8 +1,7 @@
import { invokeTool } from '@shumai/core/src/agent/tool-test-utils'
import { runnerService } from '@shumai/core/src/runner'
import { EventEmitter } from 'node:events'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { createSandboxedBashTool } from './sandboxed-bash'
import { createRunnerBashTool } from './runner-bash'
vi.mock('@shumai/core/src/runner', () => ({
runnerService: {
@@ -14,8 +13,7 @@ vi.mock('@shumai/core/src/runner', () => ({
},
}))
describe('createSandboxedBashTool', () => {
const mockCwd = '/mock/cwd'
describe('createRunnerBashTool', () => {
const mockTeamId = 'team-1'
const mockRunner = {
id: 'runner-1',
@@ -38,24 +36,22 @@ describe('createSandboxedBashTool', () => {
)
})
const createMockSignal = () => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- signal needs to be partially compatible with AbortSignal
const signal = new EventEmitter() as any
signal.addEventListener = vi.fn((event: string, cb: (ev: Event) => unknown) => {
signal.on(event, cb)
})
signal.removeEventListener = vi.fn((event: string, cb: (ev: Event) => unknown) => {
signal.off(event, cb)
})
signal.aborted = false
return signal
}
const createMockSignal = () => new AbortController().signal
const createMockOnUpdate = () => vi.fn()
it.each(['C:\\Windows\\System32\\cmd.exe', 'C:\\Program Files\\Git\\bin\\bash.exe'])(
'tells the agent to use the runner OS and shell syntax for %s',
(shell) => {
const tool = createRunnerBashTool({}, { environment: { platform: 'win32', shell } })
expect(tool.description).toContain('Runner OS: win32')
expect(tool.description).toContain(shell)
expect(tool.description).toContain('Use syntax and programs compatible with this environment')
},
)
it('should execute a command on the runner with injected env vars', async () => {
const tool = createSandboxedBashTool(
mockCwd,
const tool = createRunnerBashTool(
{ TEST_ENV: 'test-value' },
{
teamId: mockTeamId,
@@ -78,13 +74,7 @@ describe('createSandboxedBashTool', () => {
const onUpdate = createMockOnUpdate()
const signal = createMockSignal()
const result = await invokeTool(
tool,
'1',
{ command: 'echo hello', source: 'user' },
signal,
onUpdate,
)
const result = await invokeTool(tool, '1', { command: 'echo hello' }, signal, onUpdate)
expect(runnerService.requireOnlineRunner).toHaveBeenCalledWith(mockTeamId, null)
expect(runnerService.exec).toHaveBeenCalledWith(
@@ -95,15 +85,15 @@ describe('createSandboxedBashTool', () => {
collectOutput: false,
}),
)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- ToolResponse content is an array of content blocks
expect((result as any).content[0].text).toContain('hello output')
expect(result.content).toContainEqual(
expect.objectContaining({ text: expect.stringContaining('hello output') }),
)
expect(onUpdate).toHaveBeenCalled()
})
it('should stage loaded skills and inputFiles before executing', async () => {
const loadedSkills = [{ id: 'skill-1', assetId: 'asset-1', hash: 'hash-1' }]
const tool = createSandboxedBashTool(
mockCwd,
const tool = createRunnerBashTool(
{},
{
teamId: mockTeamId,
@@ -126,7 +116,6 @@ describe('createSandboxedBashTool', () => {
'1',
{
command: 'ffmpeg -i .pi/generated/gen1/input.png out.mp4',
source: 'user',
inputFiles: ['.pi/generated/gen1/input.png'],
},
createMockSignal(),
@@ -140,31 +129,12 @@ describe('createSandboxedBashTool', () => {
)
})
it('should block user-requested commands when restrictedUser is enabled', async () => {
const tool = createSandboxedBashTool(
mockCwd,
it('allows editor commands without a source classification', async () => {
const tool = createRunnerBashTool(
{},
{
teamId: mockTeamId,
restrictedUser: true,
},
)
const signal = createMockSignal()
await expect(
invokeTool(tool, '1', { command: 'echo hack', source: 'user' }, signal, createMockOnUpdate()),
).rejects.toThrow(/not permitted for your role/)
expect(runnerService.exec).not.toHaveBeenCalled()
})
it('should allow skill-requested commands when restrictedUser is enabled', async () => {
const tool = createSandboxedBashTool(
mockCwd,
{},
{
teamId: mockTeamId,
restrictedUser: true,
role: 'editor',
},
)
@@ -181,24 +151,25 @@ describe('createSandboxedBashTool', () => {
})
const signal = createMockSignal()
const result = await invokeTool(tool, '1', { command: 'echo skill', source: 'skill' }, signal)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- ToolResponse content is an array of content blocks
expect((result as any).content[0].text).toContain('skill output')
const result = await invokeTool(tool, '1', { command: 'echo skill' }, signal)
expect(result.content).toContainEqual(
expect.objectContaining({ text: expect.stringContaining('skill output') }),
)
})
it('should handle runner execution errors', async () => {
const tool = createSandboxedBashTool(mockCwd, {}, { teamId: mockTeamId })
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
vi.mocked(runnerService.exec).mockRejectedValue(new Error('Runner disconnected'))
const signal = createMockSignal()
await expect(
invokeTool(tool, '1', { command: 'fail', source: 'user' }, signal, createMockOnUpdate()),
invokeTool(tool, '1', { command: 'fail' }, signal, createMockOnUpdate()),
).rejects.toThrow('Runner disconnected')
})
it('should handle non-zero exit codes', async () => {
const tool = createSandboxedBashTool(mockCwd, {}, { teamId: mockTeamId })
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
vi.mocked(runnerService.exec).mockResolvedValue({
executionId: 'exec-1',
@@ -211,12 +182,27 @@ describe('createSandboxedBashTool', () => {
const signal = createMockSignal()
await expect(
invokeTool(tool, '1', { command: 'exit 1', source: 'user' }, signal, createMockOnUpdate()),
invokeTool(tool, '1', { command: 'exit 1' }, signal, createMockOnUpdate()),
).rejects.toThrow(/exited with code 1/)
})
it('reports signal termination as a command failure', async () => {
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
vi.mocked(runnerService.exec).mockResolvedValue({
executionId: 'signaled',
output: '',
exitCode: null,
signal: 'SIGTERM',
timedOut: false,
aborted: false,
})
await expect(invokeTool(tool, '1', { command: 'command' }, createMockSignal())).rejects.toThrow(
'terminated by SIGTERM',
)
})
it('should handle timeouts', async () => {
const tool = createSandboxedBashTool(mockCwd, {}, { teamId: mockTeamId })
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
vi.mocked(runnerService.exec).mockImplementation(async (_runnerId, opts) => {
opts.onOutput?.('output before timeout\n', 'stdout')
@@ -234,7 +220,7 @@ describe('createSandboxedBashTool', () => {
const executePromise = invokeTool(
tool,
'1',
{ command: 'sleep 10', timeout: 2, source: 'user' },
{ command: 'sleep 10', timeout: 2 },
signal,
createMockOnUpdate(),
)
@@ -244,7 +230,7 @@ describe('createSandboxedBashTool', () => {
})
it('should handle abort signals', async () => {
const tool = createSandboxedBashTool(mockCwd, {}, { teamId: mockTeamId })
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
vi.mocked(runnerService.exec).mockResolvedValue({
executionId: 'exec-1',
@@ -257,12 +243,12 @@ describe('createSandboxedBashTool', () => {
const signal = createMockSignal()
await expect(
invokeTool(tool, '1', { command: 'long-run', source: 'user' }, signal, createMockOnUpdate()),
invokeTool(tool, '1', { command: 'long-run' }, signal, createMockOnUpdate()),
).rejects.toThrow(/aborted/)
})
it('should truncate output exceeding max lines (2000 lines) and report runner log file path', async () => {
const tool = createSandboxedBashTool(mockCwd, {}, { teamId: mockTeamId })
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
vi.mocked(runnerService.exec).mockImplementation(async (_runnerId, opts) => {
const lines: string[] = []
@@ -283,14 +269,8 @@ describe('createSandboxedBashTool', () => {
})
const signal = createMockSignal()
const result = await invokeTool(
tool,
'1',
{ command: 'generate lines', source: 'user' },
signal,
)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- content block array
const text = (result as any).content[0].text
const result = await invokeTool(tool, '1', { command: 'generate lines' }, signal)
const text = result.content.map((block) => (block.type === 'text' ? block.text : '')).join('')
expect(text).toContain('line-3000')
expect(text).toContain(
'[Showing lines 1001-3000 of 3000. Full output: .pi/logs/bash-exec1.log]',
@@ -305,7 +285,7 @@ describe('createSandboxedBashTool', () => {
})
it('should truncate output exceeding max bytes (50KB) and report partial line size', async () => {
const tool = createSandboxedBashTool(mockCwd, {}, { teamId: mockTeamId })
const tool = createRunnerBashTool({}, { teamId: mockTeamId })
vi.mocked(runnerService.exec).mockImplementation(async (_runnerId, opts) => {
const longLine = 'a'.repeat(60000)
@@ -322,14 +302,8 @@ describe('createSandboxedBashTool', () => {
})
const signal = createMockSignal()
const result = await invokeTool(
tool,
'1',
{ command: 'long single line', source: 'user' },
signal,
)
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- content block array
const text = (result as any).content[0].text
const result = await invokeTool(tool, '1', { command: 'long single line' }, signal)
const text = result.content.map((block) => (block.type === 'text' ? block.text : '')).join('')
expect(text).toMatch(
/Showing last 50\.0KB of line 1 \(line is 58\.6KB\)\. Full output: \.pi\/logs\/bash-exec2\.log/,
)
@@ -6,6 +6,7 @@ import { logger } from '@shumai/core/src/logger'
import { quotaService } from '@shumai/core/src/quota/quota-service'
import { runnerService } from '@shumai/core/src/runner'
import type { TeamMemberRole } from '@shumai/db'
import type { RunnerEnvironment } from '@shumai/dtos'
import { Type } from 'typebox'
import { OutputAccumulator } from '../utils/output-accumulator'
import {
@@ -15,19 +16,9 @@ import {
type TruncationResult,
} from '../utils/truncate'
const BashSource = Type.String({
enum: ['user', 'skill'],
description:
'Indicates who requested this bash command. Set to "user" when the end user directly ' +
'requested the command. Set to "skill" when the command is required by a skill you loaded ' +
'via the read_skill tool. Users without the owner role can only execute commands with ' +
'source="skill"; source="user" will be blocked.',
})
const BashParameters = Type.Object({
command: Type.String({ description: 'The bash command to execute.' }),
command: Type.String({ description: "The command to execute using the runner's shell." }),
timeout: Type.Optional(Type.Number({ description: 'Maximum execution time in seconds.' })),
source: BashSource,
inputFiles: Type.Optional(
Type.Union([
Type.Array(Type.String(), {
@@ -51,42 +42,31 @@ export interface LoadedSkillRef {
hash: string
}
export interface SandboxedBashOptions {
export interface RunnerBashOptions {
teamId?: string
agentRunnerId?: string | null
runnerId?: string
userId?: string
role?: TeamMemberRole | null
/**
* When true, the tool only allows commands requested by a loaded skill (source="skill").
* Commands requested directly by the end user (source="user") are blocked.
*/
restrictedUser?: boolean
environment?: RunnerEnvironment
getLoadedSkills?: () => Iterable<LoadedSkillRef>
}
export const createSandboxedBashTool = (
_cwd: string,
export const createRunnerBashTool = (
skillEnvs: Record<string, string> = {},
options?: SandboxedBashOptions,
options?: RunnerBashOptions,
): AgentTool<typeof BashParameters, BashDetails> => {
return {
name: 'bash',
description: `Execute a bash command on the configured runner environment. Returns stdout and stderr. Output is truncated to last ${DEFAULT_MAX_LINES} lines or ${DEFAULT_MAX_BYTES / 1024}KB (whichever is hit first). If truncated, full output is saved to a log file in .pi/logs/ on the runner. Optionally provide a timeout in seconds, or inputFiles to stage generated files (.pi/generated/...) into the runner workspace before running. When in a restricted (non-owner) user context, only commands required by a skill loaded via the read_skill tool are allowed, and you must set source="skill".`,
description: `Execute a shell command on the configured runner. ${options?.environment ? `Runner OS: ${options.environment.platform}. Shell: ${options.environment.shell}. Use syntax and programs compatible with this environment.` : 'The runner reports its OS and shell when connected.'} Returns stdout and stderr. Output is truncated to last ${DEFAULT_MAX_LINES} lines or ${DEFAULT_MAX_BYTES / 1024}KB (whichever is hit first). If truncated, full output is saved to a log file in .pi/logs/ on the runner. Optionally provide a timeout in seconds, or inputFiles to stage generated files (.pi/generated/...) into the runner workspace before running.`,
parameters: BashParameters,
outputLimits: { maxBytes: DEFAULT_MAX_BYTES, maxLines: DEFAULT_MAX_LINES, retain: 'tail' },
execute: async (
{ command, timeout, source, inputFiles },
{ command, timeout, inputFiles },
api,
context,
): Promise<AgentToolResult<BashDetails>> => {
const signal = context.abortSignal
if (options?.restrictedUser && source !== 'skill') {
throw new Error(
'Bash commands requested directly by the user (source="user") are not permitted for your role. ' +
'Only commands required by a loaded skill (source="skill") can be executed.',
)
}
if (options?.teamId) {
await quotaService.checkQuota(
{
@@ -104,7 +84,7 @@ export const createSandboxedBashTool = (
if (options?.runnerId) {
if (!runnerService.isRunnerConnected(options.runnerId)) {
throw new Error(
'Runner is currently offline. Please start your runner with "shumai runner start" and try again.',
'Runner is currently offline. Please start your runner with "shumai-cli runner start" and try again.',
)
}
targetRunnerId = options.runnerId
@@ -211,35 +191,38 @@ export const createSandboxedBashTool = (
}
if (signal?.aborted || execResult.aborted) {
logger.error({ command, outputText }, 'Sandboxed bash command aborted')
logger.error({ command, outputText }, 'Bash command aborted')
throw new Error(appendStatus(outputText, 'Command aborted'))
}
if (execResult.timedOut) {
logger.error({ timeout, command, outputText }, 'Sandboxed bash command timed out')
throw new Error(
appendStatus(outputText, `Sandboxed bash command timed out after ${timeout} seconds`),
)
logger.error({ timeout, command, outputText }, 'Bash command timed out')
throw new Error(appendStatus(outputText, `Bash command timed out after ${timeout} seconds`))
}
if (execResult.error) {
logger.error(
{ error: execResult.error, command, outputText },
'Sandboxed bash command encountered runner error',
'Bash command encountered runner error',
)
throw new Error(appendStatus(outputText, execResult.error))
}
if (execResult.signal) {
logger.error(
{ signal: execResult.signal, command, outputText },
'Bash command was terminated',
)
throw new Error(appendStatus(outputText, 'Bash command terminated by ' + execResult.signal))
}
if (execResult.exitCode !== 0 && execResult.exitCode !== null) {
logger.error(
{ exitCode: execResult.exitCode, command, outputText },
'Sandboxed bash command exited with failure code',
'Bash command exited with failure code',
)
throw new Error(
appendStatus(
outputText,
`Sandboxed bash command exited with code ${execResult.exitCode}`,
),
appendStatus(outputText, `Bash command exited with code ${execResult.exitCode}`),
)
}
+27
View File
@@ -8,6 +8,7 @@ import { s3Service, getStorageBackend } from '@shumai/core/src/s3/s3'
import * as archiveStorageModule from '@shumai/core/src/s3/archive-storage'
import { uploadService } from '@shumai/core/src/upload/upload'
import { metadataService } from '@shumai/core/src/metadata/metadata'
import { watermarkService } from '@shumai/core/src/watermark/watermark'
import { AssetService } from './asset'
import { ensureJpegInStorage } from '@shumai/core/src/s3/preview-converter'
@@ -22,6 +23,7 @@ vi.mock('@shumai/core/src/s3/s3', () => ({
deletePrefix: vi.fn().mockResolvedValue(1),
copyObject: vi.fn().mockResolvedValue(undefined),
getStoragePath: vi.fn((bucket: string, key: string) => `/mock/data/${bucket}/${key}`),
listObjects: vi.fn().mockResolvedValue([]),
},
getStorageBackend: vi.fn().mockReturnValue('local'),
}))
@@ -47,6 +49,31 @@ describe('AssetService', () => {
vi.clearAllMocks()
})
it('checks generated file expiration hourly while continuing regular cleanup', async () => {
vi.useFakeTimers({ toFake: ['Date', 'setTimeout', 'clearTimeout'] })
const startedAt = Date.now()
const regularCleanup = vi
.spyOn(watermarkService, 'purgeOrphanWatermarkConfigs')
.mockResolvedValue(0)
try {
await assetService.startCleanupJob()
await vi.waitFor(() => expect(s3Service.listObjects).toHaveBeenCalledTimes(1))
expect(s3Service.listObjects).toHaveBeenCalledWith(expect.any(String), 'temp/generated/')
await vi.advanceTimersByTimeAsync(5_000)
await vi.waitFor(() => expect(regularCleanup).toHaveBeenCalledTimes(2))
expect(s3Service.listObjects).toHaveBeenCalledTimes(1)
vi.setSystemTime(startedAt + 60 * 60 * 1000)
await vi.advanceTimersByTimeAsync(5_000)
await vi.waitFor(() => expect(s3Service.listObjects).toHaveBeenCalledTimes(2))
} finally {
assetService.stopCleanupJob()
regularCleanup.mockRestore()
vi.useRealTimers()
}
})
const setupBasicAssets = async () => {
const user = await prisma.user.create({
data: {
+7 -1
View File
@@ -1585,7 +1585,9 @@ export class AssetService {
private cleanupJobsRunning = false
async startCleanupJob() {
if (this.cleanupJobsRunning) return
this.cleanupJobsRunning = true
let nextGeneratedCleanupAt = 0
const runCleanup = async () => {
if (!this.cleanupJobsRunning) return
@@ -1594,8 +1596,12 @@ export class AssetService {
await this.expireTrashedAssets()
await this.purgePendingAssets()
await this.purgeUnreferencedStorageKeys()
await this.purgeExpiredGeneratedFiles()
await watermarkService.purgeOrphanWatermarkConfigs()
const now = Date.now()
if (now >= nextGeneratedCleanupAt) {
nextGeneratedCleanupAt = now + 60 * 60 * 1000
await this.purgeExpiredGeneratedFiles()
}
} catch (e: unknown) {
console.error('Error in asset cleanup job:', e)
}
+235 -1
View File
@@ -1,4 +1,5 @@
import { prisma } from '@shumai/db'
import * as advisoryLocks from '@shumai/db/src/advisory-lock'
import { setupTestDbHooks } from '@shumai/db/test'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { s3Service } from '../s3/s3'
@@ -32,6 +33,24 @@ function createMockWs() {
}
}
function reconcile(
service: RunnerService,
runnerId: string,
ws: RunnerSocket,
activeIds: string[] = [],
) {
service.handleMessage(
runnerId,
ws,
JSON.stringify({
type: 'reconcile',
environment: { platform: 'linux', shell: '/bin/sh' },
activeExecutions: activeIds.map((executionId) => ({ executionId, nextOffset: 0 })),
completedExecutions: [],
}),
)
}
describe('RunnerService', () => {
setupTestDbHooks()
@@ -49,12 +68,84 @@ describe('RunnerService', () => {
teamId = team.id
})
afterEach(() => {
afterEach(async () => {
service.stopHeartbeatMonitor()
await service.stopServerTopology()
vi.restoreAllMocks()
})
it('allows replicas without runners and enforces a single server while runners exist', async () => {
const replica = new RunnerService()
try {
await service.startServerTopology()
await replica.startServerTopology()
await expect(service.createRunner(teamId, { name: 'Blocked' })).rejects.toMatchObject({
status: 409,
message: expect.stringContaining('single main Shumai server'),
})
expect(await service.listRunners(teamId)).toEqual([])
await replica.stopServerTopology()
const { runner } = await service.createRunner(teamId, { name: 'Exclusive runner' })
await expect(replica.startServerTopology()).rejects.toThrow('single main Shumai server')
await service.stopServerTopology()
await replica.startServerTopology()
await expect(service.startServerTopology()).rejects.toThrow('single main Shumai server')
const acquireLease = advisoryLocks.tryAcquireUpgradeableAdvisoryLock
vi.spyOn(advisoryLocks, 'tryAcquireUpgradeableAdvisoryLock').mockImplementationOnce(
async (lockKey, exclusive) => {
// The newcomer has already looked up the existing runner.
await replica.deleteRunner(teamId, runner.id)
return acquireLease(lockKey, exclusive)
},
)
await service.startServerTopology()
await expect(replica.createRunner(teamId, { name: 'Blocked again' })).rejects.toMatchObject({
status: 409,
})
} finally {
await replica.stopServerTopology()
}
})
describe('Key & workspace path helpers', () => {
it('keeps validated OS and shell metadata only for the current connection', async () => {
const { runner, key } = await service.createRunner(teamId, { name: 'Windows runner' })
const connection = createMockWs()
const authenticated = {
id: runner.id,
teamId,
name: runner.name,
keyHash: hashRunnerKey(key),
}
expect(service.getRunnerEnvironment(runner.id)).toBeUndefined()
await service.registerConnection(authenticated, connection.ws)
const environment = { platform: 'win32', shell: 'C:\\Windows\\System32\\cmd.exe' }
const reconcile = {
type: 'reconcile',
environment,
activeExecutions: [],
completedExecutions: [],
}
service.handleMessage(runner.id, connection.ws, JSON.stringify(reconcile))
expect(service.getRunnerEnvironment(runner.id)).toEqual(environment)
service.handleMessage(
runner.id,
connection.ws,
JSON.stringify({ ...reconcile, environment: { platform: 'invalid', shell: '' } }),
)
expect(service.getRunnerEnvironment(runner.id)).toEqual(environment)
const replacement = createMockWs()
await service.registerConnection(authenticated, replacement.ws)
service.handleMessage(runner.id, connection.ws, JSON.stringify(reconcile))
expect(service.getRunnerEnvironment(runner.id)).toBeUndefined()
service.handleMessage(runner.id, replacement.ws, JSON.stringify(reconcile))
service.handleDisconnect(runner.id, replacement.ws)
expect(service.getRunnerEnvironment(runner.id)).toBeUndefined()
})
it('generates and hashes runner keys', () => {
const { key, keyPrefix, keyHash } = generateRunnerKey()
expect(key).toMatch(/^shm_rn_[0-9a-f]{48}$/)
@@ -166,6 +257,25 @@ describe('RunnerService', () => {
})
describe('WebSocket connection lifecycle & eviction', () => {
it('waits for reconciliation before reporting online or dispatching a command', async () => {
const { runner, key } = await service.createRunner(teamId, { name: 'Handshaking runner' })
const conn = createMockWs()
await service.registerConnection(
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
conn.ws,
)
expect(conn.lastSentJson()).toHaveProperty('type', 'welcome')
expect(service.isRunnerConnected(runner.id)).toBe(false)
await expect(service.exec(runner.id, { command: 'echo ready' })).rejects.toThrow(
'Runner is not connected',
)
expect(conn.sent).toHaveLength(1)
reconcile(service, runner.id, conn.ws)
expect(service.isRunnerConnected(runner.id)).toBe(true)
})
it('evicts existing connection with 4000 when a new connection arrives', async () => {
const { runner, key } = await service.createRunner(teamId, { name: 'Runner WS' })
const keyHash = hashRunnerKey(key)
@@ -176,6 +286,7 @@ describe('RunnerService', () => {
{ id: runner.id, teamId, name: runner.name, keyHash },
conn1.ws,
)
reconcile(service, runner.id, conn1.ws)
expect(service.isRunnerConnected(runner.id)).toBe(true)
await service.registerConnection(
@@ -186,6 +297,7 @@ describe('RunnerService', () => {
code: 4000,
reason: 'Replaced by a newer connection',
})
reconcile(service, runner.id, conn2.ws)
expect(service.isRunnerConnected(runner.id)).toBe(true)
})
@@ -259,6 +371,7 @@ describe('RunnerService', () => {
{ id: r2.runner.id, teamId, name: r2.runner.name, keyHash: hashRunnerKey(r2.key) },
conn2.ws,
)
reconcile(service, r2.runner.id, conn2.ws)
const online = await service.requireOnlineRunner(teamId, r2.runner.id)
expect(online.id).toBe(r2.runner.id)
})
@@ -272,6 +385,7 @@ describe('RunnerService', () => {
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
conn.ws,
)
reconcile(service, runner.id, conn.ws)
const chunks: Array<{ stream: string; text: string }> = []
const execPromise = service.exec(runner.id, {
@@ -348,6 +462,7 @@ describe('RunnerService', () => {
conn.ws,
)
reconcile(service, runner.id, conn.ws)
const streamed: string[] = []
const noCollectPromise = service.exec(runner.id, {
command: 'yes | head -n 100',
@@ -421,6 +536,7 @@ describe('RunnerService', () => {
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
conn.ws,
)
reconcile(service, runner.id, conn.ws)
const abortController = new AbortController()
const execPromise = service.exec(runner.id, {
@@ -454,6 +570,121 @@ describe('RunnerService', () => {
expect(res.aborted).toBe(true)
})
it.each([false, true])(
'replays cancellation on reconnect after notifying caller=%s',
async (reported) => {
const { runner, key } = await service.createRunner(teamId, {
name: 'Cancelled while offline',
})
const authenticated = {
id: runner.id,
teamId,
name: runner.name,
keyHash: hashRunnerKey(key),
}
const conn = createMockWs()
await service.registerConnection(authenticated, conn.ws)
reconcile(service, runner.id, conn.ws)
const abort = new AbortController()
const execution = service.exec(runner.id, { command: 'sleep 30', signal: abort.signal })
const { executionId } = conn.lastSentJson()
service.handleDisconnect(runner.id, conn.ws)
abort.abort()
if (reported) await expect(execution).resolves.toMatchObject({ aborted: true })
const reconnected = createMockWs()
await service.registerConnection(authenticated, reconnected.ws)
expect(reconnected.lastSentJson()).toMatchObject({ inFlightExecutionIds: [executionId] })
reconcile(service, runner.id, reconnected.ws, [executionId])
const forwarded = reconnected.lastSentJson()
service.handleMessage(
runner.id,
reconnected.ws,
JSON.stringify({
type: 'exec.result',
executionId,
exitCode: 0,
signal: null,
timedOut: false,
}),
)
expect(forwarded).toEqual({ type: 'exec.cancel', executionId })
await expect(execution).resolves.toMatchObject({ aborted: true })
},
)
it('restages a generated input that was removed from the runner workspace', async () => {
vi.spyOn(s3Service, 'headObject').mockResolvedValue({
key: 'temp/generated/example/input.jpg',
size: 100,
lastModified: new Date(),
contentType: 'image/jpeg',
eTag: 'fixture',
})
vi.spyOn(s3Service, 'presign').mockResolvedValue('https://signed.example.com/input.jpg')
const { runner, key } = await service.createRunner(teamId, { name: 'Generated file runner' })
const conn = createMockWs()
await service.registerConnection(
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
conn.ws,
)
reconcile(service, runner.id, conn.ws)
const destPath = '.pi/generated/example/input.jpg'
const acknowledgeDownload = () => {
const download = conn.lastSentJson()
expect(download).toMatchObject({ type: 'file.download', destPath })
service.handleMessage(
runner.id,
conn.ws,
JSON.stringify({
type: 'file.download.result',
executionId: download.executionId,
ok: true,
}),
)
}
const first = service.ensureGeneratedFileOnRunner(runner.id, destPath)
await vi.waitFor(() => expect(conn.lastSentJson()?.type).toBe('file.download'))
acknowledgeDownload()
await expect(first).resolves.toBe(destPath)
const second = service.ensureGeneratedFileOnRunner(runner.id, destPath)
const stat = conn.lastSentJson()
expect(stat).toMatchObject({ type: 'file.stat', srcPath: destPath })
service.handleMessage(
runner.id,
conn.ws,
JSON.stringify({
type: 'file.stat.result',
executionId: stat.executionId,
ok: true,
exists: false,
}),
)
await vi.waitFor(() => expect(conn.lastSentJson()?.type).toBe('file.download'))
acknowledgeDownload()
await expect(second).resolves.toBe(destPath)
const third = service.ensureGeneratedFileOnRunner(runner.id, destPath)
const present = conn.lastSentJson()
expect(present).toMatchObject({ type: 'file.stat', srcPath: destPath })
service.handleMessage(
runner.id,
conn.ws,
JSON.stringify({
type: 'file.stat.result',
executionId: present.executionId,
ok: true,
exists: true,
isFile: true,
}),
)
await expect(third).resolves.toBe(destPath)
expect(conn.lastSentJson()).toEqual(present)
})
it('handles downloadToRunner, uploadFromRunner, statOnRunner, and session caching', async () => {
vi.spyOn(s3Service, 'presign').mockResolvedValue('https://signed.example.com/file')
const { runner, key } = await service.createRunner(teamId, { name: 'File Runner' })
@@ -462,6 +693,7 @@ describe('RunnerService', () => {
{ id: runner.id, teamId, name: runner.name, keyHash: hashRunnerKey(key) },
conn.ws,
)
reconcile(service, runner.id, conn.ws)
// 1. downloadToRunner
const dlPromise = service.downloadToRunner(runner.id, {
@@ -581,6 +813,7 @@ describe('RunnerService', () => {
{ id: runner.id, teamId, name: runner.name, keyHash },
conn1.ws,
)
reconcile(service, runner.id, conn1.ws)
const execPromise = service.exec(runner.id, { command: 'long-job' })
const startMsg = conn1.lastSentJson()
@@ -599,6 +832,7 @@ describe('RunnerService', () => {
conn2.ws,
JSON.stringify({
type: 'reconcile',
environment: { platform: 'linux', shell: '/bin/sh' },
activeExecutions: [{ executionId: startMsg.executionId, nextOffset: 0 }],
completedExecutions: [],
}),
+139 -103
View File
@@ -11,6 +11,8 @@ import {
RunnerFileStatResultMessage,
RunnerFileUploadResultMessage,
RunnerInfo,
RunnerEnvironment,
RUNNER_REPLACED_CLOSE_CODE,
RunnerToServerMessage,
runnerToServerMessageSchema,
ServerToRunnerMessage,
@@ -25,6 +27,7 @@ import type {
RunnerExecResult,
RunnerSocket,
} from './types'
import { RunnerTopology } from './runner-topology'
const RUNNER_KEY_PREFIX = 'shm_rn_'
const DEFAULT_RECONNECT_GRACE_MS = 30_000
@@ -42,6 +45,7 @@ interface ActiveRunnerSession {
lastPongAt: number
stagedSkills: Map<string, string>
stagedGeneratedFiles: Set<string>
environment?: RunnerEnvironment
}
interface PendingExecOperation {
@@ -58,6 +62,8 @@ interface PendingExecOperation {
resolve: (result: RunnerExecResult) => void
reject: (err: Error) => void
timeoutTimer?: ReturnType<typeof setTimeout>
abortTimer?: ReturnType<typeof setTimeout>
cancelRequested?: boolean
cleanupSignal?: () => void
}
@@ -130,11 +136,23 @@ export function validateRunnerWorkspacePath(targetPath: string): string {
export class RunnerService {
private sessions = new Map<string, ActiveRunnerSession>()
private closedSockets = new WeakSet<RunnerSocket>()
private pendingOps = new Map<string, PendingOperation>()
private disconnectGraceTimers = new Map<string, ReturnType<typeof setTimeout>>()
private credentialRevisions = new Map<string, number>()
private heartbeatTimer: ReturnType<typeof setInterval> | null = null
private reconnectGraceMs = DEFAULT_RECONNECT_GRACE_MS
private serverTopology?: RunnerTopology
async startServerTopology(): Promise<AbortSignal> {
this.serverTopology ??= await RunnerTopology.start()
return this.serverTopology.signal
}
async stopServerTopology(): Promise<void> {
await this.serverTopology?.close()
this.serverTopology = undefined
}
setReconnectGraceMs(ms: number): void {
this.reconnectGraceMs = ms
@@ -155,7 +173,7 @@ export class RunnerService {
this.handleDisconnect(runnerId, session.ws, 1001)
continue
}
this.sendToSession(session, { type: 'ping', ts: now })
if (session.environment) this.sendToSession(session, { type: 'ping', ts: now })
}
}, HEARTBEAT_INTERVAL_MS)
if (typeof this.heartbeatTimer.unref === 'function') {
@@ -171,7 +189,11 @@ export class RunnerService {
}
isRunnerConnected(runnerId: string): boolean {
return this.sessions.has(runnerId)
return this.getReadySession(runnerId) !== undefined
}
getRunnerEnvironment(runnerId: string): RunnerEnvironment | undefined {
return this.sessions.get(runnerId)?.environment
}
// ----------------------------------------------------------------------
@@ -197,6 +219,14 @@ export class RunnerService {
}
async createRunner(teamId: string, req: CreateRunnerRequest): Promise<CreateRunnerResponse> {
const create = () => this.persistRunner(teamId, req)
return this.serverTopology ? this.serverTopology.changeRunners(create, true) : create()
}
private async persistRunner(
teamId: string,
req: CreateRunnerRequest,
): Promise<CreateRunnerResponse> {
const name = req.name.trim()
const { key, keyPrefix, keyHash } = generateRunnerKey()
@@ -343,6 +373,11 @@ export class RunnerService {
}
async deleteRunner(teamId: string, runnerId: string): Promise<void> {
const remove = () => this.removeRunner(teamId, runnerId)
await (this.serverTopology ? this.serverTopology.changeRunners(remove) : remove())
}
private async removeRunner(teamId: string, runnerId: string): Promise<void> {
this.bumpCredentialRevision(runnerId)
await prisma.$transaction(async (tx) => {
@@ -428,11 +463,11 @@ export class RunnerService {
if (!this.isRunnerConnected(resolved.runner.id)) {
if (resolved.isBoundToAgent) {
throw new Error(
`Runner "${resolved.runner.name}" is currently offline. Please start the runner ("shumai runner start") and try again.`,
`Runner "${resolved.runner.name}" is currently offline. Please start the runner ("shumai-cli runner start") and try again.`,
)
}
throw new Error(
`Default runner "${resolved.runner.name}" is currently offline. Please start the runner ("shumai runner start") and try again.`,
`Default runner "${resolved.runner.name}" is currently offline. Please start the runner ("shumai-cli runner start") and try again.`,
)
}
@@ -466,6 +501,7 @@ export class RunnerService {
},
select: { id: true, teamId: true, name: true },
})
if (this.closedSockets.has(ws)) return
const currentRevision = this.credentialRevisions.get(runner.id) ?? 0
if (!verified || currentRevision !== startRevision) {
try {
@@ -476,19 +512,13 @@ export class RunnerService {
return
}
const existingTimer = this.disconnectGraceTimers.get(runner.id)
if (existingTimer) {
clearTimeout(existingTimer)
this.disconnectGraceTimers.delete(runner.id)
}
const existingSession = this.sessions.get(runner.id)
if (existingSession && existingSession.ws !== ws) {
logger.info({ runnerId: runner.id }, 'Evicting previous runner WebSocket connection')
// Remove from sessions before closing so handleDisconnect is a no-op for the old socket
this.sessions.delete(runner.id)
try {
existingSession.ws.close(4000, 'Replaced by a newer connection')
existingSession.ws.close(RUNNER_REPLACED_CLOSE_CODE, 'Replaced by a newer connection')
} catch {
// Ignore close errors
}
@@ -506,19 +536,6 @@ export class RunnerService {
this.sessions.set(runner.id, session)
this.startHeartbeatMonitor()
await prisma.runner
.update({
where: { id: runner.id },
data: { lastConnectedAt: new Date() },
})
.catch((err) => {
logger.warn({ err, runnerId: runner.id }, 'Failed to update runner lastConnectedAt')
})
if (this.sessions.get(runner.id) !== session) {
return
}
const inFlightExecutionIds = this.getInFlightExecutionIdsForRunner(runner.id)
this.sendToSession(session, {
type: 'welcome',
@@ -559,7 +576,13 @@ export class RunnerService {
case 'pong':
break
case 'reconcile':
if (session.environment) break
this.handleReconcileMessage(session, msg)
session.environment = msg.environment
this.clearDisconnectGraceTimer(runnerId)
void prisma.runner
.update({ where: { id: runnerId }, data: { lastConnectedAt: new Date() } })
.catch((err) => logger.warn({ err, runnerId }, 'Failed to update runner lastConnectedAt'))
break
case 'exec.output':
this.handleExecOutputMessage(runnerId, msg)
@@ -580,6 +603,7 @@ export class RunnerService {
}
handleDisconnect(runnerId: string, ws: RunnerSocket, code?: number): void {
this.closedSockets.add(ws)
const session = this.sessions.get(runnerId)
if (!session || session.ws !== ws) {
return
@@ -601,6 +625,7 @@ export class RunnerService {
}
if (code === 1008 || this.reconnectGraceMs <= 0) {
this.clearDisconnectGraceTimer(runnerId)
this.failPendingOperationsForRunner(
runnerId,
new Error('Runner disconnected while operation was in progress'),
@@ -608,6 +633,7 @@ export class RunnerService {
return
}
if (this.disconnectGraceTimers.has(runnerId)) return
const timer = setTimeout(() => {
this.disconnectGraceTimers.delete(runnerId)
if (!this.isRunnerConnected(runnerId)) {
@@ -629,10 +655,7 @@ export class RunnerService {
// ----------------------------------------------------------------------
async exec(runnerId: string, options: RunnerExecOptions): Promise<RunnerExecResult> {
const session = this.sessions.get(runnerId)
if (!session) {
throw new Error('Runner is not connected')
}
const session = this.requireReadySession(runnerId)
if (options.signal?.aborted) {
throw new Error('Command aborted before execution started')
@@ -658,51 +681,22 @@ export class RunnerService {
const safetyTimeoutMs = Math.max((timeoutSec + 15) * 1000, 15_000)
const timeoutTimer = setTimeout(() => {
const pending = this.pendingOps.get(executionId)
if (!pending) return
if (pending?.kind !== 'exec') return
this.cleanupPendingOp(executionId)
const currentSession = this.sessions.get(runnerId)
const currentSession = this.getReadySession(runnerId)
if (currentSession) {
this.sendToSession(currentSession, { type: 'exec.cancel', executionId })
}
resolve({
executionId,
output: pending.kind === 'exec' ? this.readBufferedExecOutput(pending) : '',
output: this.readBufferedExecOutput(pending),
exitCode: null,
signal: 'SIGKILL',
timedOut: true,
aborted: false,
timedOut: !pending.cancelRequested,
aborted: pending.cancelRequested ?? false,
})
}, safetyTimeoutMs)
let cleanupSignal: (() => void) | undefined
if (options.signal) {
const onAbort = () => {
const currentSession = this.sessions.get(runnerId)
if (currentSession) {
this.sendToSession(currentSession, { type: 'exec.cancel', executionId })
}
// Give runner a short window to report exec.result, otherwise resolve as aborted
const abortTimer = setTimeout(() => {
const pending = this.pendingOps.get(executionId)
if (!pending) return
this.cleanupPendingOp(executionId)
resolve({
executionId,
output: pending.kind === 'exec' ? this.readBufferedExecOutput(pending) : '',
exitCode: null,
signal: 'SIGKILL',
timedOut: false,
aborted: true,
})
}, 2_000)
if (typeof abortTimer.unref === 'function') {
abortTimer.unref()
}
}
options.signal.addEventListener('abort', onAbort, { once: true })
cleanupSignal = () => options.signal?.removeEventListener('abort', onAbort)
}
const pendingOp: PendingExecOperation = {
kind: 'exec',
executionId,
@@ -717,7 +711,26 @@ export class RunnerService {
resolve,
reject,
timeoutTimer,
cleanupSignal,
}
if (options.signal) {
const signal = options.signal
const onAbort = () => {
pendingOp.cancelRequested = true
// Release the caller after a short grace period. Retain cancellation for reconnects.
pendingOp.abortTimer = setTimeout(() => {
this.reportCancelledExecution(pendingOp)
pendingOp.onOutput = undefined
pendingOp.collectOutput = false
pendingOp.outputChunks = []
pendingOp.bufferedOutputBytes = 0
}, 2_000)
pendingOp.abortTimer.unref?.()
const currentSession = this.getReadySession(runnerId)
if (currentSession)
this.sendToSession(currentSession, { type: 'exec.cancel', executionId })
}
signal.addEventListener('abort', onAbort, { once: true })
pendingOp.cleanupSignal = () => signal.removeEventListener('abort', onAbort)
}
this.pendingOps.set(executionId, pendingOp)
@@ -735,10 +748,7 @@ export class RunnerService {
timeoutMs?: number
},
): Promise<RunnerFileDownloadResultMessage> {
const session = this.sessions.get(runnerId)
if (!session) {
throw new Error('Runner is not connected')
}
const session = this.requireReadySession(runnerId)
const normalizedDestPath = validateRunnerWorkspacePath(options.destPath)
const executionId = ulid()
@@ -777,10 +787,7 @@ export class RunnerService {
srcPath: string,
timeoutMs = DEFAULT_FILE_OP_TIMEOUT_MS,
): Promise<RunnerFileStatResultMessage> {
const session = this.sessions.get(runnerId)
if (!session) {
throw new Error('Runner is not connected')
}
const session = this.requireReadySession(runnerId)
const normalizedSrcPath = validateRunnerWorkspacePath(srcPath)
const executionId = ulid()
@@ -819,10 +826,7 @@ export class RunnerService {
timeoutMs?: number
},
): Promise<RunnerFileUploadResultMessage> {
const session = this.sessions.get(runnerId)
if (!session) {
throw new Error('Runner is not connected')
}
const session = this.requireReadySession(runnerId)
const normalizedSrcPath = validateRunnerWorkspacePath(options.srcPath)
const executionId = ulid()
@@ -859,10 +863,7 @@ export class RunnerService {
runnerId: string,
skill: { id: string; assetId: string; hash: string },
): Promise<string> {
const session = this.sessions.get(runnerId)
if (!session) {
throw new Error('Runner is not connected')
}
const session = this.requireReadySession(runnerId)
const destPath = `.pi/skills/${skill.id}`
if (session.stagedSkills.get(skill.id) === skill.hash) {
@@ -903,10 +904,7 @@ export class RunnerService {
}
async ensureGeneratedFileOnRunner(runnerId: string, virtualPath: string): Promise<string> {
const session = this.sessions.get(runnerId)
if (!session) {
throw new Error('Runner is not connected')
}
const session = this.requireReadySession(runnerId)
const normalizedPath = validateRunnerWorkspacePath(virtualPath)
if (!normalizedPath.startsWith('.pi/generated/')) {
@@ -916,7 +914,10 @@ export class RunnerService {
}
if (session.stagedGeneratedFiles.has(normalizedPath)) {
return normalizedPath
const stat = await this.statOnRunner(runnerId, normalizedPath)
if (stat.ok && stat.exists && stat.isFile && this.getReadySession(runnerId) === session) {
return normalizedPath
}
}
// Virtual path .pi/generated/<ulid>/<filename> maps to storage key temp/generated/<ulid>/<filename>
@@ -954,34 +955,54 @@ export class RunnerService {
// Internal Helpers
// ----------------------------------------------------------------------
private getReadySession(runnerId: string): ActiveRunnerSession | undefined {
const session = this.sessions.get(runnerId)
return session?.environment ? session : undefined
}
private requireReadySession(runnerId: string): ActiveRunnerSession {
const session = this.getReadySession(runnerId)
if (!session) throw new Error('Runner is not connected')
return session
}
private handleReconcileMessage(
session: ActiveRunnerSession,
msg: Extract<RunnerToServerMessage, { type: 'reconcile' }>,
): void {
const completedIds = new Set<string>()
for (const completed of msg.completedExecutions) {
completedIds.add(completed.executionId)
this.handleExecResultMessage(session.runnerId, completed)
}
const activeIds = new Set(msg.activeExecutions.map((a) => a.executionId))
for (const executionId of activeIds) {
const pending = this.pendingOps.get(executionId)
if (
pending?.runnerId !== session.runnerId ||
pending.kind !== 'exec' ||
pending.cancelRequested
) {
// A timeout or cancellation may have occurred after the welcome snapshot.
this.sendToSession(session, { type: 'exec.cancel', executionId })
}
}
for (const [executionId, pending] of this.pendingOps.entries()) {
if (pending.runnerId !== session.runnerId) continue
if (completedIds.has(executionId) || activeIds.has(executionId)) {
continue
}
if (pending.kind === 'exec') {
if (pending.kind !== 'exec') {
// Re-send idempotent file control operations on reconnect.
this.sendToSession(session, pending.message)
} else if (!activeIds.has(executionId)) {
this.cleanupPendingOp(executionId)
if (pending.cancelRequested) {
this.reportCancelledExecution(pending)
continue
}
pending.reject(
new Error(
`Runner reconnected without active execution ${executionId} (runner process may have restarted)`,
),
)
} else {
// Re-send idempotent file control operations on reconnect
this.sendToSession(session, pending.message)
}
}
}
@@ -1024,8 +1045,8 @@ export class RunnerService {
output: this.readBufferedExecOutput(pending),
exitCode: msg.exitCode,
signal: msg.signal,
timedOut: msg.timedOut,
aborted: msg.aborted ?? false,
timedOut: msg.timedOut && !pending.cancelRequested,
aborted: pending.cancelRequested || (msg.aborted ?? false),
fullOutputPath: msg.fullOutputPath,
error: msg.error,
})
@@ -1126,11 +1147,7 @@ export class RunnerService {
private revokeActiveSession(runnerId: string, reason: string): void {
this.bumpCredentialRevision(runnerId)
const timer = this.disconnectGraceTimers.get(runnerId)
if (timer) {
clearTimeout(timer)
this.disconnectGraceTimers.delete(runnerId)
}
this.clearDisconnectGraceTimer(runnerId)
const session = this.sessions.get(runnerId)
if (session) {
@@ -1145,6 +1162,24 @@ export class RunnerService {
this.failPendingOperationsForRunner(runnerId, new Error(reason))
}
private clearDisconnectGraceTimer(runnerId: string): void {
const timer = this.disconnectGraceTimers.get(runnerId)
if (!timer) return
clearTimeout(timer)
this.disconnectGraceTimers.delete(runnerId)
}
private reportCancelledExecution(pending: PendingExecOperation): void {
pending.resolve({
executionId: pending.executionId,
output: this.readBufferedExecOutput(pending),
exitCode: null,
signal: 'SIGKILL',
timedOut: false,
aborted: true,
})
}
private failPendingOperationsForRunner(runnerId: string, error: Error): void {
for (const [executionId, pending] of this.pendingOps.entries()) {
if (pending.runnerId === runnerId) {
@@ -1161,8 +1196,9 @@ export class RunnerService {
if (pending.timeoutTimer) {
clearTimeout(pending.timeoutTimer)
}
if (pending.kind === 'exec' && pending.cleanupSignal) {
pending.cleanupSignal()
if (pending.kind === 'exec') {
clearTimeout(pending.abortTimer)
pending.cleanupSignal?.()
}
}
@@ -0,0 +1,76 @@
import { HTTPException } from 'hono/http-exception'
import { prisma } from '@shumai/db'
import { tryAcquireUpgradeableAdvisoryLock } from '@shumai/db/src/advisory-lock'
import { logger } from '@shumai/core/src/logger'
const RUNNER_TOPOLOGY_KEY = 'shumai:runner-topology'
const SINGLE_SERVER_REQUIRED =
'Agent runners require a single main Shumai server per database. Stop the other main server replicas.'
async function hasConfiguredRunners() {
return (await prisma.runner.findFirst({ select: { id: true } })) !== null
}
/** Each main server holds a shared lease. A configured runner requires an exclusive lease. */
export class RunnerTopology {
private mutations = Promise.resolve()
private constructor(
private readonly lease: NonNullable<
Awaited<ReturnType<typeof tryAcquireUpgradeableAdvisoryLock>>
>,
) {}
static async start() {
const configured = await hasConfiguredRunners()
let lease = await tryAcquireUpgradeableAdvisoryLock(RUNNER_TOPOLOGY_KEY, configured)
if (!lease && configured) {
// If the last runner was just deleted, the incumbent may now hold a shared lease.
lease = await tryAcquireUpgradeableAdvisoryLock(RUNNER_TOPOLOGY_KEY, false)
}
if (!lease) throw new Error(SINGLE_SERVER_REQUIRED)
try {
// A concurrently stopping server may have changed the catalog.
if (await hasConfiguredRunners()) {
if (!(await lease.tryUpgrade())) throw new Error(SINGLE_SERVER_REQUIRED)
} else {
await lease.downgrade()
}
return new RunnerTopology(lease)
} catch (error) {
await lease.release()
throw error
}
}
get signal() {
return this.lease.signal
}
/** Serialize runner creation and deletion until the exclusive lease can be safely downgraded. */
async changeRunners<T>(operation: () => Promise<T>, adding = false): Promise<T> {
const previous = this.mutations
const finished = Promise.withResolvers<void>()
this.mutations = finished.promise
await previous
try {
if (adding && !(await this.lease.tryUpgrade())) {
throw new HTTPException(409, { message: SINGLE_SERVER_REQUIRED })
}
return await operation()
} finally {
try {
if (this.lease.exclusive && !(await hasConfiguredRunners())) await this.lease.downgrade()
} catch (error) {
logger.warn({ err: error }, 'Failed to downgrade the runner server lease')
} finally {
finished.resolve()
}
}
}
async close() {
await this.mutations
await this.lease.release()
}
}
+44
View File
@@ -717,6 +717,50 @@ describe('S3Service implementations', () => {
expect(keysAll.length).toBe(3)
})
it('lists a nested prefix without scanning unrelated directories', async () => {
await localS3.putObject('test-bucket', 'temp/generated/one.jpg', 'one', 3)
await localS3.putObject('test-bucket', 'files/unrelated.txt', 'abc', 3)
const readDirectory = vi.spyOn(fs.promises, 'readdir')
try {
await expect(localS3.listObjects('test-bucket', 'temp/generated/')).resolves.toEqual([
'temp/generated/one.jpg',
])
expect(readDirectory.mock.calls.map(([directory]) => directory)).toEqual([
path.join(TEST_BASE_PATH, 'test-bucket', 'temp', 'generated'),
])
readDirectory.mockClear()
await expect(localS3.listObjects('test-bucket', 'temp/missing/')).resolves.toEqual([])
expect(readDirectory.mock.calls.map(([directory]) => directory)).not.toContain(
path.join(TEST_BASE_PATH, 'test-bucket'),
)
await expect(localS3.listObjects('test-bucket', 'temp/generated/on')).resolves.toEqual([
'temp/generated/one.jpg',
])
} finally {
readDirectory.mockRestore()
}
})
it('continues listing when a file disappears during a local prefix scan', async () => {
const removedKey = 'temp/generated/removed.jpg'
const remainingKey = 'temp/generated/remaining.jpg'
await localS3.putObject('test-bucket', removedKey, 'old', 3)
await localS3.putObject('test-bucket', remainingKey, 'new', 3)
const stat = fs.promises.stat
const removedPath = path.join(TEST_BASE_PATH, 'test-bucket', removedKey)
const statSpy = vi.spyOn(fs.promises, 'stat').mockImplementation(async (target, options) => {
if (target === removedPath) await fs.promises.unlink(removedPath)
return stat(target, options)
})
try {
await expect(localS3.listObjects('test-bucket', 'temp/generated/')).resolves.toEqual([
remainingKey,
])
} finally {
statSpy.mockRestore()
}
})
it('should handle headObject correctly', async () => {
await localS3.putObject('test-bucket', 'test.txt', 'hello world', 11)
const head = await localS3.headObject('test-bucket', 'test.txt')
+23 -6
View File
@@ -1012,16 +1012,33 @@ export class LocalStorageService implements S3Service {
async listObjects(bucket: string, prefix: string): Promise<string[]> {
const bucketDir = path.join(this.basePath, bucket)
const keys: string[] = []
if (!fs.existsSync(bucketDir)) {
const directoryPrefix = prefix.slice(0, prefix.lastIndexOf('/') + 1)
const directory = path.join(bucketDir, directoryPrefix.slice(0, -1))
const relativeDirectory = path.relative(bucketDir, directory).split(path.sep).join('/')
if (
directoryPrefix &&
(!relativeDirectory ||
relativeDirectory === '..' ||
relativeDirectory.startsWith('../') ||
relativeDirectory + '/' !== directoryPrefix)
)
return keys
}
const ignoreMissing = (error: unknown) => {
if (
error instanceof Error &&
'code' in error &&
(error.code === 'ENOENT' || error.code === 'ENOTDIR')
)
return undefined
throw error
}
const walkDir = async (currentDir: string) => {
const files = await fs.promises.readdir(currentDir)
const files = (await fs.promises.readdir(currentDir).catch(ignoreMissing)) ?? []
for (const file of files) {
const fullPath = path.join(currentDir, file)
const stat = await fs.promises.stat(fullPath)
const stat = await fs.promises.stat(fullPath).catch(ignoreMissing)
if (!stat) continue
if (stat.isDirectory()) {
await walkDir(fullPath)
@@ -1037,7 +1054,7 @@ export class LocalStorageService implements S3Service {
}
}
await walkDir(bucketDir)
await walkDir(directory)
return keys
}
+65
View File
@@ -39,3 +39,68 @@ export async function acquireAdvisoryLock(key: string, signal?: AbortSignal) {
},
}
}
/** Hold a shared server lease, with a non-blocking upgrade to exclusive ownership. */
export async function tryAcquireUpgradeableAdvisoryLock(key: string, exclusive: boolean) {
const client = await locks.connect()
const lost = new AbortController()
const onError = (error: Error) => lost.abort(error)
client.on('error', onError)
const tryExclusive = async () => {
const result = await client.query<{ acquired: boolean }>(
'SELECT pg_try_advisory_lock(hashtextextended($1, 0)) AS acquired',
[key],
)
return result.rows[0]?.acquired === true
}
let acquired = false
try {
if (exclusive && !(await tryExclusive())) return null
const result = await client.query<{ acquired: boolean }>(
'SELECT pg_try_advisory_lock_shared(hashtextextended($1, 0)) AS acquired',
[key],
)
lost.signal.throwIfAborted()
acquired = result.rows[0]?.acquired === true
if (!acquired) return null
} finally {
if (!acquired) {
client.off('error', onError)
client.release(true)
}
}
let released = false
const assertHeld = () => {
if (released) throw new Error('The server lease has been released')
lost.signal.throwIfAborted()
}
return {
signal: lost.signal,
get exclusive() {
return exclusive
},
async tryUpgrade() {
assertHeld()
if (!exclusive) exclusive = await tryExclusive()
return exclusive
},
async downgrade() {
assertHeld()
if (!exclusive) return
await client.query('SELECT pg_advisory_unlock(hashtextextended($1, 0))', [key])
exclusive = false
},
async release() {
if (released) return
released = true
try {
if (!lost.signal.aborted) await client.query('SELECT pg_advisory_unlock_all()')
} finally {
client.off('error', onError)
client.release(true)
}
},
}
}
+9
View File
@@ -48,6 +48,8 @@ export type RotateRunnerKeyResponse = z.infer<typeof rotateRunnerKeyResponseSche
// WebSocket Wire Protocol Schemas
// ----------------------------------------------------------------------
export const RUNNER_REPLACED_CLOSE_CODE = 4000
export const runnerWelcomeMessageSchema = z.object({
type: z.literal('welcome'),
runnerId: z.string(),
@@ -136,8 +138,15 @@ export const runnerExecResultMessageSchema = z.object({
})
export type RunnerExecResultMessage = z.infer<typeof runnerExecResultMessageSchema>
export const runnerEnvironmentSchema = z.object({
platform: z.enum(['darwin', 'linux', 'win32']),
shell: z.string().min(1).max(1024),
})
export type RunnerEnvironment = z.infer<typeof runnerEnvironmentSchema>
export const runnerReconcileMessageSchema = z.object({
type: z.literal('reconcile'),
environment: runnerEnvironmentSchema,
activeExecutions: z.array(
z.object({
executionId: z.string(),
+10
View File
@@ -262,6 +262,16 @@ describe('Workflow E2E - read_asset', () => {
{ id: runner.id, teamId: team.id, name: runner.name, keyHash: hashRunnerKey(key) },
runnerSocket,
)
runnerService.handleMessage(
runner.id,
runnerSocket,
JSON.stringify({
type: 'reconcile',
environment: { platform: 'linux', shell: '/bin/sh' },
activeExecutions: [],
completedExecutions: [],
}),
)
const video = await createFileAsset(
project.id,
@@ -513,7 +513,7 @@ export function RunnerSettings({ teamId }: { teamId: string }) {
<div className="flex items-center gap-2">
<Input
readOnly
value={`shumai runner start --url ${serverOrigin} --key ${revealedKeyData.key}`}
value={`shumai-cli runner start --url ${serverOrigin} --key ${revealedKeyData.key}`}
className="font-mono text-xs bg-muted"
/>
<Button
@@ -522,7 +522,7 @@ export function RunnerSettings({ teamId }: { teamId: string }) {
size="icon"
onClick={() =>
handleCopy(
`shumai runner start --url ${serverOrigin} --key ${revealedKeyData.key}`,
`shumai-cli runner start --url ${serverOrigin} --key ${revealedKeyData.key}`,
'cmd',
)
}
@@ -543,7 +543,7 @@ export function RunnerSettings({ teamId }: { teamId: string }) {
<div className="flex items-center gap-2">
<Input
readOnly
value={`shumai runner start --url ${serverOrigin} --key ${revealedKeyData.key} --sandbox`}
value={`shumai-cli runner start --url ${serverOrigin} --key ${revealedKeyData.key} --srt sandbox.json`}
className="font-mono text-xs bg-muted"
/>
<Button
@@ -552,7 +552,7 @@ export function RunnerSettings({ teamId }: { teamId: string }) {
size="icon"
onClick={() =>
handleCopy(
`shumai runner start --url ${serverOrigin} --key ${revealedKeyData.key} --sandbox`,
`shumai-cli runner start --url ${serverOrigin} --key ${revealedKeyData.key} --srt sandbox.json`,
'sandboxCmd',
)
}
+5 -5
View File
@@ -1703,10 +1703,10 @@
"task_type_restore_asset": "Restore Asset",
"agent_runners": "Agent Runners",
"agent_runners_settings": "Agent Runners",
"runners_description": "Register and manage remote runners that execute bash commands and skills for your agents.",
"runners_description": "Register and manage remote runners. Everyone with shell access to a runner can read files in its workspace, including files from other projects.",
"runners": "Runners",
"create_runner": "Create Runner",
"create_runner_description": "Register a new runner for your team. You will receive a one-time runner key to connect from shumai-cli.",
"create_runner_description": "Register a runner for your team and get a one-time key for shumai-cli. Share the runner only with users who may read all files in its workspace.",
"runner_name": "Runner Name",
"runner_name_placeholder": "e.g. Mac Mini Runner",
"runner_description": "Description (Optional)",
@@ -1732,7 +1732,7 @@
"runner_quickstart_command_label": "Quickstart Command",
"runner_quickstart_sandbox_label": "Start with Sandbox (Optional)",
"runner_empty_title": "No runners configured",
"runner_empty_description": "When no runner is configured, the bash and download_asset tools are automatically hidden from agents. Create a runner and start it with shumai-cli to enable bash execution.",
"runner_empty_description": "When no runner is configured, the bash and download_asset tools are automatically hidden from agents. Create a runner and start it with shumai-cli to enable command execution.",
"runner_created_successfully": "Runner created successfully",
"runner_updated_successfully": "Runner updated successfully",
"runner_deleted_successfully": "Runner deleted successfully",
@@ -1743,8 +1743,8 @@
"failed_to_delete_runner": "Failed to delete runner",
"failed_to_rotate_runner_key": "Failed to rotate runner key",
"runner_copied_to_clipboard": "Copied to clipboard",
"agent_runner_label": "Bash Runner",
"agent_runner_description": "Select which runner executes bash commands and downloads workspace assets for this agent.",
"agent_runner_label": "Command Runner",
"agent_runner_description": "Select which runner executes shell commands and downloads workspace assets for this agent.",
"agent_runner_team_default": "Team Default Runner",
"agent_runner_team_default_with_name": "Team Default ({name})",
"agent_runner_none_configured_warning": "No runners are configured for this team yet. The bash and download_asset tools will remain unavailable until you register a runner in Team Settings → Agent Runners."
+48 -1
View File
@@ -1630,5 +1630,52 @@
"filter_archived_only": "보관됨",
"filter_not_archived": "보관되지 않음",
"task_type_archive_asset": "에셋 보관",
"task_type_restore_asset": "에셋 복원"
"task_type_restore_asset": "에셋 복원",
"agent_runners": "에이전트 러너",
"agent_runners_settings": "에이전트 러너 설정",
"runners_description": "원격 러너를 등록하고 관리합니다. 러너에 셸 접근 권한이 있는 사용자는 다른 프로젝트의 파일을 포함해 해당 워크스페이스의 모든 파일을 읽을 수 있습니다.",
"runners": "러너",
"create_runner": "러너 생성",
"create_runner_description": "팀용 러너를 등록하고 shumai-cli에 사용할 일회성 키를 발급받습니다. 워크스페이스의 모든 파일을 읽어도 되는 사용자와만 러너를 공유하세요.",
"runner_name": "러너 이름",
"runner_name_placeholder": "예: Mac Mini 러너",
"runner_description": "설명(선택 사항)",
"runner_description_placeholder": "예: 영상 처리용 로컬 macOS 러너",
"runner_set_as_default": "팀 기본 러너로 설정",
"runner_default_badge": "기본",
"runner_status_connected": "온라인",
"runner_status_disconnected": "오프라인",
"runner_key_prefix_label": "키 접두사",
"runner_last_seen_label": "마지막 연결: {time}",
"runner_never_connected": "연결 기록 없음",
"runner_set_default_action": "기본으로 설정",
"runner_rotate_key_action": "키 순환",
"runner_edit_action": "러너 편집",
"runner_edit_description": "이 러너의 이름 또는 설명을 업데이트합니다.",
"runner_delete_confirm_title": "러너 삭제",
"runner_delete_confirm_description": "\"{name}\" 러너를 삭제하시겠습니까? 활성 연결은 즉시 종료되며, 이 러너에 연결된 에이전트는 팀 기본 러너로 전환됩니다.",
"runner_rotate_key_confirm_title": "러너 키 순환",
"runner_rotate_key_confirm_description": "\"{name}\"의 키를 순환하면 현재 키가 즉시 무효화되고 활성 세션이 연결 해제됩니다. 새 키로 러너를 다시 시작해야 합니다.",
"runner_key_dialog_title": "러너 키 저장",
"runner_key_dialog_description": "지금 이 러너 키를 복사하세요. 보안을 위해 다시 표시되지 않습니다.",
"runner_key_label": "러너 키",
"runner_quickstart_command_label": "빠른 시작 명령",
"runner_quickstart_sandbox_label": "샌드박스로 시작(선택 사항)",
"runner_empty_title": "구성된 러너 없음",
"runner_empty_description": "구성된 러너가 없으면 에이전트에서 bash 및 download_asset 도구가 자동으로 숨겨집니다. 명령 실행을 활성화하려면 러너를 생성하고 shumai-cli로 시작하세요.",
"runner_created_successfully": "러너가 생성되었습니다",
"runner_updated_successfully": "러너가 업데이트되었습니다",
"runner_deleted_successfully": "러너가 삭제되었습니다",
"runner_key_rotated_successfully": "러너 키가 순환되었습니다",
"failed_to_load_runners": "러너를 불러오지 못했습니다",
"failed_to_create_runner": "러너를 생성하지 못했습니다",
"failed_to_update_runner": "러너를 업데이트하지 못했습니다",
"failed_to_delete_runner": "러너를 삭제하지 못했습니다",
"failed_to_rotate_runner_key": "러너 키를 순환하지 못했습니다",
"runner_copied_to_clipboard": "클립보드에 복사되었습니다",
"agent_runner_label": "명령 러너",
"agent_runner_description": "이 에이전트의 셸 명령을 실행하고 워크스페이스 에셋을 다운로드할 러너를 선택합니다.",
"agent_runner_team_default": "팀 기본 러너",
"agent_runner_team_default_with_name": "팀 기본({name})",
"agent_runner_none_configured_warning": "이 팀에 아직 구성된 러너가 없습니다. 팀 설정 → 에이전트 러너에서 러너를 등록할 때까지 bash 및 download_asset 도구를 사용할 수 없습니다."
}
+5 -5
View File
@@ -1703,10 +1703,10 @@
"task_type_restore_asset": "恢复归档资产",
"agent_runners": "Agent 运行器",
"agent_runners_settings": "Agent 运行器设置",
"runners_description": "注册并管理为 Agent 执行 Bash 命令和技能的远程运行器。",
"runners_description": "注册并管理远程运行器。有权在同一运行器执行命令的用户可以读取其工作区中的文件,包括其他项目留下的文件。",
"runners": "运行器",
"create_runner": "创建运行器",
"create_runner_description": "为团队注册一个新的运行器。创建后将生成一次性运行器密钥,用于通过 shumai-cli 连接。",
"create_runner_description": "为团队注册运行器并获取用于 shumai-cli 的一次性密钥。请仅在有权读取工作区内所有文件的用户之间共享运行器。",
"runner_name": "运行器名称",
"runner_name_placeholder": "例如:Mac Mini 运行器",
"runner_description": "描述(可选)",
@@ -1732,7 +1732,7 @@
"runner_quickstart_command_label": "快速启动命令",
"runner_quickstart_sandbox_label": "启用沙箱启动(可选)",
"runner_empty_title": "尚未配置运行器",
"runner_empty_description": "未配置运行器时,Agent 的 bash 和 download_asset 工具将自动隐藏。创建一个运行器并通过 shumai-cli 启动即可启用 Bash 执行能力。",
"runner_empty_description": "未配置运行器时,Agent 的 bash 和 download_asset 工具将自动隐藏。创建一个运行器并通过 shumai-cli 启动即可启用命令执行能力。",
"runner_created_successfully": "运行器创建成功",
"runner_updated_successfully": "运行器更新成功",
"runner_deleted_successfully": "运行器删除成功",
@@ -1743,8 +1743,8 @@
"failed_to_delete_runner": "删除运行器失败",
"failed_to_rotate_runner_key": "轮换运行器密钥失败",
"runner_copied_to_clipboard": "已复制到剪贴板",
"agent_runner_label": "Bash 运行器",
"agent_runner_description": "选择为该 Agent 执行 Bash 命令和下载工作空间资产的运行器。",
"agent_runner_label": "命令运行器",
"agent_runner_description": "选择为该 Agent 执行命令和下载工作空间资产的运行器。",
"agent_runner_team_default": "团队默认运行器",
"agent_runner_team_default_with_name": "团队默认({name})",
"agent_runner_none_configured_warning": "当前团队尚未配置任何运行器。在“团队设置 → Agent 运行器”中注册运行器之前,bash 和 download_asset 工具将不可用。"
+4
View File
@@ -6,6 +6,7 @@ import { parseArgs } from 'node:util'
import { fileURLToPath } from 'node:url'
import { version } from '../package.json'
import { version as cliVersion } from '../apps/cli/package.json'
import { srtBundlePlugin } from './srt-bundle'
export const cliTargets = {
'darwin-arm64': 'bun-darwin-arm64',
@@ -80,6 +81,7 @@ export async function assembleCliRelease(outdir: string, targets = cliTargetName
export async function buildCliRelease(outdir: string, targets = cliTargetNames) {
validateCliVersion(version)
if (cliVersion !== version) throw new Error('Root and CLI package versions must match')
const notices = await readFile(join(repoRoot, 'apps/cli/THIRD-PARTY-NOTICES.txt'), 'utf8')
await mkdir(outdir, { recursive: true })
const staging = await mkdtemp(join(outdir, '.build-'))
try {
@@ -90,6 +92,8 @@ export async function buildCliRelease(outdir: string, targets = cliTargetNames)
entrypoints: [join(repoRoot, 'apps/cli/src/index.ts')],
target: 'bun',
minify: true,
banner: '/*\n' + notices + '*/',
plugins: [srtBundlePlugin(...target.split('-'))],
compile: {
target: cliTargets[target],
outfile: binary,
+4
View File
@@ -2,6 +2,7 @@ import tailwindPlugin from 'bun-plugin-tailwind'
import { existsSync, readFileSync, readdirSync } from 'node:fs'
import { cp, mkdir, readdir, rm, writeFile } from 'node:fs/promises'
import path from 'node:path'
import { srtBundlePlugin } from './srt-bundle'
console.log('\n🚀 Starting NPM packaging build process for all applications...\n')
@@ -520,7 +521,10 @@ async function buildCliPackage() {
target: 'node',
outdir: binOutDir,
minify: true,
banner:
'/*\n' + readFileSync(path.join(rootDir, 'apps/cli/THIRD-PARTY-NOTICES.txt'), 'utf8') + '*/',
naming: 'shumai-cli.js',
plugins: [srtBundlePlugin()],
external: [],
define: {
'process.env.NODE_ENV': JSON.stringify('production'),
+36
View File
@@ -0,0 +1,36 @@
import { readFile } from 'node:fs/promises'
import { createRequire } from 'node:module'
import path from 'node:path'
/** Keep SRT's native helpers available in both standalone binaries and the Node bundle. */
export function srtBundlePlugin(platform?: string, arch?: string): Bun.BunPlugin {
return {
name: 'srt-vendor',
setup(build) {
build.onLoad({ filter: /[/\\]runner[/\\]srt-vendor\.ts$/ }, async () => {
const packageJson = createRequire(
new URL('../apps/cli/package.json', import.meta.url),
).resolve('@anthropic-ai/sandbox-runtime/package.json')
const names = ['java-proxy-agent/srt-proxy-agent.jar']
for (const cpu of arch ? [arch] : ['x64', 'arm64']) {
if (!platform || platform === 'linux') names.push(`seccomp/${cpu}/apply-seccomp`)
if (!platform || platform === 'windows') names.push(`srt-win/${cpu}/srt-win.exe`)
}
const assets = Object.fromEntries(
await Promise.all(
names.map(async (name) => [
name,
(await readFile(path.join(path.dirname(packageJson), 'vendor', name))).toString(
'base64',
),
]),
),
)
return {
contents: `export const bundledSrtVendor = ${JSON.stringify(assets)}`,
loader: 'js',
}
})
},
}
}
+109 -5
View File
@@ -2,10 +2,11 @@ import assert from 'node:assert/strict'
import { execFile } from 'node:child_process'
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join, resolve } from 'node:path'
import { basename, join, resolve } from 'node:path'
import { promisify, parseArgs } from 'node:util'
import { randomBytes } from 'node:crypto'
import { version } from '../package.json'
import { runnerToServerMessageSchema } from '../packages/dtos/src/runner'
import { archiveName, packCliBinary, parseCliTarget, sha256 } from './build-cli'
const execFileAsync = promisify(execFile)
@@ -14,8 +15,105 @@ function assertRecord(value: unknown): asserts value is Record<string, unknown>
assert.ok(value && typeof value === 'object' && !Array.isArray(value))
}
async function testRunnerShell(binary: string, cwd: string, env: NodeJS.ProcessEnv, srt?: string) {
const key = 'fixture-runner-key'
const executionId = srt ? 'srt-shell-quotes' : 'shell-quotes'
let output = ''
let completed = false
let fixtureFailure: unknown
await writeFile(join(cwd, 'runner input.txt'), 'input with spaces\n')
const server = Bun.serve({
hostname: '127.0.0.1',
port: 0,
fetch(request, server) {
if (new URL(request.url).pathname !== '/api/runners/ws')
return new Response(null, { status: 404 })
if (request.headers.get('authorization') !== 'Bearer ' + key)
return new Response(null, { status: 401 })
if (!server.upgrade(request)) return new Response(null, { status: 426 })
},
websocket: {
open(socket) {
socket.send(
JSON.stringify({
type: 'welcome',
runnerId: 'runner-test',
runnerName: 'Native CLI',
serverTime: Date.now(),
inFlightExecutionIds: [],
}),
)
},
message(socket, raw) {
try {
const message = runnerToServerMessageSchema.parse(JSON.parse(raw.toString()))
if (message.type === 'reconcile') {
assert.equal(message.environment.platform, process.platform)
const shell = basename(message.environment.shell).toLowerCase()
assert.equal(shell, process.platform === 'win32' ? 'bash.exe' : 'sh')
socket.send(
JSON.stringify({
type: 'exec.start',
executionId,
command:
'IFS= read -r line < "runner input.txt"\nprintf \'%s | %s\' "$line" \'$literal "quotes"\'',
timeoutSec: 15,
}),
)
} else if (message.type === 'exec.output') {
assert.equal(message.executionId, executionId)
assert.equal(message.stream, 'stdout')
output += message.data
} else if (message.type === 'exec.result') {
assert.equal(message.executionId, executionId)
assert.equal(message.exitCode, 0, JSON.stringify(message))
assert.equal(output, 'input with spaces | $literal "quotes"')
completed = true
socket.close(1008)
}
} catch (error) {
fixtureFailure ??= error
socket.close(1008)
}
},
},
})
try {
const child = Bun.spawn(
[
binary,
'runner',
'start',
'--cwd',
cwd,
'--url',
server.url.origin,
...(srt ? ['--srt', srt] : []),
],
{
env: {
...env,
...(process.platform === 'win32' || srt ? { PATH: process.env.PATH } : {}),
SHUMAI_RUNNER_KEY: key,
},
signal: AbortSignal.timeout(60_000),
stderr: 'pipe',
stdout: 'ignore',
},
)
const [exitCode, stderr] = await Promise.all([child.exited, new Response(child.stderr).text()])
if (fixtureFailure) throw fixtureFailure
assert.ok(completed, 'Runner did not complete the shell command:\n' + stderr)
assert.equal(exitCode, 1, stderr)
assert.match(stderr, /Runner shell: .+/)
assert.match(stderr, /Runner access was revoked/)
} finally {
server.stop(true)
}
}
/** Exercises the packaged installer and executable against HTTP, without a runtime on PATH. */
export async function testCliRelease(outdir: string) {
export async function testCliRelease(outdir: string, srt?: string) {
const windows = process.platform === 'win32'
const target = parseCliTarget(`${windows ? 'windows' : process.platform}-${process.arch}`)
const bytes = randomBytes(65536)
@@ -200,6 +298,10 @@ export async function testCliRelease(outdir: string) {
await install()
assert.equal((await run(['--version'])).trim(), version)
assert.match(await run(['--help']), /Command line interface/)
const workspace = join(temporary, 'runner workspace')
await mkdir(workspace)
await testRunnerShell(binary, workspace, cliEnv)
if (srt) await testRunnerShell(binary, workspace, cliEnv, srt)
const projects = JSON.parse(await run(['--json', 'project', 'ls'])) as {
data: { id: string }[]
}
@@ -273,7 +375,7 @@ export async function testCliRelease(outdir: string) {
await assert.rejects(install(), /unmanaged installation/)
assert.equal(sha256(await readFile(binary)), installedHash)
console.log(
`PASS ${target}: install, API auth, mkdir, upload/download 65536 bytes, upgrade/rollback, and failure preservation`,
`PASS ${target}: install, runner shell, API auth, mkdir, upload/download 65536 bytes, upgrade/rollback, and failure preservation`,
)
} finally {
server.stop(true)
@@ -282,6 +384,8 @@ export async function testCliRelease(outdir: string) {
}
if (import.meta.main) {
const { values } = parseArgs({ options: { out: { type: 'string', default: 'out/cli-release' } } })
await testCliRelease(resolve(values.out))
const { values } = parseArgs({
options: { out: { type: 'string', default: 'out/cli-release' }, srt: { type: 'string' } },
})
await testCliRelease(resolve(values.out), values.srt ? resolve(values.srt) : undefined)
}