Compare commits

...
14 Commits
Author SHA1 Message Date
yiling 3737418ec3 Release v0.1.2 2026-06-25 19:43:07 +08:00
Yiling-J 6d47ea7466 Update AWS_ENDPOINT_URL_S3 explanation in README
Clarified the default value for AWS_ENDPOINT_URL_S3 and its usage with custom host/port.
2026-06-25 19:15:11 +08:00
Yiling-J 84d96ff83a Revise README for environment variable setup
Updated environment variable configuration instructions for clarity and added optional settings for S3-compatible storage.
2026-06-25 19:11:39 +08:00
Yiling-J 6e985aed61 Merge pull request #155 from shumaiOne/refactor/s3-endpoint
refactor(s3): remove automatic port appending to AWS_ENDPOINT_URL_S3
2026-06-25 18:56:49 +08:00
yiling 8347a9ca15 docs(docker): document SHUMAI_SERVER_PORT and update local docker-compose env 2026-06-25 18:50:29 +08:00
yiling ab9b33320d docs(s3): document SHUMAI_SERVER_PORT and S3 endpoint port configurations 2026-06-25 18:48:23 +08:00
yiling efd6163238 refactor(s3): remove automatic port appending to AWS_ENDPOINT_URL_S3 2026-06-25 18:46:14 +08:00
yiling 0e4bdec0e3 Release v0.1.1 2026-06-23 10:32:41 +08:00
yiling 0eb03b16a4 chore(docker): add AWS_ENDPOINT_URL_S3 to local docker-compose 2026-06-23 10:27:18 +08:00
Yiling-J b216773044 Merge pull request #151 from shumaiOne/docker/entrypoint-privilege-dropping
feat(docker): add root entrypoint with privilege dropping for data volumes
2026-06-23 10:09:57 +08:00
yiling 92801ba14a fix(core): make version stack sort index lookup deterministic
When creating a version stack, the sortIndex was previously extracted
by taking files[0] from the database query result. Because findMany
without orderBy is non-deterministic, this resulted in flakiness
where the sortIndex would sometimes be null (from file2) instead of
'abc' (from file1), causing tests to fail.

We now look up the first file specifically based on the ordered
fileIds array parameter.
2026-06-23 10:02:01 +08:00
yiling 661e836f06 fix(docker): support non-root/restricted container environments in entrypoint
Checks if the container is running as root (UID 0) before executing
chown and gosu. If running directly as a non-root user (e.g. in hardened
environments like Kubernetes runAsNonRoot or via docker run --user),
it directly execs the command, preventing EACCES and permission errors.
2026-06-23 09:56:31 +08:00
yiling fb5c7ecabe chore(docker): ignore dist and data directories in .dockerignore
This adds the `dist` and `data` folders to `.dockerignore` to prevent
them from being transferred as part of the Docker build context.
This reduces the build context size by over 1.5GB+, making the build
much faster.
2026-06-23 09:53:00 +08:00
yiling a390643458 feat(docker): add root entrypoint with privilege dropping for data volumes
This updates the Dockerfile and adds a scripts/docker-entrypoint.sh script.
Under native Linux (like Debian), when docker-compose mounts a host folder
as a volume (e.g. `./data:/app/data`), and the folder doesn't exist, the
docker daemon creates the folder as root-owned (`root:root`).

Because the container runs as a non-root user `bun`, it gets permission
denied (`EACCES`) when trying to write to the data directory.

To resolve this in a beginner-friendly way without manual host-side
permissions updates:
1. The runner stage now installs `gosu`.
2. At the end of the Dockerfile, the user switches back to `root` so the
   container launches as root.
3. On launch, the entrypoint script executes. Since it runs as root, it
   automatically changes ownership of `/app/data` to `bun:bun`.
4. The entrypoint script then uses `gosu` to drop privileges and launch the
   actual server processes as the non-root `bun` user.
2026-06-23 09:46:56 +08:00
25 changed files with 140 additions and 35 deletions
+2
View File
@@ -1,4 +1,6 @@
node_modules
dist
data
Dockerfile*
docker-compose*
.dockerignore
+8
View File
@@ -47,6 +47,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
socat \
ca-certificates \
bubblewrap \
gosu \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
@@ -69,5 +70,12 @@ COPY --chown=bun:bun --from=builder /app/prisma.config.ts.prod ./prisma.config.t
# Expose port (Bun Hono defaults to 3000)
EXPOSE 3000
# Switch back to root at the end so the container runs the entrypoint as root
USER root
COPY scripts/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
ENTRYPOINT ["docker-entrypoint.sh"]
# Default command: run migrations and start API server
CMD ["sh", "-c", "bun prisma migrate deploy && bun index.js"]
+26 -6
View File
@@ -49,13 +49,32 @@ Docker Compose is the fastest way to get Shumai running. You do not need to clon
```bash
curl -o docker-compose.yaml https://raw.githubusercontent.com/shumaiOne/shumai/main/docker-compose/local/docker-compose.yaml
```
3. **Configure Environment Variables (Remote Deployments only):**
If deploying Shumai to a remote server (e.g., AWS EC2, VPS), edit docker-compose.yaml and add AWS_ENDPOINT_URL_S3 under the environment section, setting it to your server's public IP address or domain name (e.g., http://12.34.56.78).
4. Start the services in detached mode:
3. Configure environment variables (optional):
* `SHUMAI_SERVER_PORT` controls the port the Shumai server listens on. The default is `3000`.
* By default, Shumai uses the bundled local storage service in this Docker Compose setup. To use an external S3-compatible storage instead, set `AWS_ENDPOINT_URL_S3` to the provider's endpoint URL and configure the corresponding AWS credentials and bucket settings as needed.
```
AWS_ENDPOINT_URL_S3: S3_ENDPOINT_URL
S3_REGION: S3_REGION
S3_BUCKET: BUCKET_NAME
S3_ACCESS_KEY_ID: ID
S3_SECRET_ACCESS_KEY: KEY
```
* By default, `AWS_ENDPOINT_URL_S3` is set to: `http://localhost:{SHUMAI_SERVER_PORT}`, if you use local storage and expose Shumai on a custom host/port combination, set `AWS_ENDPOINT_URL_S3` to the external URL that browsers will use to upload files.
For example, if you change the port mapping in `docker-compose.yaml` from `3000:3000` to `12345:3000` and deploy on a server with IP address `12.34.56.78`, set:
```bash
AWS_ENDPOINT_URL_S3=http://12.34.56.78:12345
```
This value must be reachable from client browsers and should include the externally exposed port.
5. Start the services in detached mode:
```bash
docker compose up -d
```
5. Open your browser and access Shumai at `http://localhost:3000` (or `http://<your-server-ip>:3000` for remote deployments).
6. Open your browser and access Shumai at `http://localhost:3000` (or `http://<your-server-ip>:3000` for remote deployments).
### Option 2: Install via NPM / Package Manager
@@ -126,11 +145,12 @@ DATABASE_URL=postgresql://shumai:shumai_password@localhost:5432/shumai_db?schema
BETTER_AUTH_SECRET=ySxs7DxzHDZBbeeHNPEwBuspYwipBqz5Gk5XdBjNhWw=
STORAGE_BACKEND=local
SHUMAI_SERVER_PORT=3000
AWS_ENDPOINT_URL_S3=http://localhost
AWS_ENDPOINT_URL_S3=http://localhost:3000
```
> [!NOTE]
> If deploying on a remote server, update `AWS_ENDPOINT_URL_S3` from `http://localhost` to your server's public IP address or domain name.
> `SHUMAI_SERVER_PORT` sets the port the server starts on, while `AWS_ENDPOINT_URL_S3` is used by the browser to build file upload URLs.
> If deploying on a remote server (e.g. `http://123.456.7.8`) with a mapped port (e.g. `12345:3000` in docker-compose), set `AWS_ENDPOINT_URL_S3` to `http://123.456.7.8:12345`.
#### Step 6: Run Shumai
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent-app",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/cli",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/transcode-app",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/web-app",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"dependencies": {
+4
View File
@@ -27,8 +27,12 @@ services:
# Database Configuration
DATABASE_URL: postgres://shumai:shumai_password@shumai_postgres:5432/shumai_db?search_path=public&sslmode=disable
# Server Configuration
SHUMAI_SERVER_PORT: 3000
# S3 Configuration
STORAGE_BACKEND: local
AWS_ENDPOINT_URL_S3: http://localhost:3000
# Better Auth
BETTER_AUTH_SECRET: qSxs9DxzHDZBbeeTNPEwBuspYwipBqz5Gk5XdBjNhWw=
+22 -10
View File
@@ -37,16 +37,28 @@ Shumai is configured via environment variables. You can define these variables i
Shumai supports storing files locally or on S3-compatible endpoints.
| Variable | Description | Example / Default |
| :------------------------- | :------------------------------------------------------------------------------------------------ | :--------------------------- |
| `STORAGE_BACKEND` | Storage type. Use `local` for local file system storage, or `s3` for S3-compatible cloud storage. | `local` (default) or `s3` |
| `AWS_ENDPOINT_URL_S3` | Custom endpoint for S3 storage. Must change to public domain/IP if deploying remotely. | `http://localhost` (default) |
| `S3_BUCKET` | S3 bucket name. | `shumai` (default) |
| `S3_ACCESS_KEY_ID` | Access key credentials for S3. | `minioadmin` (default) |
| `S3_SECRET_ACCESS_KEY` | Secret key credentials for S3. | `minioadmin` (default) |
| `S3_REGION` | S3 storage region. | `auto` (default) |
| `S3_USE_SSL` | Set to `true` to use HTTPS for S3 endpoints. | `false` (default) |
| `PRESIGNED_URL_EXPIRES_IN` | Number of hours for which generated presigned download/upload links are valid. | `5` (default) |
| Variable | Description | Example / Default |
| :------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------------ |
| `STORAGE_BACKEND` | Storage type. Use `local` for local file system storage, or `s3` for S3-compatible cloud storage. | `local` (default) or `s3` |
| `AWS_ENDPOINT_URL_S3` | Endpoint for S3 storage or local storage. Used by the browser to build file upload URLs. Must include port if deploying remotely or using custom ports. | `http://localhost:3000` (respects `SHUMAI_SERVER_PORT` fallback if unset) |
| `S3_BUCKET` | S3 bucket name. | `shumai` (default) |
| `S3_ACCESS_KEY_ID` | Access key credentials for S3. | `minioadmin` (default) |
| `S3_SECRET_ACCESS_KEY` | Secret key credentials for S3. | `minioadmin` (default) |
| `S3_REGION` | S3 storage region. | `auto` (default) |
| `S3_USE_SSL` | Set to `true` to use HTTPS for S3 endpoints. | `false` (default) |
| `PRESIGNED_URL_EXPIRES_IN` | Number of hours for which generated presigned download/upload links are valid. | `5` (default) |
### Local Storage Port Configuration
When `STORAGE_BACKEND` is set to `local` (default):
- `SHUMAI_SERVER_PORT` configures the port on which the Shumai web server starts and listens.
- `AWS_ENDPOINT_URL_S3` configures the public/external endpoint used by client browsers to upload and download files. It must include the scheme and port.
For example, if you deploy Shumai on a remote server at `http://123.456.7.8` and configure port mapping `12345:3000` (exposing port 3000 internally as port 12345 externally) in Docker Compose:
- Set `SHUMAI_SERVER_PORT` to `3000`.
- Set `AWS_ENDPOINT_URL_S3` to `http://123.456.7.8:12345`.
---
+8 -1
View File
@@ -61,16 +61,23 @@ To handle media processing and security isolation, the host machine must have th
```
3. **Configure Environment Variables**:
Create a new directory, navigate into it, and create a `.env` file:
```bash
mkdir shumai && cd shumai
```
Create a `.env` file with the following variables:
```env
DATABASE_URL=postgresql://shumai:shumai_password@localhost:5432/shumai_db?schema=public
BETTER_AUTH_SECRET=your-random-32-character-secret-key
STORAGE_BACKEND=local
AWS_ENDPOINT_URL_S3=http://localhost
AWS_ENDPOINT_URL_S3=http://localhost:3000
```
> [!TIP]
> `AWS_ENDPOINT_URL_S3` must include the scheme and port (e.g. `http://localhost:3000`) where Shumai is publicly accessible. This URL is used directly by client browsers for file uploads.
4. **Start Shumai**:
> [!IMPORTANT]
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "shumai",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"repository": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/agent",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/api",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/core",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
+35
View File
@@ -191,4 +191,39 @@ describe('S3Service implementations', () => {
await expect(localS3.presign('b', 'k', 'POST')).rejects.toThrow()
})
})
describe('s3Service initialization', () => {
let originalEnv: NodeJS.ProcessEnv
beforeEach(() => {
originalEnv = { ...process.env }
})
afterEach(() => {
process.env = originalEnv
})
it('should initialize LocalStorageService with default localhost and SHUMAI_SERVER_PORT when AWS_ENDPOINT_URL_S3 is not set', async () => {
process.env.STORAGE_BACKEND = 'local'
delete process.env.AWS_ENDPOINT_URL_S3
process.env.SHUMAI_SERVER_PORT = '4567'
vi.resetModules()
const { s3Service } = await import('./s3')
const url = await s3Service.presign('bucket', 'key', 'GET')
expect(url).toContain('http://localhost:4567')
})
it('should initialize LocalStorageService with exact AWS_ENDPOINT_URL_S3 when it is set', async () => {
process.env.STORAGE_BACKEND = 'local'
process.env.AWS_ENDPOINT_URL_S3 = 'http://123.456.7.8:12345'
process.env.SHUMAI_SERVER_PORT = '4567'
vi.resetModules()
const { s3Service } = await import('./s3')
const url = await s3Service.presign('bucket', 'key', 'GET')
expect(url).toContain('http://123.456.7.8:12345')
expect(url).not.toContain('4567')
})
})
})
+2 -2
View File
@@ -452,7 +452,7 @@ export class LocalStorageService implements S3Service {
const storageBackend = process.env.STORAGE_BACKEND || 'local'
const port = process.env.SHUMAI_SERVER_PORT || '3000'
const s3Endpoint = process.env.AWS_ENDPOINT_URL_S3 || 'http://localhost'
const s3Endpoint = process.env.AWS_ENDPOINT_URL_S3 || `http://localhost:${port}`
export const s3Service: S3Service =
storageBackend === 's3'
@@ -464,4 +464,4 @@ export const s3Service: S3Service =
process.env.S3_USE_SSL === 'true',
process.env.S3_REGION || 'auto',
)
: new LocalStorageService(`${s3Endpoint}:${port}`)
: new LocalStorageService(s3Endpoint)
@@ -58,7 +58,7 @@ export class VersionStackService {
throw new Error('parent not found')
}
const firstFile = files[0]
const firstFile = files.find((f) => f.id === fileIds[0])
const sortIndex = firstFile?.sortIndex || null
const stack = await tx.asset.create({
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/db",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/dtos",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/tableprinter",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/transcode",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/webui",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"main": "./index.html",
+1 -1
View File
@@ -1 +1 @@
export const APP_VERSION = '0.1.0'
export const APP_VERSION = '0.1.2'
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@shumai/workflow-core",
"version": "0.1.0",
"version": "0.1.2",
"private": true,
"type": "module",
"main": "./src/index.ts",
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
set -e
if [ "$(id -u)" = "0" ]; then
# Check if /app/data exists and is not owned by the bun user (UID 1000)
if [ -d "/app/data" ]; then
if [ "$(stat -c '%u' /app/data)" != "1000" ]; then
echo "Correcting permissions on /app/data..."
chown -R bun:bun /app/data
fi
fi
# Run the CMD as the bun user
exec gosu bun "$@"
else
# Run the CMD directly if already running as non-root
exec "$@"
fi