diff --git a/CHANGELOG.md b/CHANGELOG.md index 932e29a4..2bc9c3fe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Added +- **transcode**: Add dcraw_emu fallback to generate WebP previews for camera RAW files without embedded JPEG previews + ### Fixed ### Changed diff --git a/Dockerfile b/Dockerfile index 1db5b90d..37c5f13b 100644 --- a/Dockerfile +++ b/Dockerfile @@ -52,6 +52,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ poppler-utils \ imagemagick \ fonts-noto-cjk \ + libraw-bin \ && . /etc/os-release \ && TARGETARCH=$(dpkg --print-architecture) \ && FFMPEG_VERSION="8.1.2-5" \ diff --git a/README.md b/README.md index fcbf6b7d..acbbfa50 100644 --- a/README.md +++ b/README.md @@ -162,23 +162,24 @@ Shumai requires the following system packages on Linux: - **`ffmpeg`** – Used for media transcoding and metadata extraction. - **`poppler`** (`poppler-utils`) – Used for PDF page image extraction and PDF sprite preview generation (`pdftoppm`). - **`imagemagick`** – Used for PSD format transcoding and color profile conversion to sRGB. +- **`dcraw_emu`** (`libraw` / `libraw-bin`) – Used for camera RAW format image decoding and preview generation. - **`bubblewrap`**, **`socat`**, and **`ripgrep`** – Required by the AI agent sandbox (`anthropic-experimental/sandbox-runtime`) for process isolation, networking, and workspace search. Install all required packages with one command: > Ubuntu/Debian ```bash -sudo apt install -y ffmpeg poppler-utils imagemagick bubblewrap socat ripgrep +sudo apt install -y ffmpeg poppler-utils imagemagick bubblewrap socat ripgrep libraw-bin ``` > Fedora ```bash -sudo dnf install -y ffmpeg poppler-utils ImageMagick bubblewrap socat ripgrep +sudo dnf install -y ffmpeg poppler-utils ImageMagick bubblewrap socat ripgrep LibRaw ``` > Arch Linux ```bash -sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep +sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep libraw ``` > [!NOTE] @@ -201,12 +202,13 @@ Shumai requires: - **`ffmpeg`** – Used for media transcoding and metadata extraction. - **`poppler`** – Used for PDF page image extraction and PDF sprite preview generation (`pdftoppm`). - **`imagemagick`** – Used for PSD format transcoding and color profile conversion to sRGB. +- **`dcraw_emu`** (`libraw`) – Used for camera RAW format image decoding and preview generation. - **`ripgrep`** – Required by the AI agent sandbox (`anthropic-experimental/sandbox-runtime`). Install the required packages with Homebrew: ```bash -brew install ffmpeg poppler imagemagick ripgrep +brew install ffmpeg poppler imagemagick ripgrep libraw ``` --- diff --git a/README_CN.md b/README_CN.md index de0b966a..5ce3ee10 100644 --- a/README_CN.md +++ b/README_CN.md @@ -187,23 +187,24 @@ mkdir shumai && cd shumai - **`ffmpeg`** —— 用于媒体转码和元数据提取。 - **`poppler`** (`poppler-utils`) —— 用于 PDF 页面图像提取和 PDF 雪碧图预览生成 (`pdftoppm`)。 - **`imagemagick`** —— 用于 PSD 格式图像转码和 sRGB 色彩空间转换。 +- **`dcraw_emu`** (`libraw` / `libraw-bin`) —— 用于相机 RAW 格式图像解码和预览图生成。 - **`bubblewrap`**、**`socat`** 和 **`ripgrep`** —— AI Agent 沙箱(`anthropic-experimental/sandbox-runtime`)所需,用于进程隔离、网络通信和工作区搜索。 可以使用以下命令一次性安装所有依赖: > Ubuntu/Debian ```bash -sudo apt install -y ffmpeg poppler-utils imagemagick bubblewrap socat ripgrep +sudo apt install -y ffmpeg poppler-utils imagemagick bubblewrap socat ripgrep libraw-bin ``` > Fedora ```bash -sudo dnf install -y ffmpeg poppler-utils ImageMagick bubblewrap socat ripgrep +sudo dnf install -y ffmpeg poppler-utils ImageMagick bubblewrap socat ripgrep LibRaw ``` > Arch Linux ```bash -sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep +sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep libraw ``` > [!NOTE] @@ -215,7 +216,7 @@ sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep > sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 > ``` > -> 或者,配置 AppArmor Profile,为相关可执行文件授予所需的 `userns` 权限。 +> 或者,配置 AppArmor Profile,为相关可执行文件授予所需的 `userns`权限。 --- @@ -226,12 +227,13 @@ sudo pacman -S --noconfirm ffmpeg poppler imagemagick bubblewrap socat ripgrep - **`ffmpeg`** —— 用于媒体转码和元数据提取。 - **`poppler`** —— 用于 PDF 页面图像提取和 PDF 雪碧图预览生成 (`pdftoppm`)。 - **`imagemagick`** —— 用于 PSD 格式图像转码和 sRGB 色彩空间转换。 +- **`dcraw_emu`** (`libraw`) —— 用于相机 RAW 格式图像解码和预览图生成。 - **`ripgrep`** —— AI Agent 沙箱(`anthropic-experimental/sandbox-runtime`)所需。 使用 Homebrew 安装所需依赖: ```bash -brew install ffmpeg poppler imagemagick ripgrep +brew install ffmpeg poppler imagemagick ripgrep libraw ``` --- diff --git a/docs/getting-started/quickstart.mdx b/docs/getting-started/quickstart.mdx index 1cbad909..71e79104 100644 --- a/docs/getting-started/quickstart.mdx +++ b/docs/getting-started/quickstart.mdx @@ -44,6 +44,9 @@ To handle media processing and security isolation, the host machine must have th - **FFmpeg & FFprobe**: Required for video transcoding and metadata extraction. - _macOS_: `brew install ffmpeg` - _Linux (Ubuntu/Debian)_: `sudo apt-get install ffmpeg` +- **dcraw_emu (LibRaw)**: Required for camera RAW image decoding and preview generation. + - _macOS_: `brew install libraw` + - _Linux (Ubuntu/Debian)_: `sudo apt-get install libraw-bin` - **Agent Sandbox Utilities**: Used to run custom automation scripts securely. - _macOS_: `brew install ripgrep` - _Linux (Ubuntu/Debian)_: `sudo apt-get install bubblewrap socat ripgrep` diff --git a/packages/core/src/transcode/raw-extract.test.ts b/packages/core/src/transcode/raw-extract.test.ts index 36112da0..48c5bbd0 100644 --- a/packages/core/src/transcode/raw-extract.test.ts +++ b/packages/core/src/transcode/raw-extract.test.ts @@ -3,6 +3,8 @@ import * as fs from 'fs' import { extractEmbeddedJpeg, validateExtractedJpeg, + validateExtractedImage, + decodeRawWithDcraw, extractAndValidateRawPreview, EXIF_ORIENTATION_TO_ROTATION, } from './raw-extract' @@ -26,13 +28,20 @@ vi.mock('sharp', () => { return { default: mockSharp } }) +// Mock child_process +vi.mock('child_process', () => ({ + execFile: vi.fn(), +})) + import { exiftool } from 'exiftool-vendored' import sharp from 'sharp' +import { execFile } from 'child_process' // eslint-disable-next-line @typescript-eslint/no-explicit-any const mockMetadata = (sharp as any)._mockMetadata as ReturnType const mockExtractBinaryTag = vi.mocked(exiftool.extractBinaryTag) const mockRead = vi.mocked(exiftool.read) +const mockExecFile = vi.mocked(execFile) describe('extractEmbeddedJpeg', () => { beforeEach(() => { @@ -202,16 +211,231 @@ describe('validateExtractedJpeg', () => { }) }) +describe('validateExtractedImage', () => { + it('is an alias for validateExtractedJpeg', () => { + expect(validateExtractedImage).toBe(validateExtractedJpeg) + }) +}) + +describe('decodeRawWithDcraw', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('decodes RAW using camera white balance, sRGB, and TIFF output without -h when longest dimension <= 8192', async () => { + mockRead.mockResolvedValueOnce({ + ImageWidth: 6000, + ImageHeight: 4000, + Orientation: 1, + } as never) + + let executedArgs: string[] = [] + mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => { + executedArgs = args as string[] + const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1] + fs.writeFileSync(tiffDest, 'tiff-data') + const callback = typeof _opts === 'function' ? _opts : cb + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(null, { stdout: '', stderr: '' }) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + + const result = await decodeRawWithDcraw('/path/to/photo.cr2') + + expect(result).not.toBeNull() + expect(fs.existsSync(result!.previewPath)).toBe(true) + expect(result!.orientation).toBeUndefined() + expect(result!.rawWidth).toBe(6000) + expect(result!.rawHeight).toBe(4000) + + expect(executedArgs).toContain('-w') + expect(executedArgs).toContain('-o') + expect(executedArgs).toContain('1') + expect(executedArgs).toContain('-T') + expect(executedArgs).toContain('-Z') + expect(executedArgs).not.toContain('-h') + expect(executedArgs[executedArgs.length - 1]).toBe('/path/to/photo.cr2') + + result!.cleanup() + expect(fs.existsSync(result!.previewPath)).toBe(false) + }) + + it('enables half-size (-h) when longest dimension > 8192px and computes upright raw dimensions', async () => { + mockRead.mockResolvedValueOnce({ + ImageWidth: 9504, + ImageHeight: 6336, + Orientation: 6, + } as never) + + let executedArgs: string[] = [] + mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => { + executedArgs = args as string[] + const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1] + fs.writeFileSync(tiffDest, 'tiff-data') + const callback = typeof _opts === 'function' ? _opts : cb + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(null, { stdout: '', stderr: '' }) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + + const result = await decodeRawWithDcraw('/path/to/photo.arw') + + expect(result).not.toBeNull() + expect(executedArgs).toContain('-h') + expect(result!.orientation).toBeUndefined() + // For orientation 6 (90 CW), upright raw dimensions are swapped + expect(result!.rawWidth).toBe(6336) + expect(result!.rawHeight).toBe(9504) + + result!.cleanup() + }) + + it('omits -h when longest dimension is exactly 8192px', async () => { + mockRead.mockResolvedValueOnce({ + ImageWidth: 8192, + ImageHeight: 5464, + Orientation: 1, + } as never) + + let executedArgs: string[] = [] + mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => { + executedArgs = args as string[] + const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1] + fs.writeFileSync(tiffDest, 'tiff-data') + const callback = typeof _opts === 'function' ? _opts : cb + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(null, { stdout: '', stderr: '' }) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + + const result = await decodeRawWithDcraw('/path/to/photo.nef') + + expect(result).not.toBeNull() + expect(executedArgs).not.toContain('-h') + result!.cleanup() + }) + + it('defaults to full size without -h when EXIF metadata cannot be read', async () => { + mockRead.mockRejectedValueOnce(new Error('Exif read failure')) + + let executedArgs: string[] = [] + mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => { + executedArgs = args as string[] + const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1] + fs.writeFileSync(tiffDest, 'tiff-data') + const callback = typeof _opts === 'function' ? _opts : cb + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(null, { stdout: '', stderr: '' }) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + + const result = await decodeRawWithDcraw('/path/to/photo.dng') + + expect(result).not.toBeNull() + expect(executedArgs).not.toContain('-h') + result!.cleanup() + }) + + it('returns null and cleans up when dcraw_emu exits with non-zero code', async () => { + mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never) + mockExecFile.mockImplementationOnce((_bin, _args, _opts, cb) => { + const callback = typeof _opts === 'function' ? _opts : cb + const err = new Error('Command failed: dcraw_emu') + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(err as any).code = 1 + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(err) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + + const result = await decodeRawWithDcraw('/path/to/photo.cr2') + + expect(result).toBeNull() + }) + + it('returns null and cleans up when dcraw_emu times out', async () => { + mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never) + mockExecFile.mockImplementationOnce((_bin, _args, _opts, cb) => { + const callback = typeof _opts === 'function' ? _opts : cb + const err = new Error('Process timed out') + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(err as any).code = 'ETIMEDOUT' + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(err) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + + const result = await decodeRawWithDcraw('/path/to/photo.cr2') + + expect(result).toBeNull() + }) + + it('returns null and cleans up when dcraw_emu is not found (ENOENT)', async () => { + mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never) + mockExecFile.mockImplementationOnce((_bin, _args, _opts, cb) => { + const callback = typeof _opts === 'function' ? _opts : cb + const err = new Error('spawn dcraw_emu ENOENT') + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(err as any).code = 'ENOENT' + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(err) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + + const result = await decodeRawWithDcraw('/path/to/photo.cr2') + + expect(result).toBeNull() + }) + + it('returns null and cleans up when output TIFF is missing or empty', async () => { + mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never) + mockExecFile.mockImplementationOnce((_bin, _args, _opts, cb) => { + const callback = typeof _opts === 'function' ? _opts : cb + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(null, { stdout: '', stderr: '' }) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + + const result = await decodeRawWithDcraw('/path/to/photo.cr2') + + expect(result).toBeNull() + }) +}) + describe('extractAndValidateRawPreview', () => { beforeEach(() => { vi.clearAllMocks() }) - it('returns previewPath, dimensions, orientation, and cleanup for valid RAW', async () => { + it('returns previewPath, dimensions, orientation, and cleanup for valid RAW using embedded JPEG first', async () => { mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => { fs.writeFileSync(dest, 'jpeg-data') }) - mockRead.mockResolvedValueOnce({ Orientation: 6 } as never) + mockRead.mockResolvedValueOnce({ Orientation: 6, ImageWidth: 4000, ImageHeight: 3000 } as never) mockMetadata.mockResolvedValueOnce({ width: 4000, height: 3000 }) const result = await extractAndValidateRawPreview('/path/to/photo.cr2') @@ -221,35 +445,133 @@ describe('extractAndValidateRawPreview', () => { expect(result!.width).toBe(4000) expect(result!.height).toBe(3000) expect(result!.orientation).toBe(6) + expect(mockExecFile).not.toHaveBeenCalled() result!.cleanup() expect(fs.existsSync(result!.previewPath)).toBe(false) }) - it('returns null when extraction fails', async () => { + it('falls back to dcraw_emu when embedded preview extraction fails', async () => { mockExtractBinaryTag .mockRejectedValueOnce(new Error('fail')) .mockRejectedValueOnce(new Error('fail')) .mockRejectedValueOnce(new Error('fail')) + mockRead.mockResolvedValueOnce({ + ImageWidth: 6000, + ImageHeight: 4000, + Orientation: 1, + } as never) + + mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => { + const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1] + fs.writeFileSync(tiffDest, 'tiff-data') + const callback = typeof _opts === 'function' ? _opts : cb + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(null, { stdout: '', stderr: '' }) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + + mockMetadata.mockResolvedValueOnce({ width: 6000, height: 4000 }) + + const result = await extractAndValidateRawPreview('/path/to/photo.cr2') + + expect(result).not.toBeNull() + expect(mockExecFile).toHaveBeenCalled() + expect(result!.width).toBe(6000) + expect(result!.height).toBe(4000) + expect(result!.rawWidth).toBe(6000) + expect(result!.rawHeight).toBe(4000) + expect(result!.orientation).toBeUndefined() + + result!.cleanup() + expect(fs.existsSync(result!.previewPath)).toBe(false) + }) + + it('falls back to dcraw_emu when extracted embedded JPEG is corrupt', async () => { + let createdJpegPath = '' + mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => { + createdJpegPath = dest + fs.writeFileSync(dest, 'corrupt-jpeg') + }) + mockRead.mockResolvedValueOnce({ Orientation: 1 } as never) + mockMetadata.mockRejectedValueOnce(new Error('Not a valid image')) + + // Fallback dcraw_emu + mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never) + mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => { + const tiffDest = (args as string[])[(args as string[]).indexOf('-Z') + 1] + fs.writeFileSync(tiffDest, 'tiff-data') + const callback = typeof _opts === 'function' ? _opts : cb + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(null, { stdout: '', stderr: '' }) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + mockMetadata.mockResolvedValueOnce({ width: 6000, height: 4000 }) + + const result = await extractAndValidateRawPreview('/path/to/photo.cr2') + + expect(result).not.toBeNull() + expect(fs.existsSync(createdJpegPath)).toBe(false) // Embedded jpeg was cleaned up + expect(fs.existsSync(result!.previewPath)).toBe(true) + + result!.cleanup() + expect(fs.existsSync(result!.previewPath)).toBe(false) + }) + + it('returns null when both embedded extraction and dcraw_emu fail', async () => { + mockExtractBinaryTag + .mockRejectedValueOnce(new Error('fail')) + .mockRejectedValueOnce(new Error('fail')) + .mockRejectedValueOnce(new Error('fail')) + + mockRead.mockRejectedValueOnce(new Error('fail')) + mockExecFile.mockImplementationOnce((_bin, _args, _opts, cb) => { + const callback = typeof _opts === 'function' ? _opts : cb + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(new Error('dcraw failed')) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any + }) + const result = await extractAndValidateRawPreview('/path/to/photo.cr2') expect(result).toBeNull() }) - it('returns null and cleans up when validation fails', async () => { - let createdPath = '' - mockExtractBinaryTag.mockImplementationOnce(async (_tag, _src, dest) => { - createdPath = dest - fs.writeFileSync(dest, 'bad-jpeg') + it('returns null and cleans up when decoded TIFF is corrupt', async () => { + mockExtractBinaryTag + .mockRejectedValueOnce(new Error('fail')) + .mockRejectedValueOnce(new Error('fail')) + .mockRejectedValueOnce(new Error('fail')) + + let createdTiff = '' + mockRead.mockResolvedValueOnce({ ImageWidth: 6000, ImageHeight: 4000 } as never) + mockExecFile.mockImplementationOnce((_bin, args, _opts, cb) => { + createdTiff = (args as string[])[(args as string[]).indexOf('-Z') + 1] + fs.writeFileSync(createdTiff, 'corrupt-tiff') + const callback = typeof _opts === 'function' ? _opts : cb + if (callback) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ;(callback as any)(null, { stdout: '', stderr: '' }) + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + return {} as any }) - mockRead.mockResolvedValueOnce({ Orientation: 1 } as never) - mockMetadata.mockRejectedValueOnce(new Error('Not a valid image')) + mockMetadata.mockRejectedValueOnce(new Error('Invalid TIFF')) const result = await extractAndValidateRawPreview('/path/to/photo.cr2') expect(result).toBeNull() - expect(fs.existsSync(createdPath)).toBe(false) + expect(fs.existsSync(createdTiff)).toBe(false) }) }) diff --git a/packages/core/src/transcode/raw-extract.ts b/packages/core/src/transcode/raw-extract.ts index 49b61c76..47cc3ad8 100644 --- a/packages/core/src/transcode/raw-extract.ts +++ b/packages/core/src/transcode/raw-extract.ts @@ -1,15 +1,37 @@ +import { execFile } from 'child_process' import { exiftool } from 'exiftool-vendored' -import sharp from 'sharp' import * as fs from 'fs' import * as os from 'os' import * as path from 'path' +import sharp from 'sharp' +import { promisify } from 'util' import { logger } from '@shumai/core/src/logger' +const execFileAsync = promisify(execFile) + export interface RawExtractResult { previewPath: string cleanup: () => void - /** EXIF orientation from the RAW container (1–8), or undefined if absent. */ + /** EXIF orientation from the RAW container (1–8), or undefined if absent or already applied. */ orientation?: number + /** Original unoriented or oriented image dimensions from EXIF if available */ + rawWidth?: number + rawHeight?: number +} + +export interface RawPreviewResult { + previewPath: string + cleanup: () => void + width: number + height: number + orientation?: number + rawWidth?: number + rawHeight?: number +} + +export interface DecodeRawOptions { + timeoutMs?: number + signal?: AbortSignal } /** @@ -68,22 +90,37 @@ export async function extractEmbeddedJpeg( if (fs.existsSync(candidatePath) && fs.statSync(candidatePath).size > 0) { logger.debug({ tag, rawFilePath, candidatePath }, 'Extracted embedded JPEG from RAW') - // Read orientation from the RAW container's EXIF metadata. - // The extracted JPEG binary stream often does NOT carry its own - // orientation tag, so we must read it from the parent RAW file - // to avoid incorrectly-rotated previews. + // Read orientation and container dimensions from the RAW container's EXIF metadata. let orientation: number | undefined + let rawWidth: number | undefined + let rawHeight: number | undefined try { const exif = await exiftool.read(rawFilePath) const rawOrientation = exif.Orientation if (typeof rawOrientation === 'number' && rawOrientation >= 1 && rawOrientation <= 8) { orientation = rawOrientation } + const w = + typeof exif.ImageWidth === 'number' + ? exif.ImageWidth + : typeof exif.ExifImageWidth === 'number' + ? exif.ExifImageWidth + : undefined + const h = + typeof exif.ImageHeight === 'number' + ? exif.ImageHeight + : typeof exif.ExifImageHeight === 'number' + ? exif.ExifImageHeight + : undefined + if (w && h && w > 0 && h > 0) { + rawWidth = w + rawHeight = h + } } catch { logger.debug({ rawFilePath }, 'Could not read EXIF orientation from RAW container') } - return { previewPath: candidatePath, cleanup, orientation } + return { previewPath: candidatePath, cleanup, orientation, rawWidth, rawHeight } } } catch { logger.debug({ tag, rawFilePath }, `Could not extract ${tag} from RAW file`) @@ -103,10 +140,123 @@ export async function extractEmbeddedJpeg( } /** - * Validate that an extracted JPEG file is decodable. + * Fallback RAW decoder using dcraw_emu (from LibRaw). + * + * Used when no usable embedded JPEG preview exists in the RAW file. + * + * Decoding options: + * - Camera white balance and sRGB output (-w -o 1) + * - Output temporary TIFF (-T -Z ) + * - Half-size processing (-h) enabled only when longest dimension > 8192px + * + * LibRaw auto-rotates the output image to upright orientation based on camera EXIF, + * so the returned orientation is undefined to avoid double-rotation in downstream Sharp. + */ +export async function decodeRawWithDcraw( + rawFilePath: string, + targetDir?: string, + options?: DecodeRawOptions, +): Promise { + const ownTempDir = !targetDir + const workDir = targetDir ?? fs.mkdtempSync(path.join(os.tmpdir(), 'raw-decode-')) + const tempTiffPath = path.join( + workDir, + `decoded-${Date.now()}-${Math.random().toString(36).slice(2, 8)}.tiff`, + ) + + const cleanup = () => { + try { + if (ownTempDir && fs.existsSync(workDir)) { + fs.rmSync(workDir, { recursive: true, force: true }) + } else if (fs.existsSync(tempTiffPath)) { + fs.rmSync(tempTiffPath, { force: true }) + } + } catch (err) { + logger.warn({ workDir, tempTiffPath, err }, 'Failed to clean up dcraw temporary files') + } + } + + // 1. Probe RAW dimensions and orientation via ExifTool to check for 8192px threshold + let useHalfSize = false + let rawWidth: number | undefined + let rawHeight: number | undefined + try { + const exif = await exiftool.read(rawFilePath) + const w = + typeof exif.ImageWidth === 'number' + ? exif.ImageWidth + : typeof exif.ExifImageWidth === 'number' + ? exif.ExifImageWidth + : undefined + const h = + typeof exif.ImageHeight === 'number' + ? exif.ImageHeight + : typeof exif.ExifImageHeight === 'number' + ? exif.ExifImageHeight + : undefined + + if (w && h && w > 0 && h > 0) { + const longest = Math.max(w, h) + if (longest > 8192) { + useHalfSize = true + } + + // Compute upright raw dimensions matching dcraw_emu's auto-rotation + const rawOrientation = exif.Orientation + const isSwapped = + typeof rawOrientation === 'number' && rawOrientation >= 5 && rawOrientation <= 8 + rawWidth = isSwapped ? h : w + rawHeight = isSwapped ? w : h + } + } catch (err) { + logger.debug({ rawFilePath, err }, 'Could not read EXIF metadata for RAW dimension check') + } + + // 2. Build dcraw_emu arguments + const args = ['-w', '-o', '1'] + if (useHalfSize) { + args.push('-h') + } + args.push('-T', '-Z', tempTiffPath, rawFilePath) + + const dcrawBin = process.env.DCRAW_EMU_PATH || 'dcraw_emu' + const envTimeout = process.env.DCRAW_EMU_TIMEOUT_MS + ? parseInt(process.env.DCRAW_EMU_TIMEOUT_MS, 10) + : undefined + const timeoutMs = options?.timeoutMs ?? (envTimeout && !isNaN(envTimeout) ? envTimeout : 60_000) + + try { + logger.debug({ dcrawBin, args, rawFilePath }, 'Executing dcraw_emu fallback') + await execFileAsync(dcrawBin, args, { + timeout: timeoutMs, + signal: options?.signal, + }) + + if (!fs.existsSync(tempTiffPath) || fs.statSync(tempTiffPath).size === 0) { + logger.warn({ rawFilePath, tempTiffPath }, 'dcraw_emu did not produce a valid output TIFF') + cleanup() + return null + } + + return { + previewPath: tempTiffPath, + cleanup, + orientation: undefined, // dcraw_emu already auto-rotates upright based on camera EXIF + rawWidth, + rawHeight, + } + } catch (err) { + logger.warn({ rawFilePath, err }, 'dcraw_emu failed to decode RAW file') + cleanup() + return null + } +} + +/** + * Validate that an extracted or decoded image file (JPEG, TIFF, etc.) is decodable. * Returns dimensions if valid, null if corrupt/unusable. */ -export async function validateExtractedJpeg( +export async function validateExtractedImage( filePath: string, ): Promise<{ width: number; height: number } | null> { try { @@ -120,34 +270,66 @@ export async function validateExtractedJpeg( } } +/** Backward compatibility alias */ +export const validateExtractedJpeg = validateExtractedImage + /** - * Extract and validate an embedded JPEG from a RAW file to a temporary file. + * Extract and validate a preview from a RAW file to a temporary file. + * + * 1. Attempts ExifTool embedded JPEG extraction first. + * 2. If no embedded preview is available or extraction fails, falls back to dcraw_emu decoding. + * * Returns a validated file path + dimensions + orientation + cleanup function, or null. */ export async function extractAndValidateRawPreview( rawFilePath: string, targetDir?: string, -): Promise<{ - previewPath: string - cleanup: () => void - width: number - height: number - orientation?: number -} | null> { +): Promise { + // 1. Try ExifTool embedded JPEG extraction first const extracted = await extractEmbeddedJpeg(rawFilePath, targetDir) - if (!extracted) return null - - const dims = await validateExtractedJpeg(extracted.previewPath) - if (!dims) { - logger.warn({ rawFilePath }, 'Extracted JPEG from RAW is not decodable, skipping preview') + if (extracted) { + const dims = await validateExtractedImage(extracted.previewPath) + if (dims) { + return { + previewPath: extracted.previewPath, + cleanup: extracted.cleanup, + orientation: extracted.orientation, + rawWidth: extracted.rawWidth, + rawHeight: extracted.rawHeight, + ...dims, + } + } + logger.warn( + { rawFilePath }, + 'Extracted JPEG from RAW is not decodable, falling back to dcraw_emu', + ) extracted.cleanup() + } else { + logger.info( + { rawFilePath }, + 'No usable embedded JPEG found in RAW file, falling back to dcraw_emu', + ) + } + + // 2. Fallback to dcraw_emu + const decoded = await decodeRawWithDcraw(rawFilePath, targetDir) + if (!decoded) { + return null + } + + const dims = await validateExtractedImage(decoded.previewPath) + if (!dims) { + logger.warn({ rawFilePath }, 'Decoded TIFF from dcraw_emu is not decodable, skipping preview') + decoded.cleanup() return null } return { - previewPath: extracted.previewPath, - cleanup: extracted.cleanup, - orientation: extracted.orientation, + previewPath: decoded.previewPath, + cleanup: decoded.cleanup, + orientation: decoded.orientation, + rawWidth: decoded.rawWidth, + rawHeight: decoded.rawHeight, ...dims, } } diff --git a/packages/core/src/transcode/transcode.test.ts b/packages/core/src/transcode/transcode.test.ts index 790f44fc..875f2c0a 100644 --- a/packages/core/src/transcode/transcode.test.ts +++ b/packages/core/src/transcode/transcode.test.ts @@ -506,6 +506,26 @@ describe('TranscodeService', () => { expect(info.mimeType).toBe('') }) + it('should prioritize rawWidth and rawHeight in getImageInfo when available', async () => { + const rawPath = path.join(tempDir, 'highres.arw') + const cleanupSpy = vi.fn() + vi.spyOn(rawExtract, 'extractAndValidateRawPreview').mockResolvedValueOnce({ + previewPath: '/tmp/decoded-half.tiff', + cleanup: cleanupSpy, + width: 4752, + height: 3168, + rawWidth: 9504, + rawHeight: 6336, + orientation: undefined, + }) + + const info = await transcodeService.getImageInfo(rawPath) + expect(info.originalWidth).toBe(9504) + expect(info.originalHeight).toBe(6336) + expect(info.mimeType).toBe('jpeg') + expect(cleanupSpy).toHaveBeenCalled() + }) + it('should transcode RAW image by extracting preview to temp file and feeding to sharp', async () => { const rawPath = path.join(tempDir, 'sample.nef') const outputFile = path.join(tempDir, 'output-raw.webp') @@ -528,6 +548,31 @@ describe('TranscodeService', () => { expect(cleanupSpy).toHaveBeenCalled() }) + it('should transcode RAW image from dcraw_emu fallback TIFF without double-rotating', async () => { + const rawPath = path.join(tempDir, 'fallback.dng') + const outputFile = path.join(tempDir, 'output-fallback.webp') + const cleanupSpy = vi.fn() + vi.spyOn(rawExtract, 'extractAndValidateRawPreview').mockResolvedValueOnce({ + previewPath: '/tmp/decoded-dcraw.tiff', + cleanup: cleanupSpy, + width: 3168, + height: 4752, + rawWidth: 6336, + rawHeight: 9504, + orientation: undefined, + }) + + await transcodeService.transcodeImage(rawPath, outputFile, 1920, 85) + + expect(sharp).toHaveBeenCalledWith('/tmp/decoded-dcraw.tiff', { limitInputPixels: false }) + const mockSharp = vi.mocked(sharp).mock.results[vi.mocked(sharp).mock.results.length - 1].value + expect(mockSharp.rotate).not.toHaveBeenCalled() + expect(mockSharp.toColorspace).toHaveBeenCalledWith('srgb') + expect(mockSharp.webp).toHaveBeenCalledWith({ quality: 85 }) + expect(mockSharp.toFile).toHaveBeenCalledWith(outputFile) + expect(cleanupSpy).toHaveBeenCalled() + }) + it('should apply orientation rotation when transcoding RAW image', async () => { const rawPath = path.join(tempDir, 'rotated.cr3') const outputFile = path.join(tempDir, 'output-rotated.webp') diff --git a/packages/core/src/transcode/transcode.ts b/packages/core/src/transcode/transcode.ts index 016dfc40..03bd1895 100644 --- a/packages/core/src/transcode/transcode.ts +++ b/packages/core/src/transcode/transcode.ts @@ -784,7 +784,7 @@ export class TranscodeService { } } - // RAW branch — extract embedded JPEG for metadata + // RAW branch — extract embedded JPEG or decode RAW for metadata if (typeof input === 'string' && isRawImage(input)) { const extracted = await extractAndValidateRawPreview(input) if (extracted) { @@ -793,8 +793,10 @@ export class TranscodeService { extracted.orientation !== undefined && extracted.orientation >= 5 && extracted.orientation <= 8 - const originalWidth = isSwapped ? extracted.height : extracted.width - const originalHeight = isSwapped ? extracted.width : extracted.height + const sourceWidth = extracted.rawWidth ?? extracted.width + const sourceHeight = extracted.rawHeight ?? extracted.height + const originalWidth = isSwapped ? sourceHeight : sourceWidth + const originalHeight = isSwapped ? sourceWidth : sourceHeight return { originalWidth, @@ -1545,13 +1547,15 @@ export class TranscodeService { input = Buffer.from(await resp.arrayBuffer()) } - // RAW branch — extract embedded JPEG to temporary file + orientation, then treat as normal image + // RAW branch — extract embedded preview or decode RAW to temporary file + orientation, then treat as normal image let rawOrientation: number | undefined let rawCleanup: (() => void) | null = null if (typeof input === 'string' && isRawImage(input)) { const extracted = await extractAndValidateRawPreview(input) if (!extracted) { - throw new Error(`Cannot generate preview for RAW file: no usable embedded JPEG in ${input}`) + throw new Error( + `Cannot generate preview for RAW file: no usable embedded preview or decoded image in ${input}`, + ) } input = extracted.previewPath rawOrientation = extracted.orientation diff --git a/packages/transcode/src/activities/transcode.test.ts b/packages/transcode/src/activities/transcode.test.ts index 3c9b19d1..9a39c250 100644 --- a/packages/transcode/src/activities/transcode.test.ts +++ b/packages/transcode/src/activities/transcode.test.ts @@ -998,6 +998,24 @@ describe('Transcode Activities', () => { ).rejects.toThrowError(/Image transcoding failed/) }) + it('should throw non-retryable ApplicationFailure when transcodeImageActivity fails with dcraw error', async () => { + vi.mocked(s3Service.headObject).mockRejectedValue(new Error('Not found')) + vi.mocked(transcodeService.transcodeImage).mockRejectedValue( + new Error('dcraw_emu failed to decode RAW file'), + ) + + await expect( + transcodeImageActivity({ + assetKey: 'photo.cr2', + filePath: '/tmp/photo.cr2', + imageSpec: { width: 800, height: 600, quality: 90, format: 'webp' }, + }), + ).rejects.toMatchObject({ + message: expect.stringContaining('Image transcoding failed'), + nonRetryable: true, + }) + }) + it('should throw non-retryable ApplicationFailure when generateSpriteActivity fails with error', async () => { vi.mocked(s3Service.headObject).mockRejectedValue(new Error('Not found')) vi.mocked(transcodeService.generateSprite).mockRejectedValue(new Error('FFmpeg failed')) diff --git a/packages/transcode/src/activities/transcode.ts b/packages/transcode/src/activities/transcode.ts index 2fb44c50..50b3f551 100644 --- a/packages/transcode/src/activities/transcode.ts +++ b/packages/transcode/src/activities/transcode.ts @@ -772,6 +772,8 @@ export async function transcodeImageActivity( lowerMsg.includes('imagemagick') || lowerMsg.includes('magick') || lowerMsg.includes('convert') || + lowerMsg.includes('dcraw') || + lowerMsg.includes('raw') || lowerMsg.includes('spawn') || lowerMsg.includes('format') ) {