Reference Image Preprocessing & Trust Boundary
Browser-side EXIF/sRGB/white-background normalization (image-prep.ts) paired with low-cost Worker header sniffing (imageSniff.ts).
Implemented on 2026-11-11 (decisions/reference-image-preprocessing.md), Slloy splits reference image handling across a Browser Normalization Pipeline (frontend/site/image-prep.ts) and a Worker Trust-Boundary Header Sniffer (server/src/imageSniff.ts).
1. Architectural Rationale: Why Browser-First + Worker Header Sniffing?
- Cloudflare Workers Have No
sharp/libvips: Decoding and re-encoding up to6multi-megabyte reference images in pure JavaScript inside a Worker would exhaust CPU time limits and the128 MBisolate memory ceiling. - Browsers Have Hardware-Accelerated Image Pipelines: The browser's native image decoder and
<canvas>automatically apply EXIF Orientation, convert ICC/CMYK profiles tosRGB, strip GPS/EXIF metadata on re-encode, and compress to WebP or JPEG before a single byte is uploaded over the network. - The Worker Remains the Security Trust Boundary: Because client code can be bypassed, the Worker never trusts the
data:image/...;base64,prefix alone. Instead,server/src/imageSniff.tsparses the binary headers (PNG IHDR,JPEG SOF + EXIF,WebP VP8X/VP8/VP8L) to verify actual format, byte size, and pixel dimensions at near-zero CPU cost.
2. Browser Normalization Pipeline (frontend/site/image-prep.ts)
All upload entry points (workbench.tsx image reference & video first-frame, effect-create.tsx photo upload) invoke prepareImageFile():
- Raw File Gate (Pre-Decode):
file.typemust beimage/png,image/jpeg, orimage/webp(GIF and HEIC/HEIF are rejected immediately).- Raw file size must be
<= 30 MB(allowing modern48 MPsmartphone photos that previously failed the raw8 MBlimit).
- Decode & EXIF Orientation Baking:
- Decodes via
<img>+ Object URL (falling back tocreateImageBitmap(file, { imageOrientation: "from-image" })). - Immediately verifies post-decode dimensions: shortest side
>= 64px(>= 300pxfor video first-frame), longest side<= 8192px, total pixels<= 50,000,000(50 MP).
- Decodes via
- White-Background Alpha Compositing:
- Fills the canvas with
#ffffffbeforedrawImageso transparent PNG/WebP regions become deterministic pure white across all upstream channels.
- Fills the canvas with
- Native Resolution Preservation (No Default Downscaling):
- Draws at the original image dimensions (
1:1pixel mapping).
- Draws at the original image dimensions (
- Two Preset Output Formats:
- Image-to-Image Reference (
Seedream/Qwen-Image): Encodes as WebPq = 0.90(automatically falling back to JPEGq = 0.92if the browser's canvas returns a non-image/webpblob). Shortest side>= 64px. - Video First-Frame (
Seedance/Wan/Kling): Encodes directly as JPEGq = 0.92(required because Kling 3.0/4.0 first-frame endpoints only accept JPG/PNG, not WebP). Shortest side>= 300px.
- Image-to-Image Reference (
- Iterative Size Convergence (
<= 2.5 MBTarget):- If the encoded data URL exceeds
2.5 MB:- Steps down quality first (WebP
0.82 -> 0.72; JPEG0.84 -> 0.74). - If still
> 2.5 MB, scales dimensions down by0.8xper step (up to8iterations) withimageSmoothingQuality: "high". - Throws
ImagePrepErrorif still over budget.
- Steps down quality first (WebP
- If the encoded data URL exceeds
3. Server Trust-Boundary Contract (server/src/imageSniff.ts)
Every /api/generate, /api/video/generate, and /api/effects/* route passes incoming data URLs through checkRefImage():
Verification Steps & Configuration Defaults
| Config Variable | Default | Description |
|---|---|---|
REF_MAX_BYTES | 8 MB (8,388,608 bytes) | Maximum decoded byte length per reference image |
REF_IMAGE_MIN_SIDE | 64 px | Minimum shortest side (width and height after EXIF orientation) |
REF_IMAGE_MAX_SIDE | 8,192 px | Maximum longest side |
REF_IMAGE_MAX_PIXELS | 50,000,000 (50 MP) | Maximum total pixel area (width * height) |
- Validates
data:image/(png|jpe?g|webp);base64,prefix and base64 decodability. - Decodes the first
1 MBslice (falling back to full buffer only if a JPEG SOF marker sits past1 MB) to parse binary headers:- PNG: Parses
IHDRchunk dimensions. - JPEG: Walks
SOF0..SOF15markers for width/height and parsesAPP1EXIF TIFF IFD0 tag0x0112(Orientation5..8swaps width and height). - WebP: Parses
RIFF....WEBP(VP8,VP8L,VP8X) headers for width/height.
- PNG: Parses
- Verifies that the binary magic header matches the declared MIME type in the data URL prefix.
- Enforces
REF_MAX_BYTES,REF_IMAGE_MIN_SIDE,REF_IMAGE_MAX_SIDE, andREF_IMAGE_MAX_PIXELS. - Logs parsed dimensions in
[gen] startasrefDims=3000x2000,1024x1024(up to6images).
Deterministic HTTP 400 Error Codes
| Error Code | Condition |
|---|---|
BAD_IMAGE_FORMAT | Prefix/declared MIME not in {png, jpeg, jpg, webp}, or invalid base64 payload |
IMAGE_MIME_MISMATCH | Declared MIME does not match binary header (e.g., data:image/png;base64,... wrapping JPEG bytes) |
IMAGE_UNREADABLE | Truncated, corrupted, or forged header from which dimensions cannot be parsed |
IMAGE_TOO_LARGE | Decoded bytes > REF_MAX_BYTES, longest side > 8192, or total pixels > 50 MP |
IMAGE_TOO_SMALL | Shortest side < 64px |
BAD_REF_IMAGE / REF_NOT_SUPPORTED | Reference image count exceeds limit (> 6), or selected model (z-image-turbo) has supportsRefImages: false |
Public Site Design System & Tokens
Complete token specification, dark/monochrome themes, typography scale, Base UI constraints, Motion transitions, and design-tokens test enforcement.
Client Disconnect & Request Cancellation Observability
Architecture decision and telemetry design for measuring Cloudflare Workers request cancellation behavior, clientRequestId correlation, and stale pending recovery.