Skip to Content
Slloy Docs
Architecture, Design & Testing

Reference Image Preprocessing & Trust Boundary

Browser-side EXIF/sRGB/white-background normalization (image-prep.ts) paired with low-cost Worker header sniffing (imageSniff.ts).

Implemented on 2026-11-11 (decisions/reference-image-preprocessing.md), Slloy splits reference image handling across a Browser Normalization Pipeline (frontend/site/image-prep.ts) and a Worker Trust-Boundary Header Sniffer (server/src/imageSniff.ts).

1. Architectural Rationale: Why Browser-First + Worker Header Sniffing?

  • Cloudflare Workers Have No sharp / libvips: Decoding and re-encoding up to 6 multi-megabyte reference images in pure JavaScript inside a Worker would exhaust CPU time limits and the 128 MB isolate memory ceiling.
  • Browsers Have Hardware-Accelerated Image Pipelines: The browser's native image decoder and <canvas> automatically apply EXIF Orientation, convert ICC/CMYK profiles to sRGB, strip GPS/EXIF metadata on re-encode, and compress to WebP or JPEG before a single byte is uploaded over the network.
  • The Worker Remains the Security Trust Boundary: Because client code can be bypassed, the Worker never trusts the data:image/...;base64, prefix alone. Instead, server/src/imageSniff.ts parses the binary headers (PNG IHDR, JPEG SOF + EXIF, WebP VP8X/VP8/VP8L) to verify actual format, byte size, and pixel dimensions at near-zero CPU cost.

2. Browser Normalization Pipeline (frontend/site/image-prep.ts)

All upload entry points (workbench.tsx image reference & video first-frame, effect-create.tsx photo upload) invoke prepareImageFile():

  1. Raw File Gate (Pre-Decode):
    • file.type must be image/png, image/jpeg, or image/webp (GIF and HEIC/HEIF are rejected immediately).
    • Raw file size must be <= 30 MB (allowing modern 48 MP smartphone photos that previously failed the raw 8 MB limit).
  2. Decode & EXIF Orientation Baking:
    • Decodes via <img> + Object URL (falling back to createImageBitmap(file, { imageOrientation: "from-image" })).
    • Immediately verifies post-decode dimensions: shortest side >= 64px (>= 300px for video first-frame), longest side <= 8192px, total pixels <= 50,000,000 (50 MP).
  3. White-Background Alpha Compositing:
    • Fills the canvas with #ffffff before drawImage so transparent PNG/WebP regions become deterministic pure white across all upstream channels.
  4. Native Resolution Preservation (No Default Downscaling):
    • Draws at the original image dimensions (1:1 pixel mapping).
  5. Two Preset Output Formats:
    • Image-to-Image Reference (Seedream / Qwen-Image): Encodes as WebP q = 0.90 (automatically falling back to JPEG q = 0.92 if the browser's canvas returns a non-image/webp blob). Shortest side >= 64px.
    • Video First-Frame (Seedance / Wan / Kling): Encodes directly as JPEG q = 0.92 (required because Kling 3.0/4.0 first-frame endpoints only accept JPG/PNG, not WebP). Shortest side >= 300px.
  6. Iterative Size Convergence (<= 2.5 MB Target):
    • If the encoded data URL exceeds 2.5 MB:
      1. Steps down quality first (WebP 0.82 -> 0.72; JPEG 0.84 -> 0.74).
      2. If still > 2.5 MB, scales dimensions down by 0.8x per step (up to 8 iterations) with imageSmoothingQuality: "high".
      3. Throws ImagePrepError if still over budget.

3. Server Trust-Boundary Contract (server/src/imageSniff.ts)

Every /api/generate, /api/video/generate, and /api/effects/* route passes incoming data URLs through checkRefImage():

Verification Steps & Configuration Defaults

Config VariableDefaultDescription
REF_MAX_BYTES8 MB (8,388,608 bytes)Maximum decoded byte length per reference image
REF_IMAGE_MIN_SIDE64 pxMinimum shortest side (width and height after EXIF orientation)
REF_IMAGE_MAX_SIDE8,192 pxMaximum longest side
REF_IMAGE_MAX_PIXELS50,000,000 (50 MP)Maximum total pixel area (width * height)
  1. Validates data:image/(png|jpe?g|webp);base64, prefix and base64 decodability.
  2. Decodes the first 1 MB slice (falling back to full buffer only if a JPEG SOF marker sits past 1 MB) to parse binary headers:
    • PNG: Parses IHDR chunk dimensions.
    • JPEG: Walks SOF0..SOF15 markers for width/height and parses APP1 EXIF TIFF IFD0 tag 0x0112 (Orientation 5..8 swaps width and height).
    • WebP: Parses RIFF....WEBP (VP8 , VP8L, VP8X) headers for width/height.
  3. Verifies that the binary magic header matches the declared MIME type in the data URL prefix.
  4. Enforces REF_MAX_BYTES, REF_IMAGE_MIN_SIDE, REF_IMAGE_MAX_SIDE, and REF_IMAGE_MAX_PIXELS.
  5. Logs parsed dimensions in [gen] start as refDims=3000x2000,1024x1024 (up to 6 images).

Deterministic HTTP 400 Error Codes

Error CodeCondition
BAD_IMAGE_FORMATPrefix/declared MIME not in {png, jpeg, jpg, webp}, or invalid base64 payload
IMAGE_MIME_MISMATCHDeclared MIME does not match binary header (e.g., data:image/png;base64,... wrapping JPEG bytes)
IMAGE_UNREADABLETruncated, corrupted, or forged header from which dimensions cannot be parsed
IMAGE_TOO_LARGEDecoded bytes > REF_MAX_BYTES, longest side > 8192, or total pixels > 50 MP
IMAGE_TOO_SMALLShortest side < 64px
BAD_REF_IMAGE / REF_NOT_SUPPORTEDReference image count exceeds limit (> 6), or selected model (z-image-turbo) has supportsRefImages: false

On this page