Spending Safeguards, Worker Crons & Ops Alerts
Tiered daily video spending caps for paid vs. unpaid users, runtime app_settings switches, Worker Cron schedules, and 6-hour cooldown Resend alerts.
Because a single 1080p video generation can cost up to ¥21.60 (15s on Wan 3.0 or Kling 4.0), Slloy enforces runtime spending guardrails stored in PostgreSQL (app_settings), backed by two Cloudflare Worker Cron schedules and automated email alerts.
1. Tiered Daily Video Spending Caps (app_settings)
Configured in server/src/config.ts, seeded by migration 0002_seed_video_safeguards.sql (ON CONFLICT DO NOTHING), and editable live in /admin/settings (PUT /api/admin/settings) without redeploying the Worker.
Why Caps Are Tiered by Payment History (repo.userHasPaid)
A flat ¥20 / day cap protects the platform from free-credit farming (signup 100 + daily check-in 10), but blocks legitimate paying customers—for example, a user who bought pack_enterprise ($949.90 for 150,000 credits) running Seedance 2.0 PRO 1080p (¥13.97 cost per 5s clip) would be locked out after generating just one video per day.
POST /api/video/generate checks await repo.userHasPaid(userId) (whether the user has at least one paid order in orders) and applies separate dual-track limits (both Credits/day and Real Cost CNY/day):
app_settings Key | Target User Segment | Default Value | Unit & Scope | Setting 0 Behavior |
|---|---|---|---|---|
videoDailyCreditCap | Unpaid Users (free signup / check-in credits only) | 3,000 credits | Max net video credits spent per calendar day (Asia/Shanghai) | 0 = Unlimited |
videoDailyCnyCap | Unpaid Users | ¥20 (20 CNY) | Max cumulative upstream CNY cost per calendar day | 0 = Unlimited |
videoDailyCreditCapPaid | Paid Users (orders has status = 'paid') | 50,000 credits | Max net video credits spent per calendar day (Asia/Shanghai) | 0 = Unlimited |
videoDailyCnyCapPaid | Paid Users | ¥1,000 (1000 CNY) | Max cumulative upstream CNY cost per calendar day | 0 = Unlimited |
[!IMPORTANT] How Daily CNY Spend (
todayVideoSpend) Is Counted:
- For
status = 'success'rows: sumsgenerations.cost_cny.- For
pendingorfailedrows: sumsvideoCny(model, usage_tokens || est_tokens)(orcost_cnyfor per-second models). Even when a failed video refunds user credits, upstream providers (like Ark) may still bill the compute, so failed attempts count toward the user's daily CNY cap to prevent repeated failing calls from draining channel balance.- The CNY cap (
cnyLimitActive = true) is enforced whenevervideoDailyGuard === "1", regardless of whethervideoPriceGuardis enabled.
Runtime Feature Switches in app_settings
All three boolean switches in app_settings normalize strictly to "1" (enabled) or "0" (disabled) and have no implicit code fallback (if a row is missing from app_settings, it evaluates to disabled and /admin/settings displays "未设置"):
app_settings Key | Seeded Default | Behavior When Enabled ("1") |
|---|---|---|
videoDailyGuard | "1" (Enabled) | Enforces the tiered daily credit and daily CNY caps on POST /api/video/generate, returning HTTP 429 DAILY_CAP when exceeded. |
videoPriceGuard | "0" (Disabled) | Charges max(floorCredits, ceil(estCny / CREDIT_CNY)) at video creation. Keep at "0" in normal operation because CREDIT_CNY = 0.011 is the historical cost anchor (not BASE_UNIT_CNY = 0.0571), which would overcharge catalog prices by 5x. |
videoTrueUp | "1" (Enabled) | Enables post-settlement credit true-up (trueUpCredits with TRUEUP_THRESHOLD = 1.2 using BASE_UNIT_CNY) during foreground browser polling (GET /api/video/status/:id) when measured Ark tokens exceed 1.2x nominal cost. |
2. Cloudflare Worker Cron Schedules (server/src/scheduled.ts)
Configured in wrangler.jsonc under triggers.crons and dispatched by the Worker's scheduled() handler to runScheduled(event.cron). Each job runs inside its own try / catch block so a failure in one job never prevents subsequent jobs in the same Cron tick from running.
1. SWEEP_CRON = "*/5 * * * *" (Every 5 Minutes)
Runs three tasks sequentially:
jobs.sweep()->videoSweeper.sweepOnce():- Stale Pending Videos: Queries
listStalePendingVideos({ minAgeMin: 5, limit: 20 }), evaluatesdecideVideoAction(g, { maxAgeHours: 6 }), rescues completed upstream videos to R2 (allowTrueUp: false), refunds failed/expired (>= 24h) videos, and force-fails zombie tasks older than6hours. - Stale Pending Images (
sweepStaleImages): QuerieslistStalePendingImages({ minAgeMin: 10, limit: 50 })and refunds any image generation stillpendingafter10minutes withfailCode = "TIMEOUT". - Orphan Reference Images (
sweepRefImages): Lists R2_refs/objects and deletes any temporary reference image older than1hour (REF_IMAGE_MAX_AGE_MS).
- Stale Pending Videos: Queries
jobs.refreshBlockedDomains():- Reloads
blocked_email_domainsfrom PostgreSQL into isolate memory (setDynamicBlockedDomains) so domains blocked by operators take effect across isolates.
- Reloads
jobs.opsAlert()->runOpsAlert():- Evaluates operational health metrics and sends rate-limited Resend alert emails when thresholds are breached.
2. TIER_CRON = "0 * * * *" (Hourly at Minute 0)
Runs one task:
jobs.refreshTiers()->repo.refreshExpiredTiers():- Scans users whose active subscription or annual credit pack entitlement has expired and downgrades
users.plan(pro/basic->free) when no active entitlement remains.
- Scans users whose active subscription or annual credit pack entitlement has expired and downgrades
3. Operational Alerting & 6-Hour Cooldown (server/src/opsAlert.ts)
runOpsAlert() queries repo.opsMetrics() and evaluates four thresholds via evaluateOpsMetrics():
agedPendingImages > 5(category: "agedImages"): More than5images remainpendingpast the10-minute threshold aftersweepOnce()just ran.agedPendingVideos > 3(category: "agedVideos"): More than3videos remainpendingfor over30minutes.failureRatePct > 20%(category: "failureRate"): Over the last1hour, at least10generations completed (totalLastHour >= 10) andfailedLastHour / totalLastHour > 20%.timeoutRefunds24h > 10(category: "timeoutRefunds"): More than10image generations were refunded withfail_code = 'TIMEOUT'in the last24hours (primary shadow indicator for Cloudflare Queue consumer stalls or DLQ accumulation, since the CD API token does not have Queue depth read permissions).
Alert Dispatch & Cooldown Mechanism
- Zero Noise When Healthy: When
verdict.ok === true,runOpsAlert()returns silently with zero DB writes and zero log output. - Structured Error Log: On any violation, emits
[ops-alert] event=violation level=errorwith all metric counts and violation descriptions. - Cross-Isolate 6-Hour Email Cooldown:
- Computes a category-specific cooldown key:
ops-alert:email:<categories.join(",")>(e.g.,ops-alert:email:agedImages,timeoutRefunds). - Calls
repo.hitRateLimit(cooldownKey, 1, config.opsAlert.cooldownSeconds)wherecooldownSecondsdefaults to21,600seconds (6hours). - If acquired (
true), sends a single summary email viacreateMailer().send(...)(Resend) toOPS_ALERT_EMAIL_TO(prefixed with[Staging]whenconfig.staging.enabledis true). - Rollback on Send Failure: If Resend throws an error while sending,
runOpsAlertcallsrepo.clearRateLimit(cooldownKey)so the cooldown slot is released and the next 5-minute Cron tick retries sending the email.
- Computes a category-specific cooldown key:
Error Classification & Channel Fault Attribution
How Slloy maps raw upstream errors from Ark, DashScope, and Kling into 11 localized user-facing failCodes and 3 operator fault categories.
Prompt Library Architecture & Recommendation Engine
Prompt data model, 8 deliverable families (A-H), 10 target platforms, 60-second isolate cache with generation invalidation, ring-queue recommendations, and slot replacement state machine.