Vidora: Architecting an AI Video Pipeline That Survives Failure

Vidora is an AI SaaS platform I'm building for an international client: a user pastes a script (or lets GPT write one), picks a niche, an aspect ratio, a duration, a motion style, and a voice, and gets back a fully rendered, faceless MP4 in their dashboard. No avatars, no talking heads: the engine turns the script into a sequence of ~10-second scenes, generates a cinematic image per scene, simulates camera motion, and layers in narration and subtitles.

The generation pipeline looks like this:

Script (paste or GPT)
  → Scene breakdown (GPT): ~10s scenes {text, emotion, environment}
  → Image prompt per scene (GPT)
  → Scene image(s) (Stability AI SDXL)
  → Motion clip per scene (FFmpeg: zoom / pan / parallax / crossfade)
  → Voiceover (ElevenLabs)
  → Subtitles (Whisper → FFmpeg burn-in)
  → Stitch → final.mp4

Every interesting engineering problem in this project comes from one fact: this pipeline is long, expensive, and touches five external providers, any of which can fail at any moment, and users are paying credits for the result.

The State Machine Is the Product

My first instinct was to chain Celery tasks: task A calls task B calls task C. I abandoned that quickly. When a chain dies at stage 5 of 7, the queue knows a task failed, but nothing authoritative knows what the job had already produced.

Instead, the pipeline is a database-backed state machine. Each job row tracks its current stage, and crucially, preserves the output of every completed stage: scene breakdown JSON, image URLs, motion clip paths, audio files. If SDXL rate-limits at scene 14 of 20, retrying resumes from scene 14. Nothing regenerates, nothing double-bills. A watchdog sweeps for jobs stuck in a stage past their deadline and re-dispatches them.

The queue system dispatches work; the database is the truth. That inversion is the single most important design decision in the project.

Three Queues, Because Work Comes in Three Shapes

All async work runs on Celery with Redis as the broker, split into three queues:

• fast: GPT calls: script generation, scene breakdown, image prompts. Seconds each, cheap, high concurrency.
• render: SDXL image generation. Provider-rate-limited, the true bottleneck; worker concurrency is tuned to the vendor quota.
• media: FFmpeg motion, subtitle burn-in, assembly. CPU-heavy local work with completely different scaling characteristics.

One shared queue would let a burst of heavy renders starve everything else. With three, each shape of work scales independently. On top of that, job priority follows user tier: paid jobs ride high priority, trial jobs ride low, so when a provider rate-limits, it's the free tier that slows down, never the paid SLA.

Redis earns its keep well beyond brokering: it backs Celery's result state, rate-limit counters against each provider, and the locks that keep the watchdog from double-dispatching a stuck job.

Billing You Can Trust: Idempotency + Atomic Credits

Two rules protect the money path:

Client-generated request UUIDs. The frontend generates the ID; a retried submit with the same UUID is a no-op. Double-click safe, network-blip safe.

Credit deduction is atomic with job creation. One transaction: deduct credits, insert the job. No path where credits vanish without a job, or a job renders unpaid.

Credits themselves follow a config-driven formula (duration / 30 × mode_multiplier): pricing lives in configuration, so the client can rebalance economics without a deploy.

Cost Engineering the Visuals

AI image generation is the dominant cost, so the product exposes it as a choice: Mode 1 generates one SDXL image per scene and lets FFmpeg fake the life: slow zooms, pans, parallax; Mode 2 generates three images per scene (fixed seed for identity continuity, progressive prompts) and crossfades between them for a semi-animated feel at roughly 3× image cost. FFmpeg is the quiet hero of the whole system: motion simulated with filters costs fractions of a cent, looks intentional, and ships today. The visual stage is config-driven and swappable, so a real animation backend can slot in later without touching the pipeline.

Supabase: RLS and Realtime Instead of Custom Infrastructure

Supabase provides PostgreSQL, auth, and two features that deleted whole subsystems from my roadmap. Row-Level Security enforces tenancy in the database itself: even a buggy API query physically cannot leak another user's projects. Realtime streams job-row changes to the dashboard over WebSocket, so as the state machine advances, users watch their video progress live, with no polling endpoints and no custom socket server. The pipeline writes state; Supabase does the rest.

Rendered assets land in Backblaze B2 (S3-compatible at a fraction of the price), and Stripe handles subscriptions feeding the credit system.

Lessons Learned

Make the database the source of truth for long pipelines. Queues dispatch; they should never be the only place that knows what happened.

Preserve intermediate outputs. Resumable pipelines are the difference between a rate-limit event costing you a retry and costing you a full re-render.

Segment queues by the shape of work. API-bound, provider-limited, and CPU-heavy tasks have nothing in common operationally, so don't make them share a lane.

Fake the expensive thing when the fake is good enough. FFmpeg camera motion over one great image delivers most of the perceived value of animation at a hundredth of the cost.

Vidora is still in active development (the visual engine keeps evolving), but the architecture underneath has already survived provider outages, rate-limit storms, and my own refactors. That's what it was designed for.