Skip to content

directors-cut: storyboard first, one action per shot, a director that checks its frames - #11

Merged
SamyMe merged 1 commit into
masterfrom
edenai-assistant/directors-cut-storyboard
Sep 29, 2026
Merged

SamyMe merged 1 commit into
masterfrom
edenai-assistant/directors-cut-storyboard

Conversation

@SamyMe

@SamyMe SamyMe commented Sep 29, 2026

Copy link
Copy Markdown
Member

The first Director's Cut sent PixVerse a long style paragraph followed by the action, as text-to-video. Planned actions didn't happen, the light drifted between shots, and sound levels jumped from shot to shot. This rework follows PixVerse's, Google's (Veo 3.1), OpenAI's (Sora 2) and Runway's prompting guides, plus recent research on AI film pipelines (MovieAgent, VISTA, ViMax). The sources are linked in the demo README.

before and after

What changed

  • The director writes for the models.
    • A cast, where each character has one fixed look.
    • A world: the place, the time of day and the light.
    • Shots, each with a first frame, one large visible action, one camera move and two to four concrete sounds.
    • The code pastes the looks and the light into every frame word for word.
  • Gemini 2.5 Flash Image draws the storyboard. It makes a reference sheet per character, then each shot's first frame from those sheets and the establishing frame. A frame costs $0.039.
  • The director reviews the storyboard using image input, and has a frame with a defect redrawn once. GLM 5.3 and DeepSeek V4 Pro can't see images, so Claude Sonnet reviews their films.
  • PixVerse animates each frame with image-to-video from a motion-only prompt, or makes one timecoded multi-shot take from the first frame.
  • The edit levels each shot's loudness, then normalizes the cut to −16 LUFS.
  • Reshoots can redraw the frame as well as re-render the shot.
  • Removed:
    • Continuity mode. The research says chaining last frames builds up drift.
    • PixVerse V5.5, which is superseded by V6.
  • GLM 5.3 now gets 16,000 tokens and low reasoning effort. At 8,000 it thought until no JSON fit.
  • The page has five stages (Direct, Draw, Review, Shoot, Edit), cast sheets, frame images on the cards, and editable first-frame and action fields. It also works at phone width.

Tested with real calls

Film Director, camera Time Cost
Robot, 3 shots GPT-6 Sol, V6 121 s $0.94
Lighthouse, 4 shots Claude Sonnet, V6 127 s $1.18
Samurai, 2 shots GPT-6 Astra, C1 137 s $0.79
Bike ad, one multi-shot take Kimi K3, V6 246 s $0.82
Reshoot with a redrawn frame Claude Sonnet, V6 66 s $0.28
  • All eight directors returned valid plans in the new format.
  • Dialogue: a speech-to-text check heard both of the keeper's lines exactly as written.
  • Loudness: the cuts sit around −16 LUFS, from raw shots between −24 and −49 LUFS.
  • Fake Eden AI server: the retry without references, reshoots and input validation passed against it.

🤖 Generated with Claude Code

… checks its frames

The first version sent PixVerse a long style paragraph followed by the action, as
text-to-video. Planned actions didn't happen, light drifted between shots and
sound levels jumped. Following PixVerse's, Google's, OpenAI's and Runway's
prompting guides and recent film-agent research:

- The director writes a cast (one fixed look each), a world (place, time of
  day, light) and shots with a first frame, one visible action, one camera move
  and concrete sounds. Code pastes looks and light into every frame word for word.
- Gemini 2.5 Flash Image draws a reference sheet per character, then each
  shot's first frame from those sheets and the establishing frame.
- The director reviews the storyboard (image input) and has frames with a
  defect redrawn once. Text-only directors are reviewed by Claude Sonnet.
- PixVerse animates each frame (image-to-video) from a motion-only prompt, or
  one timecoded multi-shot take from the first frame.
- The edit levels each shot's loudness, then normalizes the cut to -16 LUFS.
- Reshoots can redraw the frame. Continuity mode and PixVerse V5.5 are removed.
- GLM 5.3 gets 16k tokens and low effort: at 8k it thought until no JSON fit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SamyMe
SamyMe merged commit 2055eab into master Sep 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant