Module 3 of 6 · Intermediate · 10 min read

Video generation workflows

Plan, generate and edit AI video like a director: shot lists, consistent characters, sound and a final cut.

Key takeaways

1Plan in shots, not whole films: generators make short clips.
2Write prompts with shot type, camera move, action beats, light and sound.
3Use reference images or start frames to keep characters and products consistent.
4Edit the clips together in a real editor, then add music, voice and titles.

01Where AI helps

Most video models generate clips of a few seconds to a few tens of seconds. Longer pieces are built shot by shot and edited together.

Short social clips, ads and product teasers.
Storyboards and animatics before a real shoot.
B-roll, establishing shots and background plates.
Animating still images and illustrations.
Scripts, shot lists and voice-over drafts.

02The workflow

The Creative AI path's video module covers prompting fundamentals; this guide is the end-to-end production process.

Write the script or concept and break it into a shot list.
Create key frames or reference images for characters, products and locations.
Generate each shot, several takes at a time, from text or a start frame.
Pick the best takes and trim them in an editor.
Add voice-over, music, sound effects, captions and titles.
Review on the target platform and device, then export.

03Tools to reach for

Video models differ a lot in motion quality, physics, length and audio. Compare the video models on this site's Models pages before choosing.

Text-to-video and image-to-video models, some with native audio.
Image models for key frames and consistent characters.
Video editors, many with AI captions, cut-down and reframing features.
Voice, music and sound-effect generators for the audio track.

04Prompt: script to shot list

Let a chat model turn an idea into shots you can generate one by one.

Example

Turn this concept into a 30-second video for {{platform}}: {{concept}}. Write a shot list as a table: shot number, duration (3 to 8 seconds), shot type, camera movement, action, setting, lighting, on-screen text, and audio. Keep the main character and colour palette consistent and describe them once at the top so I can reuse the description in every shot prompt.

05Prompt: a single shot

Write like a director: one clear action per shot.

Example

Medium close-up, slow push-in. {{character description from the shot list}} sits at a café window on a rainy evening, looks down at a handwritten letter, then smiles and looks up towards the door. Warm tungsten interior light, cool blue street light outside, rain streaks on the glass. Shallow depth of field. Sound: soft rain, distant traffic, a cup set down on a saucer. No text on screen.

06Prompt: image to video

Starting from a frame you've approved is the most reliable way to control the look.

Example

Animate this image. Keep the composition, the character's face and clothing, and the lighting exactly as in the frame. Motion: the camera slowly orbits 20 degrees to the right while the character turns her head to follow a passing bird. Duration about 5 seconds. Natural, subtle motion; no morphing or new objects.

07Pitfalls

Generate several takes per shot and choose; it is usually faster than perfecting one prompt.

Too much in one shot: several actions per prompt produce muddled motion. One beat per shot.
Drifting characters: reuse the same description and reference images in every shot.
Physics and hands: watch for warping, extra fingers and objects passing through each other.
Real people: never generate realistic video of real people without consent, and don't create misleading footage.
Platform rules: many platforms require labelling realistic AI-generated video.

Finished reading?

Mark it complete to track your progress, then test yourself with the Build with AI: Software & Media quiz.

Next: Animation and motion design →