Releasing video content powered by OpenAI Codex and Seedance 2.0 / Detailed explanation of the production method
First, please watch the video below.
This is the behind-the-scenes footage of the AI drama (Episode 1) released the other day. It is a making-of video where the cast introduces the behind-the-scenes of the drama production.
Duration: 6 minutes 21 seconds
Environment: OpenAI Codex
Video Generation: Seedance 2.0 / Image Generation: Nano Banana Pro/2, OpenAI Image 2
Music Generation: Suno
I was able to bring the "live-action expression" I envisioned to life almost exactly as I imagined it. I believe I was able to draw out the performance of Seedance 2.0 to a certain extent.
As I wrote in my previous article, Episode 1 is the first video content I have been able to produce exactly as I intended after using video generation AI for over three years.
And this behind-the-scenes footage (the making-of for Episode 1) has succeeded in reflecting my intentions even further.
This time, as a "step toward new video production," I am challenging myself with experimental methods.
Seedance 2.0 is neither a physics simulator nor a future-prediction AI, but it learns by integrating video, images, audio, and text, and can combine movement, direction, narrative, and referential consistency at a fairly high level.
In other words, when a creator prompts it with, "Imagine what happens 10 seconds from now," it is possible to convert the ambiguity of the prompt into the next development that makes sense visually.
The areas where Seedance 2.0 excels are left to the model, and I emphasize the "seeds of the next action" contained in the image slightly through prompts.
For example, in a fight scene, I include things like "the opponent's center of gravity is collapsing to the right," "the fist is being pulled back," or "the camera follows the moment of the counterattack." For a drama, I instruct acting such as "she sends a gaze that suggests she is hiding something" or "after a silence, the other person takes a step closer."
Seedance 2.0 then complements the 10 seconds of movement, acting, and camera work quite well from there.
This approach is extremely powerful for predicting "physically natural movements" and "visual causality" (punch -> fall, walk -> scenery changes).
On the other hand, in predicting highly logical and literary story developments, such as "10 seconds later, an unexpected joke or complex psychological foreshadowing is resolved," which is the forte of combinations like "Image Generation AI + LLM" such as Nano Banana Pro/2 or OpenAI Image 2, it goes beyond the scope of physical simulation.
It is important to use "semantic reasoning" and "physical reasoning" differently depending on the purpose.
I will explain in detail.
Fighting scenes
“A character throws a punch at an opponent in a stance, the opponent staggers and collapses against a wall” is an area you can leave to Seedance 2.0.
What the creator needs to specify here is just “who is attacking,” “where it hits,” “whether the result is a knockdown or an evasion,” and “whether the emotion is anger or anxiety.”
Fine details like weight shifting, clothing movement, and reflexive flinching are often more natural when left to the AI model.
However, when it reaches the level of “intentionally missing the first punch, breaking a wall light to trigger the opponent's trauma, and using that psychological opening to connect to a second strike,” it is no longer physics, but semantic design.
If you don't break this down into beats rather than letting Seedance 2.0 imagine it, it will fall apart. In other words, detailed prompt instructions are required.
Walking scenes
“A character walks, the scenery flows by, the destination approaches” can be left to the model.
For walking (movement) without special meaning, B-roll, etc., you get better results by leaving it to the AI model rather than giving detailed instructions.
However, for “while walking, the moment they see a shop sign, they recall foreshadowing from their childhood, and the expression immediately after becomes a sign of later betrayal,” you need context control, not just visual continuity, so you must specify the meaning of the expression and the visual elements to be inserted. Detailed prompt instructions are necessary.
Jokes
“Insert some unexpected joke” does not work well.
Even if “unexpected” is a concept humans can imagine, it is ambiguous to the model.
You must at least instruct on the four points: setup, trigger, punchline, and reaction.
For example, you need to break it down into visible causes and visible results, such as “during a serious conversation, the vending machine behind them malfunctions, cans fall in a row, and the two fall silent before turning around at the same time.” If you do that, you can leave only the physical reaction part to Seedance 2.0.
What you leave to Seedance 2.0 is the “future where you can understand what happens next just by looking,” and what the creator decides is the “future where you cannot understand what happens next unless you know the context.”
The former is physics, movement, and visual continuity, while the latter is logic, psychology, and literary meaning.
By cutting at this boundary, it becomes easier to avoid both rigidity due to over-description and semantic collapse due to leaving everything to the AI.
Tasks to leave to OpenAI Codex
Decompose scene ideas into “semantic reasoning” and “physical reasoning”
For example, if the scene is “Hikari avoids an enemy attack and changes the atmosphere of the room with an unexpected joke 10 seconds later,” have the model classify how much is physical continuity that can be left to Seedance and where the semantic design that humans must fix begins.
Generate prompts for Seedance 2.0 from the classification results
We will have it create three types: 'Model Version,' 'Hybrid Version,' and 'Fully Fixed Version,' and compare their behaviors.
Logging generation results
We record which prompts succeeded and which parts collapsed in Markdown. This allows for the accumulation of operational know-how.
Extracting 'success patterns' and 'failure patterns' from past logs
This is the most important part. Codex is better suited for the task of establishing rules while reading through file sets than for creating single-shot prompts.
Folder structure (example):
Separate prompts, scene designs, generation logs, and evaluation criteria.
AI_Drama_Tools/
AGENTS.md
README.md
prompts/
seedance2/
future_prediction/
physical_continuation.md
semantic_reveal.md
hybrid_scene.md
comedy_timing.md
fight_scene.md
scene_cards/
Hikari_Natural_Musou/
scene_001.md
scene_002.md
generation_logs/
2026-05/
2026-05-08_fight_future_test.md
2026-05-08_comedy_reveal_test.md
evaluation/
rubrics/
physical_reasoning_score.md
semantic_reasoning_score.md
prompt_failure_patterns.md
tools/
count_prompt_length.py
summarize_generation_logs.py
AGENTS.md (example)
# AI_Drama_Tools Project Instructions
このフォルダは、AIドラマ「ひかりの天然無双」および「COLORS - Posthuman Protocol」の制作に使う、Seedance 2.0向けプロンプト研究・管理フォルダです。
## 基本方針
Seedance 2.0には、画面内の物理的・視覚的連続性を任せる。
ただし、伏線回収、ジョークのオチ、心理的反転、複雑な意味設計は、必ずプロンプト内で明示する。
## 分類ルール
以下は「物理的推論」として扱う。
- パンチから相手がよろめく
- 歩行に合わせて背景が変化する
- 視線が動く
- 重心が崩れる
- カメラが被写体を追う
- 衣服や髪が動作に合わせて揺れる
以下は「意味的推論」として扱う。
- 予想外のジョーク
- 伏線回収
- 裏切りの示唆
- 心理的駆け引き
- 隠された情報の開示
- 物語上の皮肉
- キャラクター同士の関係性の変化
## プロンプト作成ルール
1. まずシーンを「意味イベント」と「物理イベント」に分解する。
2. 意味イベントは明示的に書く。
3. 物理イベントは過剰に細かく書きすぎず、Seedance 2.0に自然補完させる。
4. 否定プロンプトは避ける。
5. 生成時間に対してイベントを詰め込みすぎない。
6. セリフがある場合は、話者、表情、前後の間を明記する。
7. 最終出力は、英語版と日本語版の両方を作る。
8. 3,500字以内に収める必要がある場合は、文字数を確認する。
## 完了条件
出力には以下を含める。
- シーン分解
- Seedanceに任せる部分
- プロンプトで固定する部分
- Seedance 2.0用プロンプト英語版
- Seedance 2.0用プロンプト日本語版
- 想定される失敗
- 改善案Prompts to input into Codex
このフォルダは、Seedance 2.0向けのAIドラマ制作プロンプト管理フォルダです。
次のシーン案を読み込み、以下の手順で整理してください。
#シーン案
ひかりが敵の攻撃をかわす。10秒後、敵は勝ったと思って笑うが、ひかりが一言ジョークを言い、敵の背後で装置が誤作動して形勢が逆転する。
#実行内容
1. このシーンを「物理的推論」と「意味的推論」に分解してください。
2. Seedance 2.0に任せてよい部分を抽出してください。
3. プロンプトで明示しないと崩れる部分を抽出してください。
4. Seedance 2.0用のプロンプトを3種類作ってください。
- 物理連続性を重視する版
- 意味イベントを固定する版
- 両方のバランスを取る版
5. 英語版と日本語版を作ってください。
6. 各プロンプトについて、想定される失敗と改善方針を書いてください。
7. 結果を prompts/seedance2/future_prediction/scene_001.md に保存してください。Logs after generation
# Generation Log
## Prompt
prompts/seedance2/future_prediction/scene_001.md
## Result
- ひかりの回避動作は自然だった。
- 敵の笑いは出たが、ジョークの意味が伝わらなかった。
- 装置の誤作動は発生したが、形勢逆転として見えなかった。
- カメラは良いが、最後の間が短い。
## Evaluation
Physical reasoning: 4/5
Semantic reasoning: 2/5
Comedy timing: 2/5
Character consistency: 4/5
## Next Fix
ジョークの内容と、装置誤作動が敵に不利に働く因果を明示する。generation_logs/2026-05/ にあるログを読み、失敗原因を分類してください。
分類軸は以下です。
- 物理的推論の失敗
- 意味的推論の失敗
- セリフ・話者の失敗
- カメラ・編集テンポの失敗
- キャラクター一貫性の失敗
- 参照画像の扱いの失敗
その後、prompts/seedance2/future_prediction/scene_001.md を改善し、改訂版を作ってください。
元のプロンプトは残し、v2として保存してください。General workflow
Brainstorming scene direction with ChatGPT
Having Codex break down the scene
Have Codex create 3 prompt drafts for Seedance 2.0
Generate using Seedance 2.0
Record the results in generation_logs
Have Codex read the logs and categorize the causes of failure
Have Codex create v2 prompts
Reflect successful patterns in AGENTS.md or the rubric
Codex and Claude Code are editing assistants.
Drama production involves a director, screenwriter, editor, art director, production manager, and script supervisor. Among these, Codex is particularly close to a:
Production manager
Script supervisor
Documentation organizer
Prompt manager
Assistant who reads failure logs
.
In conventional generative AI workflows, the results of each generation were not saved as logs.
The reasons for success, the reasons for failure, and the differences in prompts are all forgotten over time.
Using Codex makes it easy to record them.
This expression can be left to Seedance 2.0
This psychological depiction will fall apart unless fixed in the prompt
With this amount of dialogue, it is easy for things to get swapped
This camera specification worked
This negative expression did not work
You can keep such insights inside your production folder.
And you can analyze them.
Codex and Claude Code are tools built for engineers, but creators can utilize them as 'AI assistants for developing production notes'.
Image and video generation tend to result in scattered trial and error. By using Codex, you can keep prompts, failures, improvement ideas, and templates within a folder, which becomes a future asset (analytical data).
I think it is best to view them as tools for transforming your own production know-how.
To be able to migrate to Claude Code at any time
What you should be careful about is rigid know-how that depends on a specific model.
This will become obsolete in no time.
You need to turn it into an 'asset' that can be sustained even if the model changes.
Claude Code is the best!
A few weeks later: Codex is better! Time to switch!
A few weeks later: Claude Code's restrictions have been significantly relaxed! Time to switch!
A few weeks later: Whoa, Codex usage has increased tenfold, this is amazing!
It is important to master the generative AI environment while maintaining a state where you can 'move' at any time.
Something that has happened in every era during transitional periodsSo, for a while longer, we have no choice but to proceed agilely, like riding a wave.
Related Magazine:
Updated: Friday, May 8, 2026 / Published: Friday, May 8, 2026