[Production Log] When I made a 3.5-minute conversation drama with Seedance, it could no longer be completed by AI alone
New episode 02 of the AI recipe drama 'Testament Recipe', titled 'Something Fancy', has finally been completed. It consists of about 20 cuts. That's quite a lot and it's long. Just this once.
This episode serves as the starting point for the entire series, where the family is informed of their mother's remaining life expectancy.
The mother collapses and is rushed to the hospital by ambulance. Although they feel relieved after seeing her looking well for a moment, the next day, only the father hears the test results.
After that, the father says to the siblings,
'You guys should eat something fancy today.'
He says.
A fancy meal is delivered, and just as the siblings pick up their chopsticks, the father tells them about their mother's remaining life expectancy. Unable to look at each other's faces, the three of them cry while looking in different directions.
This time, I turned that day in the family's life into a video of about 3 and a half minutes.
1. Seedance is quite compatible with conversation dramas
What I felt again this time is that Seedance handles conversations between characters quite naturally.
Especially in scenes where three people, A, B, and C, speak short lines in turn.
Even without specifying the cut breakdown in detail, it sometimes automatically organizes edits such as: - Close-up of the person speaking - Reaction of the other person - Switching to the next speaker - Returning to the whole group at the end.
It sometimes automatically organizes edits like that.
In the scene where the father sits opposite the siblings and says, 'You guys should eat something fancy today,' Seedance automatically generated cuts like:
Father sitting down -> Father's close-up -> Sister's reaction -> Brother's reaction -> Return to father -> Forced smile fades, eyes cast downward.
It generated cuts like that. It's a god-tier cut...
It's not that I specified the direction in detail.
It seems the model judged the speaker and the change in emotion, and edited it so that it works as a drama.
I was quite surprised by this.
2. The @ specification in the character sheet is excellent
This time as well, I used Seedance's character sheet reference as usual to maintain character consistency.
Since you can call up characters using the @ designation, it is easy to produce multiple scenes while maintaining distinctions in face, age, clothing, hairstyle, and between characters.
Furthermore, I found that when using the same face and clothing, the voice also tends to come out relatively similar.
It is not exactly the same voice, but the impression is that in many cases, there is no major sense of discomfort even when listening to them in succession. Can you actually fake a voice!?
In the future, I think that a voice reference function that can be used with the same feeling as a character sheet will also appear.
If it becomes possible to fix the face, clothes, voice, and even the acting, the production environment for AI serial dramas will likely change all at once.
3. Seedance: Why do you get those simple kanji wrong?
On the other hand, what I struggled with the most this time was the dialogue. Seedance is incredibly good at acting and emotional expression in voices.
But, however!!
Huh? Why did you make a mistake there? Even if I reluctantly change the kanji to hiragana, it still makes mistakes. It suddenly misreads simple kanji. I've learned a lot about its quirks with this, Seedance...
This time, the word "byonin" (sick person) wasn't stable, so I wrote it in hiragana as "びょうにん" in the prompt. Even then, it failed.
Even if it's not a difficult word, there is a theory that it's faster to use hiragana from the start if the reading is unstable, but it even got "byonin" wrong... What is "byojin"... Give me back my credits... I gave up and switched to the word "byoki" (illness). Goodbye, my credits...
The acting and dialogue are perfect, but a single misreading of one word in one place means I have to regenerate it. It's a frustrating failure typical of AI video production. Towards the end, I didn't want to waste any more credits, so I used a rough technique of generating only some of the dialogue with ElevenLabs and forcibly inserting the voice to connect it. Those who know, might know. But this way of saving it with voice might actually be pretty good lol
4. Don't make a 15-second conversation flat
In Seedance, I used many cuts of around 15 seconds. However, if you have characters talk while keeping the same composition, even 15 seconds feels surprisingly long. So this time, I was very conscious of the tempo of the conversation.
Every time the speaker changes, I insert
a front view, a profile, a reverse shot from the other person's side, a close-up of hands, and an insert of the cooking
to create small changes on the screen.
Even if the content of the dialogue is good, if the screen stays still, the viewers will leave. Conversely, if the camera moves too much, the feeling of a family's daily life disappears. Exquisite adjustment. It's difficult.
This time, I was conscious of changing the screen only at the timing when the gaze and the speaker change, rather than using flashy cuts.
5. BGM and narration carry emotions at double speed
This time, I once again realized the roles of BGM and narration.
A silent conversation drama is realistic, indeed.
However, showing a drama longer than three minutes in complete silence requires a significant amount of concentration from the viewer.
It takes time to convey emotions, and the durability of the video decreases.
When you add subtle BGM and narration to that, it delivers the emotions the video is trying to convey to the viewer about twice as fast in terms of experience. It's surprising how different it looks with and without BGM.
6. Making the AI 'cry realistically'
This time, the scene I spent the most time directing was the crying scene.
While crying performances are good at conveying emotion, if they become even slightly exaggerated, they immediately look artificial.
So, I changed the way all three characters cry.
The younger brother breaks down while looking out the window. The older sister sheds tears quietly while looking down at the food. The father tries to endure it while looking down, but sobs start to leak out halfway through.
None of them look at each other's faces.
In the actual memory, the three of them were crying while facing different directions.
I think they felt that if they looked at each other's faces, they would break down even further.
Instead of just lining up three 'crying characters',
• Gaze
• Breathing
• Shoulder movement
• How they wipe their tears
• The strength with which they suppress their emotions
I changed these for each character.
Rather than making them wail, trying to stop crying but being unable to looks more realistic as a human emotion.
7. GPT Image 2 is currently excellent at the highest level, but it still struggles with multiple characters and spatial control
I also used GPT Image 2 for the starting still images.
It is extremely strong when creating a single character or advertising visuals.
However, in scenes like this one where you have to manage:
• The three characters: father, older sister, and younger brother
• The position of the food
• The seating at the dining table
• The direction of the window
• The partitions in the restaurant
• The camera position
all at the same time, there is still some instability.
If I generate it once and it doesn't improve after giving correction instructions, I don't persist; I take over and fix it myself.
Specifically, I created the world view and rough images with GPT Image 2, and performed corrections for character placement and space using Nano Banana.
Rather than trying the same corrections over and over in GPT Image 2,
the workflow of 'generate once -> correct -> if it fails, move to a different model' is faster.
⑧ Nano Banana is consistently strong at image output. However, the reference specification is a shame.
Nano Banana remains excellent for tasks like correcting multiple characters or making partial changes while maintaining an existing image.
On the other hand, it lacks the @reference method like Seedance.
Therefore, instead of writing it like the man in attached image 1, I have to identify the reference image by appearance, such as:
・64-year-old man wearing a plaid shirt
・27-year-old woman with long black hair
・20-year-old man wearing navy clothes
This is annoyingly tedious.
The character reference accuracy is high, but the way to call them is primitive.
I want them to switch to the @reference method soon.
As always, it's brilliant, but it's a 'dumb banana' in these aspects.
⑨ AI dramas cannot be completed by the model alone.
What I felt again during this production is that even if AI models become excellent, dramas are not completed automatically.
Seedance is extremely good at acting, voices, and shot composition.
However,
which lines to keep,
when to cut the screen,
which character to zoom in on,
where to stop the BGM,
which way of crying is characteristic of that character
are things that humans need to decide.
AI realizes the direction I've thought of with surprisingly high precision.
However, deciding what to show and what not to show is still the creator's job. 'Testament Recipe' is not a drama where flashy incidents happen one after another.
Cooking, failing, laughing with family, and going to the hospital.
It is a story where you realize that those ordinary daily life moments were a testament when looking back later.
Starting from this episode, we will move on to a slightly brighter family cooking episode next time.
Next time, it will bemiso soup without dashimade by the cooking-novice older sister.
Younger brother: 'Wait, what about the dashi?'
Older sister: '...dashi?'
From here, the original format of 'Testament Recipe', or rather, a bright and fun recipe drama, begins to move little by little! This time it's a slightly sad story. 3 minutes.
P.S.: The production cost for this one was just under 10,000 yen (crying).
I will consider using 480P generation plus upscaling in the future.
If I keep spending this much on every episode, my wallet will be given a death sentence before my family does, so I have to optimize costs somewhere (lol).
Since this one was long, it was about the production cost of two episodes, after all.
That said, now that I've made it this far, I'll aim for five episodes first.
I will keep an eye on the numbers and nurture it properly.