[Visual Communication Lab] I used AI technology to create a PV for a fictional sci-fi movie
Introduction
Hello, I'm Wakabo!
May has flown by, and before I knew it, the rainy season has arrived. My writing speed on note is like a snail's, but I want to finish this note before the rainy season ends so I don't dry up. Since I'm writing it, I hope to provide interesting content, so please enjoy it until the end!
Well, this is the 4th research presentation for the "Visual Communication Lab"...! The theme this time is, quite simply, the "Who Loves AI the Most?? Championship".
Roughly speaking, it's a project where we use AI tools to create something interesting, have everyone judge it with AI, and compete!
In this article, I, researcher Ueno, also known as Wakabo, will introduce the results I presented in this championship.
Details about the Visual Communication Lab, this project, and the presentation contents of other researchers are written in the following note. I think you will enjoy this article more if you read it, so please take a look at it as well! 🙌
(04 Theme Summary: Scheduled to be published soon ↓)
About what I presented
Here is my approach to this theme in 100 characters ↓
I experimented with what would happen if I rearranged the children's song "Koinobori" (Carp Streamer) in a modern style using three types of AI tools for the music, lyrics, and visuals. Unexpectedly, it turned out like a PV for a fictional sci-fi fantasy movie.
And here is the trailer for the fictional sci-fi movie "KOINOBORI" that was created ↓ (⚠︎ Caution: Volume ⚠︎ Please turn your smartphone sideways and watch using the full screen ⚠︎)
"KOINOBORI" Synopsis
Children's Day in a certain future.
People on Earth were able to decorate space with carp streamers due to advancements in space exploration technology. While looking at the carp streamers, children dreamed of becoming space explorers themselves.
Then one day, a mysterious life form suddenly appeared on Earth from space.
They said they were interested in the carp streamers because a creature called a carp also existed on their home planet.
The people of Earth taught them the meaning of the carp streamers, and they found a common topic.
And so, the space exploration that the children had dreamed of was accelerated by this encounter. A future arrived where culture and knowledge were exchanged between Earth and other stars, leading to new discoveries.ーーーーーー
Now, as for this fictional sci-fi movie that was finished unexpectedly, I will explain how it came to be and how I used AI to make it from here on out.
AI services and other tools used
First, here are the tools I used!
As you all know, ChatGPT, the automatic music generation service CREEVO, and for image generation, I used Canva. All of these can be started for free. If you are not yet familiar with AI tools, please use this article as an opportunity to try them out! 💫
Production flow
Actually, as I wrote "unexpectedly" for this KOINOBORI, it was a product born from moving forward while surrendering to impulses and ideas with a vague final image, and even if I try to put it into words later, I think it is difficult to neatly summarize the thought process at that time.
Therefore, you might catch glimpses of logical breakdowns in the text ahead (perhaps there wasn't any to begin with), but I hope you will turn a blind eye to that and enjoy the creator's impatience, momentum, and tension.
Process 1: Collaboration with CREEVO
First, I thought about how to answer the challenge of "making something interesting using AI".
And since I was most familiar with the automatic music generation service "CREEVO" and "ChatGPT", I decided to start production by trying to create something experimentally using voice and text.
Since I developed the idea starting from the tools, I first wondered what could be done with automatic music generation AI, and as a tentative theme, I hit upon the idea of "trying to compare modern melodies added to children's songs with expired copyrights," and used CREEVO to load the lyrics of the existing children's song "Koinobori" and generate a melody.
That is this.
I wanted it to feel modern, so I set it to a Kenshi Yonezu-meets-Higedan vocal style and named the generated video 'Reiwa-era Koinobori.' (You can easily generate music like this in CREEVO!)
Once the music generation experiment was done, I struggled with how to involve ChatGPT. After some aimless wandering—like having it write a fictional second verse for the children's song 'Koinobori' or rewriting the lyrics because the originals are too old-fashioned—I gave up and decided to just create a 'Children's Day of the Future.'
Process 2: Collaboration with ChatGPT
To define what Children's Day of the Future would look like, I first wanted to clarify the 'image of children in the future.' After having GPT predict a realistic society and image of children in the not-too-distant future, I asked it to think about Children's Day in that future, and...
'Th-that's so booooring!!!'
I didn't feel like it was going to be interesting...! I had to create a punchline for my presentation, so this was a crisis. My way of explaining it wasn't good. Sorry.
Then, feeling desperate, I asked it to 'Talk about it like it's sci-fi!' and a ray of hope appeared.
Oh, that's kind of like it...!
First, I was happy that it properly included the terms related to the current Children's Day that I had persistently fed it as background knowledge. However, the story was a bit cliché, so I wanted it to be more creative and continued by asking, 'Please tell me the synopsis of that story.'
Bingo!!!
That's the way, ChatGPT!!! You're the best!! You're so efficient!! You're amazing!!!
......So, what do I do with this?
CREEVO can't turn long text into songs, so I was stuck.
But since I had such an interesting synopsis, I wanted to make use of it somehow... and I had to create a punchline for the presentation.
Process 3: Power is everything after all
The night before the presentation, feeling cornered, I decided to use brute force.
If I mix the 'Reiwa-era Koinobori' and the 'fictional movie synopsis' using music editing software, I can make it into one piece of work!!
By paying homage to the melody of the original song, I was able to retain the 'Koinobori' feel. Yay! ✌️
After struggling with GarageBand for about 3 hours, a sci-fi movie-like song was completed...! I layered the synopsis, read aloud by the 'Keita' voice data from 'Ondoku-san,' over it.
As a result, I achieved the mission of using AI to generate something I don't quite understand...!
Process 4: The greed that comes with fun
Considering the presentation environment, it would lack impact if I couldn't approach it visually as well. I wondered if I could finish it as more interesting content.
Now, how should I put it all together...................Let's just make it a movie!!
I took the synopsis text generated by GPT, used Canva's image generation system to create key visuals for the movie, and put it all together into a video!
And so, the fictional sci-fi fantasy 'KOINOBORI' (the trailer) was born.
Note: The screening of the full feature is undecided.
About the presentation
For the presentation on the day, I created a prototype in Figma and used materials stretched vertically like an LP.
The unique phrases of the AI singing were quite catchy, so I was very happy that the participants were humming along after the presentation.
Finally
By using AI, I was able to produce this much output in a very short time. This time, I proceeded with the production on a whim without a clear purpose, so there were many twists and turns, but thanks to AI helping with my thinking, I felt that there are still many groundbreaking ways to use it to push the limits of individual ability (not just technical, but limits of concentration, time, and physical strength).
I hope the use of AI technology spreads more among people who love creation!
You can see other articles from Shidenken here.
They are very interesting, so please follow the magazine too!
Look forward to the next one!