It's finally here! A Leonardo.Ai-level Image2Motion environment on your local PC!
It's finally here!
The arrival of a Leonardo.Ai-level Image2Motion environment on your local PC.
Moreover, according to this article, advanced integration and environment setup are handled for you
so you don't have to worry about them. (Thank you for sharing this useful information.)
I was in the middle of continuous drawing work to lower the effect of Asian-feman-000068.safetensors and find a landing point for attractive expressions and
toning down excessive fleshy depictions, but I interrupted that to try this out immediately.
The following is information as of the night of Saturday, 09/21/2024.
Information of this kind often becomes obsolete on a daily basis.
As I have encountered several times in past articles, there have been many cases where the prerequisites for the execution scripts introduced in reference articles
differ from the latest versions of git, etc., and when execution errors occur, you end up needing the skills to cut and try the scripts yourself.
The article I referred to was published on Thursday, 09/21/2024 at 12:00.
Since immediate execution is the right way to go and it is important to share information quickly, I have interrupted my usual topics to cover this.
First, I accessed Pinokio.
I downloaded the installer from "INSTALL" -> "windows".
When I unzipped and ran the installer, I received this confirmation.
When I pressed "Allow," this screen appeared, and since there was a note about "NO exFAT drives" for the installation destination, I installed it with the default values.
I had forgotten what format my external HDD was, and I also thought it might be problematic to intentionally create a situation where the prerequisite environment and execution environment are on different drives, which the prepared execution script might not anticipate. (The screen shows the state after pressing execute.)
The "CogStudio" I was looking for appeared in the menu.
Clicking it led me to the download screen.
It determined that neither the prerequisite environment nor the execution environment were on the C:\ drive
(all other drawing environments like Stable Diffusion Web UI Forge are built on an external HDD). Regardless, I executed "Install".
I was worried about the free space on the C:\ drive, but I ran it anyway.
My main PC is a product positioned as a gaming PC, so it was likely designed with the assumption that it wouldn't need that much storage capacity.
Once execution started, jobs were automatically executed one after another.
Twice, a dialog asking "Do you want to allow changes to the system?" appeared, and when I responded with consent, the installation of "Visual Studio Build Tools 2019" began.
Once that finished successfully, the jobs continued to execute automatically.
However, when the screen returned to its original state, many "Not Installed" items remained.
Did the prepared script fail somewhere?
Feeling disappointed, I pressed "Install" again, and the installation job started running again, and since the next screen appeared, I pressed "Download".
I finally reached the installation screen for "CogStudio".
I hesitated about choosing to customize the installation location to an external HDD,
but I went ahead and executed it anyway.
Some warnings in red text appeared, but the job didn't stop and kept
progressing.
"Cog Studio" has launched.
From the "Image to Video" tab screen, I attempted to generate a video using an unpublished image from a FLUX.1 drawing article that was originally scheduled to be released if not for this series of articles.
(The full-sized original image will be released in a later article.)
On the Image-to-Video tab screen, features to supplement the image concept with text prompts
and to adjust the amount of video transformation are also implemented.
Thinking that 8 fps would make the video look choppy, I chose the
16 fps option.
After over 2700 seconds from the start, a progress bar finally appeared, indicating the process was at 9/50 steps.
Over 4000 seconds have passed since execution, and the progress bar has advanced significantly.
I watched it reach 98% progress, and while I was taking a screen capture of the screen just before completion with Paint... wait! Error...
It says that older types of PCs do not support "bfloat16",
so in that case, use "float16".
After restarting the PC, I set it that way, removed the greedy "8fps → 16fps"
option, and re-executed it.
This time, I didn't do any screen capture work related to extra memory usage,
and just watched the status.
It was generated successfully. A small-sized GIF was generated at the same time as the mp4.
(In this article, I will post the GIF converted from the mp4 using Adobe Express with the usual size setting, along with its reduced-size GIF.)
However... of the 6 seconds generated, is 4 seconds the limit?
The end is too eerie.
However, Leonardo.Ai's Image2Motion was a 4-second output.
If you cut the end using video editing software like VideoProc Vlogger,
I think it can be said to be almost equivalent to Leonardo.Ai's Image2Motion (the loss of facial detail is also equivalent).
Even with the 8 fps setting, I don't feel it's unexpectedly choppy.
This is one that I cut at 4 seconds and lengthened (half-speed).
I tried generating it again. Since different results come out every time, it seems best to generate it multiple times
to aim for the best one.
On the other hand, "float16" + "16fps" still resulted in an error.
However, there is no logo, and there is no daily limit.
Even this is faster than current cloud-based drawing services.
There are no reprimands saying "this type of subject matter cannot be turned into a video."
(I haven't had the leisure to try it until now, but surely
that might happen with cloud-based drawing services, right?)
I am optimistic that if it were video generation of mechanical drawings like Leonardo.Ai's Image2Motion,
the flaws might be less noticeable.
This is also a video generation from a space station image in FLUX.1 from the article that was originally scheduled to be released.
(GIF converted from mp4 using Adobe Express with the usual size setting, and the simultaneously generated reduced-size GIF)
This can be called a video equivalent to Leonardo.Ai's Image2Motion that can use the full 6 seconds.
The video of the space station rotating with a sense of mass without distortion
was almost the only area of expertise for Leonardo.Ai's Image2Motion,
but this is reminiscent of that.
As expected, the cyber female soldier image was the most difficult subject for verification.
There is a level of precision that cannot be discovered otherwise.
With mechanical drawings, the flaws are not visible.
At the moment, I don't know the settings for video generation that match a vertical composition,
so it seems that with a horizontal composition, you can use the full screen of the generated video.
Just like the old mage.Space, which cleared its accumulated assets with a revamp,
farewell, Leonardo.Ai's Image2Motion.
You can just keep saying "Free users cannot use video generation for the time being" forever.
The era has already moved on.
On my laptop with Ryzen 7 + CPU VRAM 16GB + Geforce RTX4060 (GPU
VRAM 8GB), it takes about 3000 seconds to generate one video,
so I will take my time and try generating videos from various subjects without rushing.
I am very interested in whether the understanding of anatomy, such as physical laws, human muscle tissue, and bone joint movement, is also taken into account in video generation.
(Simultaneously exploring FLUX.1, searching for the future path of SD1.5, and pursuing this possibility.... If this were a company job, I'd already be on a pace of over 10 hours of overtime, and it's currently 03:32 AM...)
Perhaps starting with this, paths to unrestricted, free video generation on local PCs will emerge from others as well.
Just like the emergence of FLUX.1, the era has clearly changed again.
Thank you for watching.
(Written on 2024/09/21-22)