SYSTEM NOTICE

Auto translation by AI. Be sure, accuracy, nuances and authorial intent may not be fully reflected.
見出し画像

Video generation is finally possible with Midjourney (v1)! About how to use it and its quality.

Hello, I am AI FREAK.

A "video generation AI feature" has finally been released by Midjourney!

It has been rumored for a while that "a video generation feature is coming from Midjourney!", and the time has finally come.

For example, the image below was created with Midjourney,

and here is the video version of the above created with Midjourney.

It moves quite naturally, doesn't it?

After actually trying it out, I am quite excited as the quality is enough to threaten existing video generation AIs.

First, I will explain the simple usage and my personal review, and then I will introduce the Midjourney roadmap announced by the official team.

How to create videos with Midjourney

First, I will explain the specific usage.

By the way, video generation is only available on the Midjourney official website ( https://www.midjourney.com ).

Top screen

For those who have been using it on Discord, I think it is better to switch to using the web version.

Basic workflow: Image-to-Video (I2V)

Midjourney video uses a method called "Image-to-Video (I2V)" which animates existing images.

So, let's create an image first. Then, select the image you want to animate.

When you generate images with Midjourney, four images are generated as shown above.

Let's select one of your favorite images from these.

Then, it will look like this screen.

Looking at the bottom right

An Animate Image item has been added.

You can easily create videos just by selecting Motion from here.

There are a few types, each with

🤖 Automatic Mode
The AI automatically comes up with motion prompts. This is perfect for when you want to "just try moving it" or "see some interesting accidental results."
✍️ Manual Mode
For advanced users who want to direct the movement themselves. An "Imagine bar" opens, allowing you to describe how you want it to move using "motion prompts."

It's something like that.

As the names suggest, Low or High represents the magnitude of the movement.

Tips for effective motion prompts (Manual Mode)

By the way, when you select Manual,

you can enter prompts just like when generating images.

Unlike prompts for still images, I personally think the key is to describe "what happens over time."

Subject action: a woman walking forward, steam rising from a cup
Camera movement: slow zoom in, camera pans left
Environmental effects: soft morning fog, gentle lens flare from the setting sun

The above is just an example, but I think it's easier to create the movement you're aiming for if you start simple with things like "walking" or "swaying" and gradually combine them.

The downsides of Midjourney's video generation (honestly speaking)

The movement is nice, but since the V1 model is still in development, there are a few downsides. Let's understand the following points.

Resolution: 480p (SD quality). Honestly, this is a bit underwhelming.
Frame rate: 24fps.
Missing features: There is no text-to-video (T2V) feature, no feature to edit only a part of the video, and no audio generation.

So, compared to other video generation tools, there are some inferior points.

However, as usual, AI updates happen at lightning speed.

It's only a matter of time before the above issues are resolved, so let's play around with it as much as possible while we can.

Midjourney's grand ambition

By the way.

Midjourney's future ambitions are quite grand, so it's worth keeping them in the back of your mind. Midjourney is

not just saying 'we made a video feature because it's trendy'

—it's not that simple. Their official stance goes far beyond what we can imagine. As for what that means? Honestly, I don't fully understand it myself either. Haha.

Until now, Midjourney could create 'images' with AI. But in the future, they plan to create

'a 3D world where you can move freely in real-time'

It means AI will be able to instantly create a moving world that you can move around in freely, just like a game.

And to achieve that, they say four elements are necessary:

Image (Image model): Technology to generate beautiful images.
Video (Video model): Technology to animate the images created.
3D model: Technology to move freely through space.
Real-time processing: Technology to make everything move instantly at high speed.

Currently, progress has reached the second stage, which is video.

They plan to complete these in order over the course of a year and finally integrate them all together.

Honestly, it's a timeline that's hard to even imagine, but seeing this, I felt even more strongly that

'for now, you can't go wrong by subscribing to Midjourney'

Haha.

It seems almost certain that it will continue to be the leading AI tool from here on out.

Summary: The future of Midjourney video and what we should do

As mentioned above, Midjourney's video feature still has some technical limitations.

However, since it is the top-tier AI tool for images, it is very possible that it will reach that same level of quality in video generation as well.

For now, I'm personally looking forward to the upcoming roadmap, as we can expect improvements in resolution and control features.

I've written many other notes related to Midjourney, so please check out my magazine if you'd like.

See you in another note!




いいなと思ったら応援しよう!