ChatGPT's Image Generation AI Gets a Major Update! Goodbye DALL-E 3? [GPT 4o]

On 3/26, ChatGPT's image generation AI received a major update!
Until now, ChatGPT used an image generation AI called DALL-E 3, but this has been replaced by a native GPT-4o model called "4o Image Generation".

*Reference: From the official OpenAI website
https://openai.com/index/introducing-4o-image-generation/

Since most of the images I have been using for my notes so far have been generated via DALL-E 3 in ChatGPT, this change is actually quite a matter of life and death for me.

So, I will discuss how ChatGPT's image generation AI has changed and what you can do if you still want to use DALL-E 3 in this post.

What exactly has changed in ChatGPT's image generation?

In short, it has changed to be able to "generate practical and high-precision images".

Previous image generation AIs had a weakness in their "ability to convey information", especially with text information, and Japanese text often ended up looking like so-called 'alien language'.

Of course, you can input Japanese text in Midjourney, but it doesn't work very well, and it was just an extra feature to begin with.

With this update to ChatGPT, this "conveyance capability" has been significantly enhanced.
In particular, it has become stronger at text rendering and generating meaningful images. Specifically, it is useful for purposes like the following.

• Diagrams

• Manga-style expressions

• Logos

• Visuals for presentation materials

• Blog eye-catching images

Also, character consistency is quite high, which is another point that has been improved compared to before.

While it was possible to maintain some level of consistency with DALL-E 3 depending on the method, it is now more reproducible, allowing you to maintain it even when changing styles, art styles, or posing.

Now, let's look at some actual generation examples.
The images generated this time were created using ChatGPT 4o or Sora (https://sora.com/) (currently, Sora's image generation uses the same model as ChatGPT).

1. Blog Eye-Catching Images

The following is an eye-catching image I created by instructing it to include the text "How to write a blog post!".

Like this, text can be inserted accurately.
Of course, depending on the length of the text, it does fail occasionally, but even so, this level of text insertion was completely impossible until now.

The images are also at this level even when created with rough prompts, so I think they can be used as eye-catching images as-is without looking out of place.

Furthermore, you can also change just the text in this image.
The following is the version converted to "ChatGPT Image Generation AI Update".

In this way, while maintaining a high level of consistency across the entire image, the text changes as instructed.

Being able to change things like this easily seems convenient as it eliminates the hassle of post-processing.
(*Well, one could argue that if it's just text, you could just change it yourself in Canva or similar...)

2. Diagrams

Also, it has become possible to create diagrams well.
This is also a major step forward, as DALL-E 3 could only produce things that were completely nonsensical.

Compared to other generative AIs (like Napkin.ai or Claude), I get the impression that it has a wider range of capabilities.

Below is a diagram of how generative AI works.
Although it is quite simplified, it is made to look like a diagram, including icons and arrows in addition to the text.

Furthermore, you can of course include Japanese text.
The following is an example of a diagram showing "How a kitten gets into a pot".

Well, setting aside whether there is a need to make this into a diagram, you can also create a fairly cute art style.

Text generation is the same as image generation; depending on the prompt, it seems you can also significantly change the impression of the text.

3. Text-Centric Generations

This can also be created with quite rough prompts.

As an actual example, I will provide a "screenshot-style image of a note displayed in a web browser".

You can tell this is AI-made because the kanji are strange, but if the text were correct, some people might think it's real if it were created at this level.

4. Manga

Furthermore, you can now create manga as well.

While it was possible to create images with panels before, the text often didn't make sense, and even if you wanted to create a 4-panel manga, it wouldn't even result in four panels.

With this update, you can create a 4-panel manga in one go with a simple prompt, or you can go through the process of creating a character first and then turn it into a 4-panel manga.

Here, I will briefly show an example generated using the latter method.

1. First, generate an image of the main character.
This time, I'm going for a fantastical feel.

2. Select this image and provide additional instructions to turn it into a 4-panel manga.
The following was created with a simple prompt like, 'Keep this style and make it into a heartwarming 4-panel manga.'

Even so, it captures the original character and background well and creates something with a quite soothing touch.

If you then instruct it to include text, you can create more practical 4-panel manga.

The following is an example where I kept the reference character and created a '4-panel manga explaining note'.

The overall atmosphere, consistency, flow of explanation, and content are all quite well-made, so I think it's sufficient for creating explanatory manga.
However, the tendency for kanji characters to break down is appearing here as well.


You can try to fix just the text within ChatGPT, but it basically doesn't work well, so I think it's better to just fix it with separate software.

As you can see, while there are some issues, it's at a point where you can somehow manage to create a 4-panel manga, which is a significant improvement compared to before.

On the other hand, it seems likely that long-form generated content, or panels with many characters or fine details, will end up breaking down, so that will probably be the next challenge.

Conversely, are there any downsides to the model changing?

Of course, there are a few disadvantages.

1. It takes time to generate

First and foremost, the generation time is long.

It might be because it's congested right after release, but even setting that aside, it feels slow.
Depending on the time of day, it takes several minutes for a single generation.

2. There are cases where the generated content is risky

As you can see by looking at X, Ghibli-style images are circulating, and the freedom of art style has become quite high.

In other words, I feel a considerable looseness or riskiness in the generated content.
Honestly, OpenAI has been very strict about this kind of thing until now, and they used to block people at the prompt stage, so I think this is a quite surprising change of direction.

Well, there were really free services like Grok, so maybe OpenAI is also steering in a direction where they don't care as much.

However, when that happens, while reliance is certainly a factor, it will become more important than ever to judge for yourself whether there are any issues with 'copyright, portrait rights, etc.' regarding the output.

3. Does it turn yellow if you repeat edits?

Also, I don't know the reason, but if you refer to an image and repeat edits (this becomes the remix function in Sora), it can sometimes take on a slightly yellowish tint as shown below.

Also, there is a tendency for consistency to decrease more and more with each repetition, so I think this is a point that needs improvement.

4. The atmosphere of the illustrations is different from DALL-E 3

Regarding the quality of the generated output, the realistic style has improved to a level incomparable to the previous DALL-E 3, and the range of generated illustration styles is wide, with the quality itself being quite high.

Specific examples of the illustration style are as follows. Like this, if you don't specify a particular art style, it becomes
something simple and close to a so-called masterpiece face.

You might think, 'If the quality is high, isn't that fine?', but this is a significant point for quite a few DALL-E 3 users (I'm sure I'm not the only one, right?).

The reason is that the reason for using DALL-E 3, besides being able to use it easily, is that the generated illustrations have a uniqueness that is different from others.

Let's illustrate this as well. For example, an image like the one below. This was created with DALL-E 3.

This art style is something that other image generation AIs cannot produce well. Actually, the attempt to reproduce this with the new ChatGPT is shown below.

Like this, you can match the overall image, but the art style is different. I prefer the art style of DALL-E 3, so I'm quite worried about whether I can match it to some extent through trial and error, or what to do if I can't match it completely.

There are several reasons why it can't be matched; for instance, the generation method is different, or perhaps the learning stage is different, but I also think that the effectiveness of the prompts is quite different. It's like, the precision of the prompts is quite high, but there are drawbacks because it's too high.

To put it more simply, for example, even if you put 'watercolor' in the prompt, it didn't have such a strong presence that it would become the main subject of the art style until now. However, with this model, there are cases where the presence comes out strongly, and then the overall balance is greatly disrupted... So, the difficulty in reproducing the DALL-E 3 art style is a challenge.

That said, even if I can't reproduce the art style, it feels like I could create various things if I refine the prompts. For example, I can create images like the one below, so I don't think this feeling is bad. Therefore, I plan to continue with trial and error for the time being.

For those who still love DALL-E 3: Two ways to keep using it

We have looked at the new version of the model so far, but it is not like the old version of DALL-E 3 has become unusable.

There are at least two ways to use it.

One is the pattern of using it with Bing Image Creator.
Although the restrictions on size and prompt character count are stricter, it is possible to use DALL-E 3 just as before.

The other is the method of using it from within the GPTs in ChatGPT.
If you select "DALL-E" from within the GPTs, you can use it just as you have until now.
(This is the icon in the upper left of the image below)

So, it seems DALL-E 3 will be usable for the time being.
However, since how long it can be used depends on OpenAI's policy, it is not certain whether it will remain for a long time as a legacy model like GPT-4, or be buried early like GPT-3.5...
I think it will remain since Bing exists, but who knows.

This time it has become long, but I would like to end it here.
If you found this helpful, I would be happy if you gave it a like.

See you next time.

◯ Related Links