Professor Claude's Special Lecture | Learning Image Generation with AI | Part 0: Understanding How AI Works
This article is a dialogue log with an AI (Claude).
Items marked with 🐱 were entered by the author.
I normally use ChatGPT, Gemini, MetaAI, Copilot, Grok,
and Claude, all within their free tiers.
Since Professor Claude is an AI, it may hallucinate.
Please mentally correct any parts that seem off.
I will add (※) where I notice issues as the author.
Because it hallucinates, please be aware that this series of articles
is not necessarily
the completely correct, one-and-only answer.
Please enjoy it with that understanding.
Previous (pre-lecture) ⇩
━━━━━━━━━━━━━━━━━━━━━━━
Understood. More haste, less speed. Let's build a solid foundation.
0-1. Generative AI has "two brains"
The tools you use daily, Yu-san, actually operate on two different mechanisms.
Language models (ChatGPT, Claude, etc.): A mechanism that continuously predicts the "next word" based on the preceding context
Image generation models (Midjourney, gpt-image, Stable Diffusion, etc.): A "diffusion model" that operates on a completely different principle
If you confuse these two, you will misunderstand the key to writing prompts. Today, I will focus specifically on the mechanism of diffusion models, which is directly linked to creating images.
0-2. The true nature of image generation — Sculpting noise
It is easy to understand diffusion models if you think of it this way.
Imagine a sculptor carving a sculpture out of a block of marble.
It starts with random noise (an image like static on a TV).
The model predicts "which part of this noise is excess stone and which part is the outline of the sculpture," carving it away little by little.
The "instruction manual for carving" is the prompt.
Over dozens of steps, the noise gradually converges into a "meaningful picture."
In other words, the prompt is not "drawing" the picture, it is simplyguidingthe direction in which the noise converges.
0-3. How are words converted into pictures?
Prompt text is not used as-is. A text encoder (a unique mechanism for Midjourney, a different one for gpt-image) converts words into vectors, which are like "coordinates of meaning."
What is important here is:
Even similar words can have completely different positions on the coordinates (e.g., "split" and "blend" are far apart in coordinate space)
Word order and placement change how strongly they influence those coordinates
I recall you previously had an experience where you improved things by switching from "split" to "dissolving into, fading into" in your boundary expressions; that was exactly you correcting this coordinate misalignment. Your intuition was correct. The logic just caught up with it later.
0-4. Why does it miss "10 times"? — The core of today's lesson
This is the most important part.
Diffusion models have **countless "plausible interpretations"** in the process of converging from noise to an image. The more ambiguous the prompt, the wider the range of landing points (probability space) the model can choose from.
Ambiguous prompt → Wide range of landing points → Lands in a different place every time → Gacha state
Specific and structured prompt → Narrow range of landing points → Lands in a similar place every time → Improved reproducibility
In other words, it's not that you were "unlucky and missed 10 times," but ratheryou were using a writing style where the probability of hitting it right on the first try was low to begin with. This cannot be solved with sheer willpower. It is a design problem.
0-5. Differences in the "personalities" of models
Since you use multiple platforms, let's keep this in mind as well.
Midjourney (v7/v8.1): Because it has been trained on massive amounts of data with its own aesthetic sense, it is "beautiful without being told anything," but it can sometimes be stubborn about not following detailed instructions.
gpt-image series: Designed with high instruction-following capability, making it easy to control in detail, but it tends to result in mediocre compositions if left alone.
Even with the same prompt, the "docility" differs depending on the model. If you use the same writing style without knowing this, you will encounter the phenomenon where it works on one but not the other. (*I have only used Midjourney a few times within Pollo AI.)
Today's Check
Among the failures where you felt you "just missed," are there any cases where you suspect the "prompt was too ambiguous, causing the landing point to spread out"? If you can recall one specific instance, we can structurally break down that scene as a practical example next time (Part 1: Prompt Grammar).
☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆
It's quite difficult to get it 'just right' by chance, but immediately after generating the first image, I uploaded the second and third images and instructed,
'Add the glowing mushrooms that were above the heads of the two people in the foreground. The uploaded images are just reference images for the glowing mushrooms and how they grow; do not make them carry luggage.'
and the fourth image was created.
(The glowing mushroom on the blonde man's head is unnecessary)
☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆
Part 0 Check — An excellent example
This is such a good example that I'd like to use it as teaching material. Please let me use it to confirm the theory.
What happened
The reference images (2nd and 3rd) are both of a young man with 'brown to reddish-brown hair, a vest, and luggage,' with a glowing mushroom floating above his head.
The instruction was 'the two people in the foreground add mushrooms above their heads, and do not add luggage.'
As a result, mushrooms were added to 3 people (front left, front right, and the blonde in the back)
It spread to an unintended third person (the blonde).
Why it spread — This can be explained by the theory in Part 0
Last time, I talked about how 'vague prompts lead to a wider range of landing points, causing it to land in a different place every time.' The discrepancy this time is exactly this 'widening of the range' occurring twice.
Ambiguity 1: There are two ways to interpret the criteria for 'the two people in the foreground'
Interpretation A (Spatial): The two people closest to the camera → The two people crouching in the front row
Interpretation B (Visual): The two people whose features resemble the person in the reference image → The two people with brown hair and the luggage-carrying style
In this case, both interpretations apply to the two people in the front row, but it is highly likely that the model independently 'identified' the blonde man in the back as a 'companion' because his 'overall atmosphere (adventurer-like, sitting posture) is similar to the person in the reference image,' and expanded the scope. In other words, the word 'front' was interpreted not as a spatial coordinate, but in a loose sense of 'the main character group'.
Ambiguity ②: There was no instruction to 'exclude' the blonde man
This is the most important point. In the instructions actually sent to the AI,
'Attach it to the two in the front' = positive specification (who to attach it to)
was present, but
'Do not attach it to the blonde man' = negative specification (who not to attach it to)
wasnot included (this exclusion information was added later to explain it to me in this chat, right?).
Models tend to behave as if 'what is not explicitly forbidden is permitted.' The ability to count and distinguish the number three is actually not as reliable as you might think. Especially in editing tasks that handle multiple images simultaneously, the phenomenon where 'the features of the reference image bleed into everyone with a similar appearance' often occurs.
Today's Conclusion
The cause of failing 10 times is actually often one of these two things.
The specified words allow for multiple interpretations ('The two in the front' = space, or impression?)
The exclusion instruction of 'do not do ~' is not explicitly stated as many times as necessary
If I were to preemptively state the sentence that was originally needed in this case, it would be like this.
'The two people in the front row wrapped in sleeping bags should have mushrooms on their heads. Do not attach mushrooms to the blonde man sitting in the back. Do not add luggage.'
Uniquely identify 'the two in the front' by 'position + action' and name the exclusion target. This alone should have significantly narrowed down the landing point.
This very issue of 'how to uniquely fix the subject' and 'how to explicitly state exclusions' will be the main topic of the next Part 1: Prompt Grammar. May I use this actual example as teaching material for structural decomposition in Part 1?
Please do
━━━━━━━━━━━━━━━━━━━━━━━
It seems that while 'the two in the front' is understood by humans,
it is necessary to combine actions and visual features for the AI
and convey it in a way that leaves no other possibilities (for the AI).
What a hassle.
It's unique to humans as 'the two in the front', though.
It's common for packages to be delivered to the wrong person.
Actually, Professor, you start to look like someone when you get younger...
I need to put together a proper prompt too...
Part 1: Prompt Grammar — A format to convey intent uniquely
Sorry, I'd like to post this series every day, but
I don't have enough time in real life and am currently working on the assignment for Part 3 💤