How NegPiP Works and FLUX Support
FLUX has high expressive power, but it is very troublesome that negative prompts cannot be used. To introduce negative prompts, CFG must be enabled, but this doubles the generation time. It is tough for the already long generation time to double. Therefore, I wonder if the NegPiP mechanism can be used.
NegPiP (Negative Prompt in Prompt) is a method I devised for Stable Diffusion 1.X and 2.X. In principle, negative prompts are designed to have a weaker effect than prompts, but this mechanism allows negative prompts to be used with the same intensity as prompts. This has made it possible to erase things that could not be handled by conventional negative prompts.
https://github.com/hako-mikan/sd-webui-negpip
I will explain the mechanism a little here. What I am manipulating is the Attention mechanism, which is called the heart of generative AI. Leaving the explanation of the Attention mechanism to another time, what is important are the values called Query, Key, and Value.
In the Attention mechanism, as the name suggests, it calculates the part to pay attention to. Query and Key are used for that, and the part that should be focused on is known as the calculation result. Pictures emerge by continuing to strengthen the weight of that part. The weight part mentioned here is the Value. For example, when the prompt "anime" is input, it searches for the "anime"-like part in the image by calculating Query and Key. Usually, the Value works to enhance that part, but in NegPiP, it works to erase it by inverting the Value. It is simple. I would like this level of functionality to be implemented by default.
I named it NegPiP, but now that I think about it, I feel like Negative Attention would be more fitting, but it is such a generic name that it seems like it would be hard to find in searches.
Now, let's look at the effects of actually applying it to FLUX. In FLUX, there was a problem where it would forcibly become anime-style if you entered anime-like words. It is fine with just "Japanese", but if you enter something like "sailor uniform", it immediately becomes anime-style. So, let's set "anime" to negative.
It properly became live-action. The effect has been confirmed. I will also try the example in the repository.
The effect is clearly appearing. The effect varies at -1.5, but I don't really know why. Also, FLUX should be stronger with natural language, but I haven't verified what happens when natural language is entered yet, so I would appreciate it if you could use it experimentally.