Porting the Art Style Cultivated in Illustrious to Anima - Steps to Create a LoRA from 80 Illustrations -

Introduction

In the previous article, I introduced the steps for migrating from the Illustrious environment to the Stable Diffusion WebUI Forge - Neo + Anima environment.

After actually migrating, I was quite surprised by Anima's expressive power.
Being able to write prompts in natural language and the high level of character reproduction using only tags gives it a different feel from previous models.

However, there was something I felt immediately after the migration.
That is,“It’s beautiful, but it’s not my art.”that is what I felt.

Since I have been creating illustrations in Illustrious for a long time, I have my own unique art style and coloring tendencies.
Anima is excellent, but as it was, the atmosphere was slightly different from the illustrations I had created until now.

So this time,I created a custom art style LoRA from about 80 illustrations produced in Illustrious.
In this article, I will introduce the procedure and the actual results.

Completed Results

First, please take a look at this.

Well, Anima alone can create decent results.

However, when the LoRA is applied, the atmosphere, coloring, and impression of the lines from the illustrations I have created so far have largely returned.
It doesn't become exactly the same.
But,my honest impression is that I am now able to reproduce my own art style while maintaining Anima's expressive power.

Can you really learn from 80 images?

Yes, you can.
In fact, I think you could do it with even fewer images (like 30).
Here are the learning conditions for this time.

・Training images: approx. 80
・Purpose: Art style LoRA, not character LoRA
・Training time: approx. 10 hours
・Training iterations: 10 epochs (a unit of how many times it was trained)
・Environment: RTX 5060 Ti 16GB
It was not the case that "it's impossible unless you prepare hundreds or thousands of images."

Of course, having more images is advantageous, but with an art style LoRA, 80 images are enough to get a response.

The most important thing was image selection

Actually, the most important part of this work was not the training settings.
It was the selection of training images.
I chose about 80 images from the illustrations I had generated in the past.

The selection criteria are as follows:
・Shows my characteristic art style
・Coloring method is similar
・Color usage is similar
・Face drawing style is similar
・Image quality is stable
・Few unnecessary characters or logos

Since this is an art style LoRA, I prioritized consistency in the art style over having a consistent character.

Tools used this time

I used the following environment for this project.

・anima-lora-workspace
・sd-scripts
・Codex (not necessary if you are willing to do it manually)

There is still little information available on LoRA training environments for Anima, but I was able to train it without any issues using this combination.
If you can use Claude or Codex, you should automate the work and environment setup as much as possible (even for me, a professional engineer, doing it manually is quite tough).

Training workflow

The workflow is very simple.
① Collect training images
↓
② Generate captions with WD14 Tagger
↓
③ Organize captions with Codex
↓
④ Create training settings
↓
⑤ Start LoRA training
↓
⑥ Completion

I will summarize the details of the steps starting from the next chapter.

First, set up the training environment

You might have thought, "Alright, I'll try making an art style LoRA too," but you cannot start training immediately.
First, you need to create an environment to train the LoRA.
The folder structure of the environment I used looked like this.

C:\kohya
├─ sd-scripts
└─ anima-lora-workspace

To put it simply,
sd-scripts is the training engine itself.
And anima-lora-workspace is the working folder used to manage things like:
・Training images
・Captions
・Configuration files
・Output LoRA
all in one place.

※ From here on, I will skip the explanations of what Git is and what Python is.

Install Git

First, install Git.

After installation, open PowerShell and execute the following.

git --version

If the version is displayed, you are good to go.

Install Python

Next, install Python.
In my environment, I used Python 3.10 series.

During installation,

Add python.exe to PATH

Make sure to check this.

After installation,

python --version

run this to verify.

Download sd-scripts

Next, acquire sd-scripts, which is the core training script.
I created a folder named "kohya" directly under the C drive and set it up inside that.

cd C:\kohya
git clone https://github.com/kohya-ss/sd-scripts.git

Once completed,

C:\kohya\sd-scripts

will be created.

Create a Python virtual environment

Next, create the virtual environment.

cd C:\kohya\sd-scripts
python -m venv venv

After creation,

.\venv\Scripts\activate

run this.

Enable script execution in PowerShell

Depending on your environment, PowerShell scripts might be blocked.
In that case, open PowerShell with administrator privileges and run this.

Set-ExecutionPolicy RemoteSigned

Run this. When the confirmation screen appears, enter "Y".

Install PyTorch

Next, install PyTorch.
Since my environment uses an RTX 5060 Ti, I used a newer PyTorch environment.

Because the recommended version differs depending on the GPU, please also check the official sd-scripts README for this part.

Install necessary libraries

Next, install the dependency libraries for sd-scripts.

pip install --upgrade -r requirements.txt

This will take some time.
Let's wait while having a cup of coffee.

Configure accelerate

Anima training uses accelerate.

accelerate config

Execute the following.
I used the settings below.

This machine
No distributed training
bf16

Create an anima-lora-workspace

Next, create a working folder.

mkdir C:\kohya\anima-lora-workspace

Furthermore,

configs
datasets
logs
outputs
wd14_tagger_model

Create folders such as the following. It will look like this in the end.

C:\kohya\anima-lora-workspace
├─ configs
├─ datasets
├─ logs
├─ outputs
├─ wd14_tagger_model

Get help from Codex

Codex will appear several times in this article.
Honestly,
- Creating PowerShell scripts
- Organizing captions
- Batch editing files
Doing all these tasks manually is a hassle.

So, I proceeded by giving instructions to Codex such as,
"Create a script like this"
"Batch edit the files in this folder"

In reality, what humans should do is just:
- Decide what to train
- Select training images
- Check the results
It is overwhelmingly easier to leave the environment setup in the previous chapter to AI, and it reduces configuration errors. I recommend installing Codex or Claude.

Once you have prepared this far, the environment setup is complete.
It takes a little effort at first, but once you have created it, from the next time onwards, you just need to put in the images and train.

Now, let's actually create the training data.

Create a script for WD14 Tagger

To create captions for training images, we use WD14 Tagger.
This task refers to attaching descriptive tags to each illustration, saying "this illustration is like this."
Since it was a hassle to type long commands every time, I turned it into a PowerShell script.
The file name is as follows.

tag_images_wd14.ps1

The role of this script is to:
- Load the training image folder
- Execute WD14 Tagger
- Generate txt captions for each image

The following is the file I actually had Codex create, so please use it as a reference.

Execute WD14 Tagger

Once you have placed the training images in[[phN_open]]folder[[phN_close]]execute WD14 Tagger.Run the following in PowerShell.

datasets\style_train\10_my_style

This concludes the setup.

.\tag_images_wd14.ps1

The first time you run this, it will take a little while because it needs to download the WD14 model.

Let's wait for a while.

Once it is finished,

image001.png
image001.txt

a txt file with the same name as the image will be generated, as shown in.

When you open it,

tags like these will have been automatically generated.

Tagging for 80 images was basically completed just by waiting.
You don't need to manually write tags for each image here either.
The workflow is to first have the AI apply the tags, and then organize only the necessary parts afterward.

Then, at the beginning of every txt file,

rockey

add a trigger word like this.
By using this word during generation, you will be able to call up your art style.
You should put your own name or something similar there.

I also left caption organization to Codex

Once you have created the captions with the WD14 Tagger, you don't just use them for training as they are.
However, I didn't manually edit the text files for all 80 images either.
That would be too much work. This time, I gave instructions to Codex to organize them in bulk.
All I did was give instructions to:
・Add 'rockey' to the beginning of every caption
・Delete tags unrelated to the art style

For example,

masterpiece
best quality
absurdres
rating safe
watermark

quality tags and meta-information like these are unnecessary for learning an art style, so I marked them for deletion.
As a result, organizing the captions for 80 images was completed in a few minutes.
What I felt throughout this entire LoRA creation process is thatthe approach of having humans only make decisions while leaving the actual work to the AIis a very good fit.

The actual work I performed was:
・Selecting the training images
・Giving instructions to Codex
・Checking the training results

That was about it.
I left simple tasks like caption organization and file editing almost entirely to the AI.
Because of that, I was able to create the art style LoRA with much less effort than I had expected.

Starting LoRA training

Once you have prepared everything up to this point, it is finally time to start the training.
Normally, you would have to type a long command like 'accelerate launch anima_train_network.py' every time. Doing it manually is quite a hassle.

So, I asked Codex to create a PowerShell script for training. Here is the file I created.

train_anima_style_lora.ps1

Please download this as well and use it as a reference.

In this script, I have specified the following all at once:
・Anima base model
・Text Encoder
・VAE
・Dataset settings
・Output destination
・Training parameters

(It looks at the folders specified in the previous article, so please correct the paths if you have placed them elsewhere.)
Therefore, you do not need to memorize complex commands during training.

First, open PowerShell. Press the Windows key and

PowerShell

Next, move to the training folder.

cd C:\kohya\anima-lora-workspace

Once you have moved there, execute the following.

.\train_anima_style_lora.ps1

(You can also execute it by right-clicking the file and selecting 'Run with PowerShell'.)

Once it starts normally, a large amount of logs will begin to flow in PowerShell.

Honestly, you don't need to read all of these logs.
As long as there are no errors, you are basically fine.
I was anxious at first too, but in reality, once I confirmed that training had started, I just left it alone.

The training conditions for this time are as follows:

・Training images: approx. 80
・10 epochs
・RTX 5060 Ti 16GB
The training time was about 10 hours.
I ran it before going to bed at night and checked the results the next morning.

When training finishes,

C:\kohya\anima-lora-workspace\outputs

multiple LoRA files will be output inside.
For example,

my_illustration_style_anima_lora-000001.safetensors
my_illustration_style_anima_lora-000002.safetensors
...
my_illustration_style_anima_lora-000010.safetensors

files like this.
Here, you might think,“Why are there 10 of them?”This is because in this training setting, I set it to

epoch = 10

An epoch represents
“how many times the training images have been cycled through.”In this case, I am using about 80 images.1 epoch means 80 images have been trained once,
2 epochs means the second cycle,
3 epochs means the third cycle,
and so on.

In other words,

000001

is the state after training the images once,

000005

is the state after training 5 cycles,

000010

is the state after training 10 cycles.

So, if you ask,“Is a higher number better?”that is not necessarily the case.
As training progresses, it learns the art style at first.
However, if you train too much,
・The face becomes fixed
・The composition becomes fixed
・It remembers the original images too well
phenomena like these can occur.
This is generally called “overfitting.”

Therefore, once training is finished, perform an output test using the LoRA.
I

000001
000003
000005
000007
000010

I tried comparing a few like this.
Then,
・Some with weak art styles
・Some that are just right
・Some that are slightly over-trained become visible.

In my case this time, the very last LoRA was not necessarily the best.
I actually output and compare them, and adopt the one that best reproduces my art style.
What is important in LoRA creation isnot "finishing the training" but "actually using and comparing them". So, once training is finished, I recommend loading and trying out a few different types first.

Impressions after actually using it

I tried loading the completed LoRA into Anima.

rockey,
<lora:my_illustration_style_anima_lora:0.7>

A strength of about 0.6 to 0.8 was easy to use.

If I made it too strong, the composition and face tended to become fixed. What I felt after actually using it isthe feeling of layering my art style on top of Anima.

Rather than perfectly reproducing Illustrious, I was able to regain my own style while keeping Anima's high expressive power. This was the biggest gain this time.

This method can also be applied to character LoRAs

What I created this time is an art style LoRA.
Therefore,
・Same coloring
・Same color usage
・Same atmosphere
I chose training images by emphasizing these.

However, this way of thinking can also be applied to character LoRAs. The difference is very simple.
In art style LoRA, I emphasized "aligning the art style." On the other hand, in character LoRA, I emphasize "aligning the character."

For example,
・Same character
・Various expressions
・Various poses
・Various outfits

If you collect and train images like these, you can create a LoRA dedicated to that character.
The actual training procedure hardly changes.
1. Collect training images
2. Tagging with WD14 Tagger
3. Organize with Codex
4. Start training
5. Completion
The flow is the same.

In this article, I used an art style LoRA as an example, but
if you apply this method,
・Original characters
・Fan art characters
・Your own personal characters
It can also be applied to creating LoRAs for things like these.

I will write an article about it when I have time in the future.

What I learned from trying it and summary

Here is what I felt particularly during this training.
・80 images are enough to train
・Image selection is more important than settings
・Caption organization can be left to AI
・It is better to check every epoch
・It becomes more like my own art than Anima alone

Image selection was especially important. Even with the same 80 images, if the direction is inconsistent, I think the results will change significantly.
Conversely, if you can do the image selection properly, you can leave most of the remaining work to AI.

At first, I felt that "Anima is beautiful, but it's not my art."
However, by creating an art style LoRA, I was able to achieve both Anima's expressive power and the art style I have cultivated until now.

If there is anyone who thinks like me,"I want to migrate to Anima, but I want to keep my existing art style"I recommend trying to create an art style LoRA once.
It is much easier than you think, and you can leave most of the tedious work to AI.

Bonus: Getting an AI assistant to help with environment setup and training

After reading this far, some of you might have thought, "I want to try it, but environment setup and settings seem troublesome..."
"I'm not an engineer, so I can't understand..."
So, in writing this article, I summarized the environment setup procedure and training procedure in Markdown.

The attached files are here.

These two files summarize the procedures for:
・Environment setup
・WD14 Tagger
・Caption organization
・Dataset configuration
・Sample prompt configuration
・LoRA training
・Retraining
up to this point.

How to use them is simple. Just ask the AI assistant you usually use to,

このMarkdownを読んで、私の環境向けにLoRA学習環境を構築してください。

and that is all you need to do.

For example, if you use an AI assistant that can manipulate local files such as:
・Codex
・Claude Code
・Cursor
・Cline
you can automate a significant portion of the process.

If you read this article and were able to set up your training environment and migrate to Anima, no special thanks are needed; I would just be happy if you could quietly follow me on X and share it with others.
Now, have a great AI illustration creation life.