Trying out FLUX.1 on Google Colab

I tried out "FLUX.1" on "Google Colab" and have summarized it here.

1. FLUX.1

"FLUX.1" is the latest image generation AI model announced by "Black Forest Labs," which was founded by the developers of "Stable Diffusion."

2. FLUX.1 [dev]

The procedure for running FLUX.1 [dev] (guidance-distilled model) on "Google Colab" is as follows.

(1) Install packages.

# パッケージのインストール
!pip install git+https://github.com/huggingface/diffusers.git

(2) Model cardClick "Agree and access repository" on the.

(3) Log in to HuggingFace.

!huggingface-cli login

Follow the instructions to enter the following information.

(4) Prepare the pipeline.
Download "black-forest-labs/FLUX.1-dev".

import torch
from diffusers import  FluxPipeline

# パイプラインの準備
pipe = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev", 
    torch_dtype=torch.bfloat16
)
pipe.enable_model_cpu_offload()

(5) Prepare the prompt and execute image generation.

# プロンプトの準備
prompt = "cute cat ear maid of japanese anime style"

# 画像生成の実行
out = pipe(
    prompt=prompt, 
    guidance_scale=3.5, 
    width=1024, 
    height=1024, 
    num_inference_steps=50,
    max_sequence_length=256,
).images[0]
out.save("image.png")

It took 56 seconds for 50 steps. The memory consumption is as follows.

3. FLUX.1 [schnell]

The procedure for running FLUX.1 [schnell] (timestep-distilled model) on "Google Colab" is as follows.

(1) Install packages.

# パッケージのインストール
!pip install git+https://github.com/huggingface/diffusers.git

(2) Preparing the pipeline.
Download "black-forest-labs/FLUX.1-schnell".

import torch
from diffusers import  FluxPipeline

# パイプラインの準備
pipe = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-schnell", 
    torch_dtype=torch.bfloat16
)
pipe.enable_model_cpu_offload()

(3) Preparing the prompt and executing image generation.

# プロンプトの準備
prompt = "cute cat ear maid of japanese anime style"

# 画像生成の実行
out = pipe(
    prompt=prompt, 
    guidance_scale=0., 
    width=1024, 
    height=1024, 
    num_inference_steps=4, 
    max_sequence_length=256,
).images[0]
out.save("image.png")

It took 34 seconds for 4 steps. The memory consumption is as follows.

4. Comfy UI

For how to use "Comfy UI" with "FLUX.1", the following article was helpful.

cd "/Users/<ユーザー名>/Library/Application Support/StabilityMatrix/Packages/ComfyUI"
source venv/bin/activate
pip install torch==2.3.1 torchaudio==2.3.1 torchvision==0.18.1

Next time