Image Editing with ComfyUI Standard Nodes! How to Use SAM 3.1 and Qwen3-VL [AI Image Generation]

Introduction

Hello, I'm Kimama / Easygoing.

In this article, I will introduce image editing using standard nodes in ComfyUI.

ComfyUI is highly extensible!

Because anyone can freely create custom nodes for ComfyUI, it has unparalleled extensibility.

On the strengths of ComfyUI

Among ComfyUI custom nodes, those that are particularly useful and frequently used are increasingly being incorporated into ComfyUI's standard nodes, and official implementation has improved stability and performance.

This time, I will introduce how to use SAM 3.1 and Qwen3-VL using these standard implemented nodes.

SAM 3.1 is a region detection model

SAM 3.1 is a region detection model released by Meta.

Meta's SAM 3.1 model

The SAM 3.1 model is a model that has improved performance, mainly during video usage, compared to the SAM 3 model I introduced previously.

About the SAM 3 model

Meta's SAM 3 model required entering contact information when downloading from the public Hugging Face page, but the sam3.1_multiplex model released by Comfy Org no longer requires this, making it freely downloadable for anyone to use.

SAM 3.1 model released by Comfy-Org

Therefore, this time I will introduce how to use SAM 3.1 using ComfyUI's standard nodes.

Detecting people by instruction

Now, let's actually use the SAM 3.1 model to remove the background from the following illustration.

Workflow

Model

• checkpoint: sam3.1_multiplex_fp16.safetensors

First, since the sam3.1_multiplex_fp16.safetensors model released by Comfy Org is distributed in Checkpoint format, place the downloaded model in the same location as your Checkpoints.

For ComfyUI

• ComfyUI\models\checkpoints

For Stability Matrix

• Models\StableDiffusion

Then, load the SAM 3.1 model using the Load Checkpoint node.

Next, connect the CLIP Text Encode node just like in a standard image generation prompt and enter the target you want to detect.

Following that, connect to the model and conditioning of the SAM3 Detect node. Note that since the SAM 3.1 model does not include a VAE, there is no need to connect the vae.

With this, we were able to detect the person using the SAM 3.1 model.

Remove the background other than the person!

Currently, the person detected by SAM 3.1 is selected, but since we actually want to remove the opposite side, which is the background, we invert the area using the Invert Mask node.

Then, by inputting the first image and the background mask area into the Join Image with Alpha node, the background becomes transparent.

Here, the Alpha in the node name refers to the Alpha Channel (transparency channel).

With this, we were able to cut out the background to make it transparent from the illustration using the SAM 3.1 model.

Qwen3-VL can also understand images!

In the second half, I will introduce how to use the Qwen3-VL model.

The Qwen series is a Alibaba released Large Language Model (chat AI).

• Large Language Model: Understands text

• Vision Language Model: Understands images and text

Qwen3-VL is a model that adds image recognition capabilities to the large language model Qwen3, allowing it to understand images in addition to text.

Due to its high performance and open license, the Qwen series is increasingly being adopted as a text encoder (AI that parses prompts) for AI image generation, and the Qwen3-VL model is used as the text encoder for the Krea 2 model.

About the Krea 2 model

Describe images with Qwen3-VL!

Now, let's try adding descriptions (captions) to images using the Qwen3-VL model.

I will try it with the anime illustration from earlier with the background removed.

Workflow

Model

• qwen3vl_4b_bf16.safetensors

• qwen3vl_4b_INT8_ConvRot_HQ.safetensors(Lightweight version created by the author)

First, when loading the Qwen3-VL model, use the Load Clip node, just like with a standard text encoder.

Then, input the clip and image into the Generate Text node and write your instructions in the Prompt.

Anime illustration of a woman wearing purple clothes

Using the Qwen3-VL model, I was able to create a caption that also describes the colors and atmosphere.

Creating prompts with Qwen3-VL

Next, let's use the Qwen3-VL model to create an image generation prompt from another image.

First, apart from the previous illustration, let's try creating a caption using the following photo of a sunflower field.

Photo of a sunflower field

Next, I will combine the captions created from both images, add the quality prompt that I always use, and generate a prompt with Qwen3-VL.

Here, the instructions given to Qwen3-VL are taken directly from those used in the prompt enhancer of the official Krea-2 workflow.

Generating an image with the completed prompt

Now, let's try generating an illustration using the Krea-2-Turbo_clear model with the completed prompt.

Input images

Output image

Prompt

With this, I was able to combine the two input images to generate a completely new illustration.

The workflow used this time is attached at the end of the article.

No need for Ollama or LM Studio!

Previously, when using large language models like Qwen in ComfyUI, it was necessary to simultaneously launch and synchronize external tools such as Ollama or LM Studio.

These tools naturally compete for VRAM with ComfyUI, making operations sluggish, and frequent crashes due to memory shortages made stable operation difficult.

With ComfyUI now supporting models like SAM 3.1 and Qwen through standard nodes, model movement following ComfyUI's algorithms has become possible, which I believe has greatly improved stability, performance, and convenience for many users.

It can be said that ComfyUI is evolving into the ultimate generative AI tool that can handle everything on its own.

Is ComfyUI going to swallow up custom nodes?

However, I also hold another perspective.

Many of the excellent features that currently support ComfyUI, such as SAM 3, Qwen, and INT8_ConvRot, were originally implemented first by the custom node community.

ComfyUI-INT8-Fast custom node creator BobJohnson24 commented the following at the beginning of their repository:

I feel that as Comfy Org extracts excellent ideas from the custom node community and "optimizes performance" by eliminating compatibility with tools other than ComfyUI in order to aim for hegemony in generative AI tools, the stance of cooperating with the community that the company once advocated is fading.

2025.11.5 Comfy Org blog post

Of course, since many of ComfyUI's custom nodes are released under open licenses such as MIT or Apache-2.0, there are no legal issues with the ComfyUI core using them, and integration into the ComfyUI core might even be an honor for a custom node creator.

As a witness in 2026, I would like to watch how ComfyUI changes from a community tool into the ultimate commercial tool.

Workflow

Finally, I will introduce two of my go-to workflows using SAM 3.1 and Qwen3-VL.

SDXL → Upscale → Detailer

Model

• checkpoint: mellow_pencil-XL_clear_v2.safetensors

• checkpoint: sam3.1_multiplex_fp16.safetensors

• upscale_model: 4x_IllustrationJaNai_V1_ESRGAN_135k.safetensors (non-commercial)

Custom Nodes

• ComfyUI-Impact-Pack

Generate one illustration from two images using Krea-2-Turbo_clear (the workflow for this article)

Models

• checkpoint: sam3.1_multiplex_fp16.safetensors

• text_encoder: qwen3vl_4b_bf16.safetensors

• diffusion_model: Krea-2-Turbo_clear_BF16.safetensors

• vae: qwen_image_vae.safetensors

• upscale_model: 4x_IllustrationJaNai_V1_ESRGAN_135k.safetensors (non-commercial)

Summary: Let's Try Using SAM 3.1 and Qwen3-VL

• SAM 3.1 and LLM features integrated into the core ComfyUI

• Improved stability and performance

• Evolving into the ultimate generative AI tool

In this article, I introduced how to use the SAM 3.1 and Qwen3-VL models using ComfyUI's standard nodes.

ComfyUI accurately senses user needs, expands its basic functionality, and has made a major leap forward as a professional-grade tool.

ComfyUI will continue to evolve significantly as an indispensable tool for many creators, including myself.

Thank you for reading until the end!

Update History

2026.8.10

• Changed the workflow's upscale model from pth format to safetensors format

English Article