Fixed triton3.2.0 ＆ WaveSpeed on Inductor backend finally (but not better nodes)

Loaded triton 3.2.0 on ComfyUI

Last month, after struggling extensively, I finally managed to get the WaveSpeed Model Compile+ node running with a compromise using the cudagraphs backend. Of course, I knew that functionally, the officially recommended Inductor backend would offer better performance.

However, things hadn't been working out until now, and even though I could get it to finish running with Inductor, a close look at the command line revealed errors, meaning torch.compile wasn't actually working.

I knew that if I used the recently released triton 3.2.0 for Windows, the Inductor backend, which was impossible with 3.1.0 or earlier, would finally work correctly.

However, using this 3.2.0 caused other issues, leading to errors in several important Custom Nodes in ComfyUI. I couldn't sacrifice them just for the sake of torch.compile, so I was forced to stick with the compromise of using cudagraphs until today.

In the meantime, a path toward acceleration via MultiGPU nodes had opened up from a different angle, so I was actually proceeding with settings in the direction of removing torch.compile and combining it with MultiGPU nodes instead...

Today, I finally succeeded in loading triton 3.2.0 on ComfyUI without errors, and as shown in the figure below, I succeeded in running the Model Compile+ node with the officially recommended Inductor backend.

Tips1

The breakthrough came from the following article, which I hadn't just seen for the first time today.

To put it briefly, the breakthrough was achieved by extracting the include_libs.zip file mentioned in the article and placing it in the python_embeded folder.

Actually, I had already tested this back when I was using 3.1.0, but with 3.1.0, Inductor wouldn't work even if I installed this.
(Even if the node finished running, a close look at the command line showed errors and it wasn't optimized)

So, since it didn't make a difference whether it was there or not when using cudagraphs, I erased it from my memory.

However, today, while analyzing the error messages that occur when loading triton 3.2.0 with Cursor, the following message appeared, and I remembered, 'Wait? I have a memory of seeing the word 'include' somewhere related to triton,' which led to the solution.

triton.hファイルて何処にあるかわかるか ?

triton.hファイルは通常、Tritonのインストールディレクトリの中の以下のようなパスに存在するはずです：

D:/USERFILES/ComfyUI/python_embeded/Lib/site-packages/triton/backends/nvidia/include/triton/

または：

D:/USERFILES/ComfyUI/python_embeded/Lib/site-packages/triton/include/

確認するには、以下のPythonコードを実行して探すことができます：

Yes, it refers to the folder to be installed in addition to the whl, which is introduced in Mr. Nishikawa's article above. The path in the Cursor response above is incorrect, but seeing the folder name 'include' gave me a flash of inspiration—or rather, all the associations connected.

'Ah, I see, so 'that thing' from Mr. Nishikawa's article back then is what's needed here.'

However, the link in the article above is for Python 3.11.x, so it cannot be applied to ComfyUI running on 3.12.x.

I obtained the version for Python 3.12.x from the following.

I will be updating the torch.compile settings article above, along with the WaveSpeed-related articles and all the json files I've published, after this. Oh boy, I have to change all the WaveSpeed-equipped versions I've already released.
(Furthermore, there is an important follow-up regarding the content of this 'update,' so please read this article to the end.)

Speed tests

Regarding speed, I tested it while comparing it with the MultiGPU node combination type, but...it is quite fast from the second run onwards, especially with Flux1.

As explained in the article below, this is the second run and beyond, where I have removed Model compile+ from WaveSpeed and applied Distorch via MultiGPU to the Dual Clip.

The figure below shows the second run and beyond of the full-spec WaveSpeed using the Inductor backend. I have tested it multiple times, and WaveSpeed still has the edge. Especially from the second run onwards... this is expected due to the nature of torch.compile.

Advanced tips

On the other hand, for the Detailer node, it seems that the Inductor backend cannot be applied as a specification with the WaveSpeed Model Compile+ node, so one countermeasure is to use the cudagraphs backend and reduce-overhead mode as before.

Another countermeasure is to use torch.compile via KJNodes. For some reason, with this node, the Inductor backend works properly even for the detailer.

Fixing torch.compile on KJNodes

The reason I am fixated on triton in the first place is one thing... to run torch.compile, which is a powerful acceleration method in ComfyUI—or rather, one of its components—using the Inductor backend, which unleashes its maximum performance.

As mentioned earlier, the technology of torch.compile itself is not exclusive to WaveSpeed, and KJNodes also publishes similar nodes.

About KJNodes

Now, this is important: having tested both WaveSpeed and KJNodes torch.compile nodes recently, I feel that at least for now, KJNodes is superior.

First, the biggest reason is that with KJNodes' torch.compile, as shown in the figure below, the max-autotune feature in the Inductor backend, which currently delivers the best performance in torch.compile, works properly.

With WaveSpeed's Compile Model+ node, as mentioned earlier, the Inductor backend now works normally, but mode settings are still not possible. If you specify max-autotune or similar here, it will cause an error and fail to complete.

Although max-autotune requires processing time for compilation during the initial load, it definitely shortens the inference time... that is, the sampler processing time.

This characteristic is effective when performing generation with Flux1 on a GPU in the RTX4070 12GB class, like mine, where processing exceeds the 12GB dedicated VRAM.

I recently upgraded my core components from DDR4 to DDR5, which also provides benefits, but when you can shorten the subsequent inference time even if compilation takes longer, this max-autotune is effective for processes like Flux1 that exceed 12GB of dedicated VRAM.

In fact, within the scope of my repeated tests, I have found that running the Inductor backend with max-autotune using the KJNodes node is definitely faster than WaveSpeed's Compile Model+.

Based on this, since I succeeded in loading triton3.2.0 without errors this time, I was planning to update all the WaveSpeed-compatible workflows I have published in the past, but I have decided to change my plans slightly and adopt KJNodes for the torch.compile part.

Advanced tips

However, there are exceptional cases where I use the WaveSpeed version of the Compile Model+ node. The cause is unknown, but depending on some JSON configurations, torch.compile may not work with KJNodes.

If you encounter an error with the KJNodes version, please try the WaveSpeed version.