How to convert safetensors to GGUF on Windows ＆ Uploaded a new json of Hunyuan Video i2v
About flux tools
Previously, when converting Flux1.dev format fp16.safetensors to GGUF,
(Although I often used fp16 as is, so I didn't have many opportunities to use it outside of testing)
I used a program called Flux Tools, which is included as an accessory tool in the Forge-compatible WebUI called EasyForge, introduced by Easygoing in the article below.
(Specific usage instructions are explained below)
Recently, I needed to use the GGUF format of SkyReels-V1, which seems to be a tuning format for Hunyuan Video, and since the aforementioned Flux Tools naturally cannot support it, I had to create a new execution program.
llama.cpp
Therefore, first of all, based on the article below by shiba*2, who is essential to follow regarding technology in this field, I started by building a llama-cpp environment that runs on Windows.
First, as shown in the figure below, execute the download via git clone while the path is set in an arbitrary folder.
git clone https://github.com/ggerganov/llama.cpp
Next, as shown in the figure below, create a virtual environment in the created folder and perform activation.
cd llama.cpp
python.exe -m venv venv
venv\scripts\activate
Next, as shown in the figure below, execute the following.
pip install -e ./gguf-py
Next, as shown in the figure below, let's also update pip.
python.exe -m pip install --upgrade pip
Next, as shown in the figure below, perform the installation of torch and safetensors.
Regarding this, since it operates at high speed even if it is not for CUDA, I am installing the version for CPU operation rather than the version for CUDA which consumes a lot of size.
pip install torch safetensorsNext, please download and install the Windows installer for CMake from the following link.
Next, access the article below by shiba*2 again, and download two of the three files published in the article, convert.py and lcpp.patch, and save them directly under the llama.cpp folder created so far.
Next, download the two files below and save them in the same folder. These are batch files for performing the conversion process based on the fix_5d_tensors.py published by shiba*2 in the article above, with minor changes I made to parts such as path specifications.
Next, access the article below by shiba*2 again, and as described, use the Load Diffusion Model node and SaveModel node of ComfyUI to re-save the SkyReels bf16.safetensors file. Note that the re-saved file will be output to the output folder.
Next, save the SkyReels safetensors file that was re-saved via the "SaveModels" node on ComfyUI as per the article above into the llama.cpp folder as shown in the figure below, and change the name to "base.safetensors".
When you execute the start.bat file in the same folder, the conversion will start as shown in the figure below. In the method explained here, since it is converted to the BF16 format, the size is hardly compressed. If you use the MultiGPU node described in the article below, the dedicated VRAM consumption will always be less than 12GB, so I create it in the BF16 format, which has the highest quality.
(Since the MultiGPU node only supports the GGUF format, there is meaning in quantizing the safetensors format itself, even if it is not compressed)
When this process is completed, two files named "base.gguf" and "base2.gguf" will be created as shown in the figure below. Since two bf16 format files are created, a storage size of 50GB or more is required.
Since the file to be used is the "base2.gguf" file, delete the "base.gguf" file and the fix_5d_tensors_hyvid.py file created at the same time, and change the file name of the base2.gguf file to an arbitrary name.
2nd i2v on hunyuan video
I have released the new version of the Hunyuan Video i2v json that I previously announced. It is capable of creating much more movement than the 1st json. I have also made some minor adjustments to the 1st json.