Running MiniMax H3 on my home DGX Spark to convert live-action video into anime
Running MiniMax H3 on my home DGX Spark to convert live-action video into anime
On August 3, 2026, MiniMax released the open weights for their video generation model, "H3".
I run a pipeline on my home DGX Spark that replaces myself in live-action vlog videos with a character. Until now, I have been switching between two models: Wan2.2 Animate and SCAIL-2. I spent the entire day of the release testing whether H3 could fit into this workflow, and here are the results.
In conclusion, the character reproducibility clearly surpassed existing models. However, I arrived at an answer different from what I expected regarding the part about "how to control the pose."
What is MiniMax H3?
It is a model that takes text, images, video, and audio into the same context to generate video with stereo audio in one go. It does not add sound later; it generates the video and audio simultaneously.
The local execution configuration looks like this.
Simultaneous input of up to 9 reference images, 3 reference videos, and 3 reference audio clips
Output is up to 2K, 24fps, and about 15 seconds (the local version is capped at 768px on the short side)
The int8 version of the DiT is 21GB, and the nvfp4 version of the Qwen3-VL-32B text encoder is 15.7GB
The license allows commercial use for companies with annual revenue under $20 million
ComfyUI provided native support on the day of release (Kijai's implementation was merged that same day, and v0.30.0 was released), so installation was just a matter of downloading the model and placing it in the designated folder.
What is important in this flow is that H3 has no input port for poses. It is not a mechanism like Wan2.2 Animate that passes a skeleton to move; it uses a method of "reading movement from a reference video." I would struggle with this later.
Trying it out first
I tried passing one reference image of my original character and a selfie video I took in my room. I did not use any LoRA.
This was the first attempt. Blue-gray hair, yellow tips, deadpan eyes with one eye hidden by bangs, white cable-knit vest—the character's design was almost perfectly reproduced from a single reference image.
With the SCAIL-2 model I had been using, I had to train a dedicated character LoRA and apply it at a strength of 0.4 to 0.7 to achieve this level. I am honestly surprised that it surpassed that with just one reference image.
The background also maintains the layout of the room (the positions of the microwave and blinds). However, this is not a state where "live-action pixels remain," but rather a state where it has been "redrawn with a similar composition." This point also becomes important later.
Adjusting the art style
In the initial state, the art style was closer to cel-shading, so I increased the reference image resolution setting (set ref_image_size to max) and specified the pencil sketch texture in the prompt.
Left is the initial setting, right is after adjustment (seed is the same).The pencil hatching was transferred across the entire ceiling and walls, resulting in a texture like animation drawn on paper. The processing time is almost unchanged. If the reference image is small, the cost of the max setting is almost negligible.
Testing with actual vlog footage
I verified this with three clips under different conditions: outdoors in strong sunlight, drinking from a bottle under an elevated bridge, and walking away at a shrine at night.
The character does not break down even in bright outdoor conditions. In SCAIL-2, this was a condition that resulted in semi-realism unless the LoRA strength was increased.
The action of holding and drinking from a bottle, and even the text on the background signs, were preserved.
Even in a full-body shot at night, the walking motion was maintained while keeping the foreground beams and lanterns.
And what is worth noting is thatlong hair can be drawn freely. The replacement mode of Wan2.2 Animate had a structural constraint where it could only draw inside the mask, making it physically impossible to draw hair longer than the original person. H3 is a full-frame regeneration without the concept of a mask itself, so this constraint becomes zero.
Returning to the live-action background
Since H3 regenerates the background, if you want to use the live-action background as is, you need to extract only the character and composite it. For this part, I was able to reuse the existing pipeline (SAM3 for seed mask -> MatAnyone2 for temporal propagation) as is.
Processing 124 frames took 16 seconds. The fine silhouettes of the hair tips and the pencil outlines were preserved, and the matte quality is sufficient.
Main topic: How to control posture
This is the part where I spent the most time this time.
The weakness of H3 is thatpose tracking is loose. Because it is a method of "reading" movement from the reference video, it does not reach the level of Wan2.2 Animate, which strictly traces the skeleton. In fact, in the walking scene, the character would sometimes almost stand still.
So I tried three methods in order.
Plan 1: Pass the skeleton video as a reference video -> ✗
I extracted the skeleton with ViTPose and tried passing the stick figure video as a reference.
The result wasa complete failure. It ignored the position indicated by the skeleton and the orientation of "view from behind," and turned into a different composition closer to the face.
Left is the raw state without reference, center is skeleton only, right is "live-action + skeleton (two inputs)." The skeleton alone is useless, but it improves slightly when passed together with the live-action.
The reason is clear: "stick figure videos" are not in H3's training distribution. Inputs outside the distribution are treated only as "loose inspiration."
Plan 2: Use Wan Animate output as the reference video → ◎ ...or so I thought
So, I had the idea that if it's a 'video of an anime character moving,' it should be within the distribution, so I passed a character video with a gray background generated by Wan2.2 Animate as the reference video for H3.
The left is the Wan output, and the center and right are the H3 outputs. It followed the pose to the level of matching even the hunched-over posture. It was just as hypothesized.
I thought this solved it—but when I tried it with actual footage, problems started to appear.
The pitfall of Wan-driven generation
The left is the conventional pipeline, and the right is the Wan-driven H3. At first glance, it looks better, but if you look closely, H3 is faithfully reproducing the mistakes on the Wan side.
In the shrine cut, the depiction of the back hair in Wan is broken → The H3 output is broken in the same way
The white bottle seen in the live-action footage becomes ambiguous in the Wan output → H3 also conforms to Wan, and the small items from the live-action footage are lost
In other words, the high-precision H3 is using the low-precision Wan as a model, which is putting the cart before the horse. Even though the poses match, the details degrade.
Plan 3: Multi-view character reference → ◎ This is the correct answer
So I changed my approach. Instead of changing the motion source, I strengthened the reference image side.
H3 can accept up to 9 reference images. I dug up viewpoints other than the front from my dataset and created a 4-image composition.
Front, left profile, 3/4 back (closer to the back), and right profile. I explicitly state in the prompt, 'The third image is a reference for the hair when viewed from behind.'
The left is a single reference image, and the right is a multi-view reference. While a single image would turn into 'turning around and looking at the camera,' with the multi-view reference, the back view was maintained, and the back hair was also drawn correctly. The orientation problem was solved without going through Wan at all.
It also had an effect in crowd scenes.
A back view of walking through a crowd at a summer festival. While a single reference would result in the foreground character not being replaced and just passing through, with the multi-view reference + the explicit instruction to 'replace only the foreground character,' it was replaced correctly. The surrounding people, lanterns, and stalls remain as they were in the live-action footage.
Extra: Feeding live-action directly into latent (failed)
As a final resort to completely remove Wan, I also tried the method of using the live-action video as the initial latent for H3 and running the generation from the middle (equivalent to img2img in image generation). I have already confirmed by reading the implementation that it can be wired using only standard ComfyUI nodes.
The result was a failure. At denoise 0.6, although the structure and the bottle are perfectly preserved, the art style remains live-action and does not become anime. Conversely, if you raise it, the background drifts. The trade-off between structure preservation and style conversion is strong, and I couldn't find the perfect middle ground. There is still room to refine this.
Conclusion on pose control
Method Pose Strictness Side Effects Verdict Multi-view reference + live-action reference video Medium (position, orientation, and silhouette match) None First choice Wan Animate output as reference video High Inherits all of Wan's errors Only when movement is intense Live-action + skeleton (two inputs) Medium-low None Auxiliary only Skeleton video only ✗ Composition gets corrupted Discarded Live-action latent injection High Style does not change Pending
The correct approach was to shift my mindset from "strictly matching the pose" to "it is sufficient if the orientation and silhouette are correct."
Since multi-view referencing does not go through Wan, there is no source for inheriting errors in the first place. Because the most effective parts—the back view and the back hair—are resolved on the reference image side, the animation does not break down even if the motion details are somewhat loose.
Come to think of it, this is obvious; people watching anime are not looking to see if the arm is moving at the exact same angle as the live-action person. They are looking to see if the character looks correct. It was a case of targeting the wrong thing to control.
Lip-syncing is also possible
Since H3 generates audio simultaneously, you can match the mouth to your own audio.
I created an 8.7-second line of dialogue using a locally running TTS (irodori-tts) and passed it as the reference audio for H3. In the prompt, I explicitly stated, "Use this audio as is," "Completely ignore the mouth movements in the reference video," and "Synchronize the mouth to this audio."
The audio waveform envelope correlation was 0.983 — the input audio was maintained almost exactly as is
Changes in mouth shape, blinking, and natural head movements are also generated
There were many reports in the community that "input audio is not maintained" or "lip-syncing does not match," but I think it is highly likely that this can be solved by how you write the prompt. It was important to follow the format in the official prompt guide faithfully.
Speed: Reduced to 8 minutes per clip
In the initial state, a 5-second clip took 21 to 22 minutes. I verified the speed-up know-how shared in the community one by one using the same seed.
Configuration 5-second clip Image quality Default state (20 steps) 21-22 minutes Baseline +EasyCache 14 minutes 18 seconds Difference is at the level of diffusion fluctuation +SageAttention 2.2 12 minutes 31 seconds Same as above +15 steps 8 minutes 14 seconds Visually equivalent
Total 62% reduction, no degradation in image quality. The image above compares the before and after optimization using the same seed, and the difference is at a level where you cannot tell.
By the way, the effect of SageAttention was about 12.5% in my environment, compared to the "about 2x" reported by the community. Is it the difference in the ARM + Blackwell configuration, or is the original implementation fast to begin with? This kind of "other people's numbers don't show up exactly" is a common occurrence in home operations.
Power consumption is a maximum of 67.9W. Since the GB10 has a chronic issue where power protection kicks in and it suddenly shuts down under high load, I am operating it with the clock limited to 2100MHz, but there was plenty of headroom before reaching the danger zone (82-95W).
Summary
Here is the conclusion after running it for a day.
【基本】多面キャラクター参照（正面/左横/右横/背面寄り）＋ 実写を参照動画
→ MiniMax H3（15ステップ + EasyCache + SageAttention）
→ マット抽出 → 実写背景へ合成
【例外】走る・踊るなど動きが激しいカット
→ Wan2.2 Animate（骨格駆動）→ H3の参照動画として渡す
H3 clearly outperforms existing models in character reproducibility, image quality, and style transfer
You can freely draw characters with silhouettes that differ significantly from the original person, such as those with long hair.
Place an existing Wan pipeline in the front stage only for scenes that require strict pose adherence.
Everything, including audio and lip-sync, is completed within H3.
8 minutes for a 5-second cut. It's within a practical range.
It was thrilling that the model, released on its first day, was running in ComfyUI by the end of that same day, solving both of the existing pipeline's weaknesses (the long hair issue and the learning cost of character LoRAs).
The remaining challenge is reference images from directly behind. I am currently substituting with a 3/4 back view, but if I prepare a complete back view, the accuracy of the back hair should improve by another level. Aside from that, I want to refine the V2V process where live-action is fed in directly.
If a distilled LoRA (additional training to enable generation in fewer steps) is released, I think it will become a world of just a few minutes. As of the day after the release, it doesn't exist yet, so that's something to wait for from the community.
※Updated on 8/6
Currently verifying various things. Content that overturns previous verifications has emerged, so I would like to write about it again at a later date.
MiniMaxH3いろいろテスト pic.twitter.com/GxPTj8qusm
— かどみはる (@cdcadmi) August 5, 2026
Verification environment: DGX Spark (GB10 / 128GB unified memory / ARM64 / Blackwell), ComfyUI v0.30.0, MiniMax H3 ref2va int8 + Qwen3-VL-32B nvfp4