Generating a 30-Second Video in 28 Seconds: A Complete Local AI Optimization Guide

Generating a 30-Second Video in 28 Seconds: A Complete Local AI Optimization Guide

Introduction

Using only an RTX 5090 and local AI tools, I built and optimized a pipeline to generate 30-second narrated videos with zero API costs.

The final result: generating a 30-second video in 27.86 seconds — faster than real-time.

Environment

| Component | Details |
|-----------|---------|
| GPU | NVIDIA RTX 5090 |
| OS | Windows 11 + WSL2 (Ubuntu) |
| LLM | Gemma 3 12B (llama.cpp) |
| Image Generation | Stable Diffusion WebUI |
| Voice Synthesis | VoiceVox |
| Video Encoding | FFmpeg + NVENC |

Before & After Comparison

Processing Time Evolution

| Version | Image Generation | Total Time | Improvement |
|---------|------------------|------------|-------------|
| Initial (Euler 25steps) | 39.4s | 56.7s | - |
| Intermediate (Euler 20steps) | 31.2s | 46.2s | -18% |
| Final (LCM 4steps) | 16.3s | 27.9s | -51% |

Optimization by Stage

| Stage | Before | After | Improvement |
|-------|--------|-------|-------------|
| Scenario Generation | 3.1s | 3.1s | - |
| Image Generation (10 images) | 39.4s | 16.3s | -59% |
| Narration (10 clips) | 4.8s | 0.9s | -81% |
| Zoom Video Generation | 50s | 3.9s | -92% |
| Final Composition | 5.5s | 5.6s | - |

Key Optimizations

1. Image Generation Speedup

Problem: Image generation was 74% of total processing time

Solution:

• Switch to LCM sampler (Euler → LCM)

• Reduce steps (25 → 4)

• Adjust CFG Scale (7 → 1.5)

# Before
steps: 25, sampler: Euler a, cfg_scale: 7
→ 3.9 seconds per image

# After
steps: 4, sampler: LCM, cfg_scale: 1.5
→ 1.6 seconds per image

2. Parallel Processing

Problem: Sequential processing caused idle time

Solution:

• Image generation: 5×2 batch parallel processing

• Narration: 5-way parallel processing

• Zoom videos: 5×2 batch with NVENC

Benchmark Results:

| Parallel Mode | 10 Images Time |
|---------------|----------------|
| Sequential | 39.4s |
| 10 full parallel | 18.8s (GPU contention) |
| 5×2 batch | 16.3s (optimal) |

3. GPU Encoding

Problem: CPU processing was a bottleneck

Solution:

• Use NVENC (h264_nvenc) in FFmpeg

• Preset p4 for speed/quality balance

ffmpeg -c:v h264_nvenc -preset p4 -b:v 6M -pix_fmt yuv420p

4. VoiceVox Parallelization

Problem: Sequential processing with unnecessary sleeps

Solution:

• ThreadPoolExecutor with 5 workers

• Remove sleep calls

# Optimized
with ThreadPoolExecutor(max_workers=5) as executor:
    list(executor.map(generate_audio, texts))

Final Optimized Settings

Image Generation:
  sampler: LCM
  steps: 4
  cfg_scale: 1.5
  parallel: 5×2 batch

Narration:
  parallel: 5 workers

Video Encoding:
  encoder: h264_nvenc
  preset: p4
  fps: 60
  parallel: 5×2 batch

Sample Generation Timeline

| # | Theme | Generation Time |
|---|-------|-----------------|
| 1 | Tokyo Morning | ~3 min |
| 2 | Kyoto Seasons | ~3 min |
| 3 | Japanese Festival | ~2 min |
| 4 | Mount Fuji | ~2 min |
| 5 | Japanese Castles | 57s |
| 6 | Japanese Gardens | 46s |
| 7 | Japanese Onsen | 35s |
| 8 | Shinto Shrines | 35s |
| 9 | Japanese Sea | 28s |

Conclusion

Achievements

• ✅ 30-second video generated in 28 seconds (faster than real-time)

• ✅ Zero API costs (100% local processing)

• ✅ 85% reduction from initial processing time

Key Learnings

1. Bottleneck analysis is crucial — Image generation was 74% of the workload

2. Right-size parallelism — 5×2 batch outperformed full 10-way parallel

3. LCM sampler is powerful — Dramatic speedup with quality retention

4. Maximize GPU utilization — NVENC for video encoding acceleration

Future Improvements

• LCM-LoRA for better quality

• 4K resolution support

• Multi-GPU scaling

Environment: RTX 5090 / WSL2 / Gemma 3 12B / Stable Diffusion / VoiceVox / FFmpeg

#AIVideoGeneration #LocalAI #StableDiffusion #LCM #RTX5090 #NVIDIA #VoiceVox #FFmpeg #NVENC #VideoProduction #AIAutomation #MachineLearning #DeepLearning #GPUComputing #ParallelProcessing #Optimization #RealTimeGeneration #Gemma #LlamaCpp #WSL2 #ZeroCost #APIFree #IndieHacker #Tech #Programming #Python #VideoEditing #TextToSpeech #ImageGeneration #GenerativeAI #OpenSource #SelfHosted

Hashtags

#AIVideoGeneration #LocalAI #StableDiffusion #LCM #RTX5090 #NVIDIA #VoiceVox #FFmpeg #NVENC #VideoProduction #AIAutomation #MachineLearning #DeepLearning #GPUComputing #ParallelProcessing #Optimization #RealTimeGeneration #Gemma #LlamaCpp #WSL2 #ZeroCost #APIFree #IndieHacker #Tech #Programming #Python #VideoEditing #TextToSpeech #ImageGeneration #GenerativeAI #OpenSource #SelfHosted