Installation Procedures and Usage for the Streaming Partner AI System
This article summarizes the installation procedures and usage for the Streaming Partner AI System.
Please see above for an overview of this system.
Simply put, it is a system that allows streamers and VTubers to live stream with an original AI character.
Installation Procedures
First, purchase the system on BOOTH and download the .zip file.
*Please check the prerequisites and specifications in advance.
Compatible with Windows / Mac. Languages (Japanese, English, Korean, Simplified Chinese, Traditional Chinese).
Once you have downloaded the .zip file, extract it, run SPAI Setup 2.x.x.exe, and proceed with the installation to complete. (If a Windows warning appears during the process, click "More info" and then "Run anyway" to proceed without issues.)
Also, for Mac, installation is possible via SPAI-2.x.x-arm64.dmg (Apple) or SPAI-2.x.x.dmg (Intel).
You should now be able to launch the app from the location where you installed it or from your desktop.
*SPAI stands for Streamer Partner / AI
Setup and Configuration
After launching, I will explain the settings required to actually get it running.
Open the dialog from the settings gear icon in the top left of the screen.
AI Model Settings
This section allows you to select the AI model for generating AI character responses and the model for voice recognition.
For response generation, select one from GPT, Gemini, or Grok. You can use them by obtaining and entering an API key for each. (Registering multiple is also fine.)
Click here for GPT API key acquisition for reference
Click here for Gemini API key acquisition for reference
Click here for Grok API key acquisition for reference
Personally, I feel that GPT is stable, Gemini is fast in response, and Grok is unique. Please choose the one you like.
Next, the speech recognition model is used to recognize what the streamer says, and you can choose from GPT, GCP, or Grok.
For GPT and Grok, you can reuse the same API key you entered for responses for speech recognition as well. *For Gemini, a separate API key is required.
Basically, APIs are pay-as-you-go; please check the specifications and pricing structures of each provider. However, for personal use, they are generally inexpensive.
Character Settings
This is the character setting that will be reflected in the AI character's responses. Once you enter it as text and save it, the AI will behave based on this information.
I have embedded a link to a template document that serves as a reference for these setting texts, so I think it will be easier to get an idea if you use it as needed.
Visual Settings
For the AI character's visuals, you have the options of "PNGTuber (4 image variations for blinking and lip-syncing)," "Diff Motion (a full-scale PNGTuber tool)," and "VTubeStudio (Live2D)," so please set it up in your preferred style.
The easiest method that can be completed within this system is PNGTuber. Once you register and save 4 images, they will be displayed on the main screen and will lip-sync to the AI character's voice. Also, as mentioned later, you can output to OBS via Spout2 / Syphon, allowing you to display the avatar cleanly on your stream.
Regarding Diff Motion and VTubeStudio, you can switch expressions and poses in response to answers by setting up presets, hotkeys, and virtual audio cable outputs for synchronization.
Voice Settings
Regarding the AI character's voice, there are 5 types: "AivisSpeech," "VOICEVOX," and "COEIROINK," which run locally, and "ElevenLabs" and "Aivis Cloud API," which run in the cloud.
My personal criteria for choosing are as follows:
AivisSpeech: You have a GPU and want to create and use an original voice model
VOICEVOX / COEIROINK: You have a GPU and want to easily use existing character voices as they are
ElevenLabs: For languages other than Japanese. No GPU required. Use this if you want to use voice cloning or voice design.
Aivis Cloud API: Use this if you don't have a GPU but want to use original voice models, or if you have a subscription, etc.
For tools that run locally, you must launch the software in advance, and for tools that run in the cloud, an API key is required.
Also, for tools that run locally, the GPU version is recommended, so make sure to set the mode to GPU in the settings (AivisSpeech/VOICEVOX).
Furthermore, if you want to build an original voice model (such as your own clone voice), it is explained in the following article, and I can also provide a paid service if necessary, so please feel free to contact me.
Additionally, please ensure you check and comply with the terms of service for each tool, and be mindful of avoiding any legally or ethically problematic usage.
How to use the system
Now that the basic settings are complete, I will explain how to actually have AI character voice conversations at your preferred timing during your stream.
First, as a premise, this system has two modes: 'Conversation Mode' and 'Continuous Recording'.
I will explain how to use them later, but since both are operated using global hotkeys, please select the mode and configure the corresponding keys here.
Also, because they are global hotkeys, you can minimize this system's window (no screen operation is required during the stream).
How to use Conversation Mode
In Conversation Mode, you speak while holding down the push-to-talk hotkey, and when you release the key, the content is sent to the AI and a response is returned.
In other words, you can send as much information as you need at any time during the stream, allowing you to converse without interrupting interactions with viewers or other tasks.
How to use Continuous Recording Mode
In Continuous Recording Mode, once started, the AI is constantly recognizing audio, and when you press the hotkey, it responds based on what has been said up to that point.
Therefore, it is ideal for when you want a reaction based on the content of your stream.
How to use it for streaming/OBS?
When actually streaming with OBS, you need to project visuals and output audio, so I will explain how to do that.
First, regarding visuals, projection via Spout2 / Syphon is possible, so you can display your PNGTuber (avatar) cleanly in OBS with a transparent background.
*Spout2 requires a GPU. Please refer to the following for details.
Since Diff Motion and VTubeStudio can use Spout2 / Syphon within their respective software, you will project from there.
Next, I will explain the wiring for the AI's audio output.
As a prerequisite, please install a virtual audio cable called VB-CABLE.
VB-CABLE Once installed, go to your audio settings, set the "AI audio output destination" to "CABLE Input," and save.
Next, open the settings in OBS, set one of the mic inputs under "Global Audio Devices" to "CABLE Output," click "Apply," and then "OK."
The CABLE Output mic will be added to the audio mixer. Rename it to something easy to identify, like "AI Voice," and turn on the speaker and headphone icons below it so that the AI voice can be heard through OBS.
*Please note that with this wiring, you will not be able to hear the audio unless OBS is running.
Before your actual stream, please test everything to ensure there are no issues, not just with the wiring.
Conclusion
How was this guide on the installation, setup, and usage of the Streaming Partner AI System (v2)?
Compared to v1.x, the system is overall simpler and easier to understand, making it more accessible for both first-time users and regular daily use.
Please enjoy streaming with your own unique AI character, whether for a special project or as a continuous part of your content!
Also, if you have any requests for overall support or feature additions, including synthetic voice models, please contact us (this includes bug reports).
I sincerely support your creation of new streaming projects and content.
That is all, see you later 👋