I Tried Automatically Generating an Audiobook Problem Set - Generating Audiobooks with AI x VOICEVOX x Python

Hello, this is Rcat.
In my previous article, I created a problem set fully automatically.
Now, the next step is to turn this into an audiobook by having it read aloud.

If you're wondering what this problem set generation is, please check out my previous article first.

Introduction

Terms of Service

Please check the terms of service in advance when using information or works.

Regarding Comments

Please check the terms of service guidelines before posting a comment.

Overview

Summary of Previous Articles

You end up memorizing certification exam study problem sets after doing them a few times, right?
That's when I had the idea that if I feed the exam scope and keywords to a generative AI to create mock questions, I could generate an infinite number of problems.

At first, I created it as a chatbot, but there were issues like it not working well with a poor network connection, and it was a bit difficult to use because of the few seconds of wait time for generation.

So, in the previous article, I output the problems as a PDF file so they could be read as an e-book.

What I'm Doing This Time

Now, from here, I thought if I change the output part from PDF to audio, I could create an audiobook.

For example, the following article uses speech synthesis software to fully automatically dub videos. I just need to read the problem text aloud using the same method.

Final Form

First, I'll show you what it's like.
Well, since it's audio, I uploaded it to YouTube. If you are in an environment where it's okay to listen, please give it a try.
*It's for the G-Certification

How to Use

How to Download

I will update the content distributed in the previous article with this new material.
Please check the distribution information at the bottom.

Operation Screen and Settings

I have added dedicated settings and an output button to the main screen of this application.
Configure the following items and press the Create Audiobook button to generate the audio.

• Character
  Select the character to use for reading aloud.
  Since this application uses VOICEVOX for reading, please download and launch it in advance. For details, please check the article on fully automatic dubbing.

• Speech Speed
  This is the speed of the reading. Please try generating it once to get a feel for it before adjusting the settings.

• Answer Transition Time
  This is the interval between the end of reading the problem and options, and the start of reading the answer explanation.

• Merge
  Select whether to output the reading files one question at a time or as a single audio file combining all questions. Choose the method that is easiest for you to use.

How to View Output Data

The output location changes depending on whether you choose to merge or not.

When Merging
When merging, the combined file will be created in the Data folder located in the same folder as this tool.

When Not Merging
When not merging, a Data folder will be created inside the folder containing this tool, and inside that, there will be a folder named with the voice box and date. Individual audio data files are stored inside, so please play them from there.

Output Result Waveform

If you cannot listen to the audio, you might not know what it sounds like, so I will post the waveform as well.
This waveform is data from merging two problems.

As you can see, there is a 5-second wait time between the problem and the answer explanation, and also a 5-second wait time between problems.
A feature of this tool is that it provides a thorough explanation for all four options, which makes the commentary quite long.

Mechanism

Now, let's explain the mechanism a little.
Note that a full explanation will be integrated into the commentary article.

Processing Order

The processing order is as follows.

• Problem Generation
  Generate exam problems using a large language model.

• Text-to-Speech
  Convert text to audio using VOICEVOX.

• Data Merging
  Merge the read audio files into a single data file.

Outputting all points in hiragana

A key point of this project is that the text is output in hiragana.

Depending on the software, Japanese text-to-speech software cannot read English.
VOICEVOX cannot read it at all, so even at a level where it should be able to read the alphabet as is, it somehow results in a whispering voice.

So, what should be done? Input in hiragana.
The method is simple: just instruct the AI to output in hiragana.
That said, the fact that alphabet characters and kanji still occasionally get mixed in is something I hope will be improved in the future.

To give a few examples, it looks like this.
The top part reads "LLM" as "eru-eru-emu," but for the last part, it left the parenthetical text in English. No, I don't need that there...

By inputting in hiragana like this, you can generate stable audio.
Once the data is finished, process it with Python and reconnect it to the specified interval to complete it.
By the way, the reading for one question is split into five parts.

Summary

This time, I turned a problem set into an audiobook.
When actually using it, well, it's at a level where it's acceptable to use.
However, there is that one thing. If the options are long or the explanation is long, it's a bit of a drawback that you forget the beginning. Is it just me?
It might be easier to use if you don't merge them so that you can skip questions.
Please feel free to try it out. See you again.

Distribution Information

Distribution URL

It is being distributed from the previous article.