Transcribing with Discord's "Hold to Record" Feature - Discord x Python

Hello, this is Rcat.
In this article, I will introduce how to transcribe audio using the Discord voice recording and upload feature, which I mentioned in the following article.

Overview

First, please take a look at this.
If you cannot watch the video, the images are also fine.

What we are doing here is using the recording and sending feature built into the Discord app to perform transcription.
I usually use Gboard, so my keyboard already has voice input, but I wanted to utilize the app's built-in features so that it can be used on devices that don't have that.

Explanation

Steps to Speech Recognition

To perform speech recognition using the hold-to-record feature, you need to follow these steps.

1. Receive audio data
   This is the step to receive the sent audio file.

2. Convert the data
   Convert it into a format supported by the speech recognition library.

3. Perform speech recognition
   Input the converted data and perform transcription.
   Once completed, return the text.

Let's look at each step.

Checking how sent audio is handled

Now, how is the audio sent using the app handled?

As a result of my investigation, it seems to be an ogg file.
As a way to verify, I printed the filename of the data received by the bot, and the extension was ogg.

How to receive data

The part that receives the file looks like this.
When something is attached, not limited to audio, such as images, it is stored in attachments.
In this case, since it is only audio, the message is blank and there is only one attachment.
Therefore, if the extension of the first attachment is ogg, I determine that it is audio uploaded using the hold-to-record feature.

Next, regarding the received data, you can read it as binary data by using read(). Basically, for speed optimization, I do not save it as a file, so I am using BytesIO.

Data conversion

This time, I use the pydub library to convert ogg to wav.
The reason is that the transcription library I will use later requires WAV input.

The content is very simple, like this.
This library also supports BytesIO, so input and output are handled directly with BytesIO. It is wonderful that conversion can be done in just 3 lines.

Speech Recognition

For speech recognition, we use the SpeechRecognition library.

Here is the specific conversion procedure. Although this is also referred to as file input, it properly supports
ByteIO, so once converted, you can perform input directly. And since
Japanese text is returned, this is just a function that returns that. It is wonderful that speech recognition can be done so easily.

Distribution of samples and data

The content introduced this time is included in the AI BOT template below.
If you are interested, please refer to this to obtain it.