Sumitomo Mitsui Card's AI Operator Goes Live: 5 Hurdles Seen from the Cloud Infrastructure Side
On April 9, news broke that Sumitomo Mitsui Card had launched its "AI Operator" into production. A key feature is its design, which provides different responses for each customer during phone support.
Reading the news article, one might just think, "Wow, Japanese financial institutions are finally using AI for phone support," but as someone working in cloud infrastructure and SES, I have the feeling that the hurdles for production deployment are much higher than what is written in the article.
I will break down what makes it "high" into five points based on my field experience.
Hurdle 1: The Accuracy Gap in Speech Recognition
The biggest difference between a demo environment and actual operation is the accuracy of speech recognition.
For general Japanese transcription, 95% accuracy is the benchmark, but when used for customer support at a financial institution, at least 98% is required. There are three overlapping reasons for this:
Call quality: Phone bandwidth is narrow, and the original audio has very little information to begin with.
Professional terminology: Terms like "cashing limit," "revolving balance," and "lump-sum billing" appear frequently, which general speech recognition models have not learned.
Customer environment: Household noise, children's voices, television, street noise, and signal conditions.
When these three factors combine, my gut feeling is that accuracy drops to 85-90% immediately with a general-purpose model. To push this up to 98%, tuning for industry-specific vocabulary and adjusting acoustic models for phone audio quality are essential.
It is impossible to tell from the article how much Sumitomo Mitsui Card is doing in-house, but it shouldn't be at the level of "just calling an API." It is natural to assume that 6 months to a year of R&D went into this.
Hurdle 2: Latency Constraints (Acceptable Silence Time in Conversation)
In natural human-to-human conversation, it is common for there to be 200-500ms between the end of one person's utterance and the start of the other's. If this is exceeded, the other party becomes anxious, wondering, "Huh, can you hear me?"
To achieve the same response time with AI phone support, the processing pipeline roughly consists of:
Audio input buffering (end-of-speech detection)
Speech recognition (STT)
LLM inference
Speech synthesis (TTS)
Audio output
It consists of five stages.Running this entire process in under 500ms is physically quite difficult as long as it is running on the cloud side. LLM inference, in particular, is tricky; it can range from several hundred milliseconds to several seconds depending on the prompt length and the model size used.
Realistically, it is essential to devise ways to reduce perceived latency, such as starting inference early at the 'speech end detection' stage or using streaming generation.This engineering cannot be achieved simply by calling an API.
Hurdle 3: Core System Integration
This is the part where I, from an SES perspective, think, 'This must be tough.'
An AI operator only truly becomes meaningful when it can reference a customer's contract information, transaction history, and credit information in real-time to provide the optimal answer on the spot. Financial institution core systems are,
systems written in COBOL or old Java that are still in active use
often lack established real-time reference APIs
mostly inaccessible directly from the AI side due to security requirements
subject to strict log handling due to personal information protection concerns
In such an environment, building a mechanism to safely pass 'the information of the person currently speaking' to the LLM is many times heavier than the AI part itself. It only goes into production after you have built the RAG, set up dedicated middleware, and organized audit logs.
The term 'AI operator' makes it sound like the LLM is the main character, but the part that takes the most time in a production project is probably this core system integration part. If you don't see this, your cost estimates and delivery dates will be completely off.
Hurdle 4: Cost Estimation
If you seriously calculate the LLM call cost per minute of conversation, it is surprisingly not negligible.
Assuming a latest-generation model, with an average of 3,000 tokens of input/output per minute of conversation, and 3 to 5 minutes per call, the LLM inference cost per call is roughly several tens of yen.If there are 100,000 calls per month, it will be in the range of several million yen just for LLM inference.
In addition to this, when you add up the infrastructure for STT, TTS, and core system integration, as well as the labor costs for the operational structure, whether you can really beat the cost structure of 'several million yen per year per human operator' depends on the use case.
At the scale of Sumitomo Mitsui Card, the number of calls is orders of magnitude higher, so the recovery is fast, but if a mid-sized or smaller financial institution tries to do the same thing, they will get stuck at the economic rationality stage. I believe this 'wall of scale' will be the watershed for whether other competitors in the same industry will follow suit in the future.
Hurdle 5: Operational Responsibility (Boundary of AI Misresponses)
This is the last one, and probably the deepest topic.
When the AI provides incorrect guidance and the customer suffers damages, where does the responsibility lie?
AI vendor (model provider)
System development vendor (the company that integrated it)
Mitsui Sumitomo Card (operator)
If this boundary is not clearly defined in contracts and operational logs, it will lead to a lawsuit the moment an incident occurs.Since the Financial Services Agency's guidelines have not fully caught up either, each company has no choice but to create its own framework.
The difference between companies that can deploy to production and those that cannot often lies in management decisions on whether they can accept this risk, rather than technical capability. Even as I look at it from my perspective as a COO, most of the time when AI projects get stuck, it is not because of technology, but because of the design of the division of responsibilities.
Market Impact: Why it connects to the context of the 'Big 4 AI Era'
This story is also interesting from the perspective of tech stocks.
The production deployment of AI phone support will boost demand for the big four models—Anthropic, OpenAI, Google, and Meta—as well as specialized Voice AI vendors (such as ElevenLabs). However, only large companies with significant in-house engineering capabilities can overcome the five hurdles written above.
What can be read from this is:
Demand for 'AI API providers' will definitely grow (Anthropic, OpenAI, etc.)
The value of integration vendors that 'take it to production' will be re-evaluated (Accenture, major SIers, NTT Data, etc.)
For mid-sized companies, the market for SaaS that sells integrated solutions (speech recognition + LLM + core system linkage packages) is still open
I believe Mitsui Sumitomo Card's production deployment is a signal indicating the divide between these three market segments. It is not a story about a pure 'AI boom,' but rather a sense that the disparity in capability regarding 'who can bring AI into production' has become a axis for stock selection.
Conclusion: 3 indicators to follow in the next 3-6 months
There are three indicators I am following.
Whether other major financial institutions will follow suit (Mitsubishi UFJ, Mizuho, Resona, or credit card companies like Rakuten Card and Aeon Card)
How much sales to the Japanese financial sector will grow in the financial results of specialized voice AI vendors
The timing when major SIers like NTT Data and Nomura Research Institute commercialize 'AI operator introduction packages'
If these three things come together, the trend of 'AI phone support in Japanese financial institutions' becoming standard equipment within a few years will become realistic. Conversely, if even one of these stumbles, there is a possibility it will be remembered as an example where 'Mitsui Sumitomo Card went ahead, but others did not follow.'
https://xtech.nikkei.com/atcl/nxt/column/18/00001/11650/