[GCP Case Study] Supporting the Backbone of TV Ratings! Learning Infrastructure Strategy from Video Research Ltd.'s Google Cloud Adoption

Hello, this is Masaaki Ito.

I usually publish educational articles about GCP (Google Cloud) and Chromebooks on note (masa_cloud), promoting a fresh start in one's 30s and IT education that can be learned even on low-spec PCs.

In this article, I would like to analyze the cloud adoption case of "Video Research Ltd.," a company familiar for its TV rating surveys, from the perspective of how to utilize GCP in practical work. It is a highly educational case study on how cloud technology solves challenges in large-scale systems!

1. Launching a CCoE and Providing Multi-Cloud APIs via Cloud Run

At Video Research, a CCoE (Cloud Center of Excellence) centered on the IT department has been established to promote cloud usage company-wide. It is wonderful to see how they have created a solid foundation for cloud utilization, ranging from strengthening governance and providing development/operation support to human resource development through e-learning and certification support.

Additionally, they presented at "Google Cloud Next Tokyo '23" on the theme of "Providing APIs in a Multi-Cloud Environment Using Cloud Run and HA VPN." The architecture, which securely connects GCP with other cloud environments like AWS and provides APIs using fully managed Cloud Run, is a model for modern development platforms.

2. Withstanding 100,000 Requests per Second! Load Testing Environment with GKE Autopilot

For traffic such as live streaming, the case of building a load testing environment that exceeds 100,000 requests per second to simulate both "steady access from programs" and "sudden spike access from advertisements" is a must-see.

After comparing options such as AWS Fargate and Cloud Run, the company adopted GKE Autopilot. The reasons are as follows:

• Overwhelming resource allocation: It can run up to 12,800 Pods per cluster, meeting the requirements for high-load "slam tests".

• Reduction of operational burden: Since Autopilot automatically selects appropriate resources, the effort of infrastructure management is significantly reduced.

• Cost optimization: Billing is based on the Pods used, making it easy to keep costs down by using it only during load tests.

(*By the way, in a preliminary verification using a load tool called "vegeto" on Cloud Run, it was reported that they ran out of resources at around 30,000 RPS.)

3. Phased Migration of 300TB of Viewing Data to BigQuery

Another point to note is the migration of the data platform to BigQuery. To break away from the bloat of legacy systems, they utilized the "BigQuery Migration Service" and others to migrate approximately 300TB of past viewing data.

• Utilization of SQL Translator: Existing SQL queries are automatically converted in bulk, significantly improving the efficiency of migration work.

• Data linkage and visualization: Through their proprietary data integration solution "VR LINC," they have started linking "National TV CM Data" with "Looker," Google Cloud's BI platform. By using pre-prepared code templates (Looker Blocks), they have realized seamless and speedy visualization analysis.

Conclusion: What We Can Learn

From the Video Research case, we can see how powerful the appropriate use of containers via GKE Autopilot and Cloud Run, and the construction of a large-scale data ecosystem using BigQuery and Looker, truly are.

Although this is an enterprise case, GCP's fully managed services can be sufficiently explored and learned even from personal development or low-spec PCs. For those re-challenging the IT world in their 30s, by first touching upon these modern cloud technologies even a little, you should be able to grasp the sense of architecture required in practical work.

I will continue to share the appeal of GCP and practical knowledge, so let's learn about the cloud together!

[Ninja Secret: Addendum] We have further condensed the "correct answers for practical application" seen from a thorough survey of 120 companies, including the case study introduced in this article.

■ [Complete Archive Edition] 30 Google Cloud Generative AI Use Cases Out of 120 companies, we have carefully selected 30 that are "immediately effective for practical work." This premium-grade analysis, which hits the critical points of architecture, is now available at a special price.

Get the "real wisdom" you need to survive in the wilderness.