6 Key Lessons: Building a Cloud Vector DB from Scratch
We recently embarked on a six-month journey of building a vector database cloud service at Zilliz. In part 1 of this series, I delved into our design objectives and provided an overview of our final architecture.
Here I want to highlight some of our key insights and lessons learned along the way. My aim is to offer valuable takeaways that may assist you as you navigate the process of building your own Software as a Service (SaaS) solution.
1. Recognize the Cloud’s Limitations
Even with cloud native systems like Milvus, transitioning to cloud SaaS poses significant challenges. It extends beyond a simple deployment on Amazon Web Services (AWS) Elastic Compute Cloud (EC2) and Elastic Block Store (EBS). Open source database users must have a deep understanding of product intricacies to achieve horizontal scaling, fault recovery and performance optimization through meticulous knob tuning. The true challenge with cloud services lies in streamlining operations while maintaining high reliability and elasticity. Addressing specific constraints of the cloud environment, such as Amazon Simple Storage Service (S3) rate limits and limitations on OpenAPI call frequency, is crucial to fully capitalize on the elasticity and scalability potential of cloud computing.
2. Roll Out Features Carefully
While continuously adding new features in the product’s early stages may seem like the right thing to do to attract customers, prioritize addressing users’ genuine pain points. Maintaining a lead time of approximately six months for open source product features over the SaaS version is a good compromise. This lead time helps ensure that these features undergo thorough testing and improvement before being rolled out for service provision.
3. Set Appropriate Limits
No product is flawless. Take S3, for example. Despite its sleek interface and extensive refinement, developers can maximize its value only in certain situations. Unlike the freedom open source products have, SaaS products require stricter safeguards and constraints. These constraints form an integral part of the product and serve as guidance and education for users. Reasonable limitations can steer users toward smarter product usage, enhancing overall value and user experience.
4. Choose Cloud-Agnostic Dependency Services
Considering the adoption of cloud-agnostic services like S3, EC2 and Kubernetes, managed services (which are widely available across major cloud platforms) can offer substantial benefits in terms of cost reduction and simplification of multicloud adoption complexities. Alternatively, opting for SaaS services that inherently support multicloud usage can streamline the process. Despite potential variations in implementation among different cloud service providers, early establishment of a multicloud adaption layer can effectively minimize redundant development efforts and enhance overall efficiency.
5. Focus on Cloud FinOps
In the public cloud, seemingly affordable resources can unexpectedly result in high costs. For instance, before conducting a bill analysis, we did not anticipate that network bandwidth costs might comprise a significant portion of overall expenses. To optimize costs and maximize performance, it’s essential to understand the performance of different instance types and services thoroughly. For example, each AWS gp3 cloud disk offers 3,000 input/output operations per second (IOPS); bundling multiple disks on a single machine and configuring RAID can substantially increase disk throughput, thereby avoiding hefty bills for additional IOPS.
6. Recognize the Significance of OpenAPI
The growing adoption of artificial intelligence (AI) agents means the role of OpenAPI and related documentation is increasingly important. Traditional cloud services rely on web consoles and graph interfaces to deliver functionality, but the future interaction and integration of cloud services will increasingly depend on OpenAPI. Service automation, agent-friendliness and observability have become pivotal evaluation criteria for future cloud services.
Wrapping Up
I hope sharing our experiences will help developers who are starting their own journey of building a SaaS. Looking back, I am proud that our team did a lot and also got the chance to learn along the way, which made it even more fulfilling.
If you want to chat about your experience or learn more, connect with me on GitHub or LinkedIn. If this kind of work sounds interesting to you, check out our careers page and come join us! We will have plenty more projects like this ahead of us!
