The client offers various weather technology products that aid business decision making with insights from terabytes of weather data.
They were previously running their products and applications and querying their data on a leading cloud platform. As their business grew, their requirements from their cloud platform grew too: they needed a cloud that could help them enhance performance, accommodates the scale of their data, and offers the least latency. They wanted to convert to a microservices architecture for easy application build, use, and reuse. Also, they needed to improve the inter-services communication between their APIs. However, the process to achieve these were complex and expensive on their previous cloud. Hence, they decided to migrate to Google Cloud Platform for its high scalability, cost-effectiveness, and ease of setting up microservices architecture with Google Kubernetes Engine. Persistent, a Google Cloud Partner, offered its digital consulting services to start the client’s application modernization journey.
The migration process started with setting up the security policy, organization-level hierarchy, and access. After the migration, the client observed the following improvements and benefits:
- Microservices architecture: Persistent used Google Kubernetes Engine (GKE) to convert to microservices architecture with minimal efforts. The client saves on cost due to the traffic-based, auto scale-up and scale-down capability of GKE. Now it is easier to build, deploy, and maintain applications.
- Continuous Integration and Deployment: CI/CD was setup with Cloud Build. Now, the client is able to create automated application builds, run tests, provision environments, and scan for security vulnerabilities.
- Dataset migration: This involved the complex job of migrating SQL’s relational data to NoSQL database. Trillions of rows were transformed into a NoSQL structure, compressed with a smart custom data compression algorithm, and were then loaded into BigTable without any loss of data – all over a period of four months. The migration was followed by a series of complex, accurate validation of data.
- Interservice communication: Whenever a data point is queried, APIs communicate with many other internal microservices to get that information. Previously, any service that wouldn’t respond in this transaction couldn’t be identified and rectified. Now, Istio, a service mesh, has been used to improve inter-service communication performance and find the bottlenecks.
Further, BigQuery helps achieve high-performance queries for analytics; Google Cloud Storage is the storage bucket for storing reports and static application content; Google Memcache service reduces the search response time and serves content quickly to users; Stackdriver is for monitoring and alerts. Towards the end of migration, Persistent conducted training sessions on the best practices of GCP, GKE, and Istio.
