A leading global media measurement company manages a highly complex data ecosystem comprising more than 11,000 household panels and over 10 million data points generated daily.
The organization relies on accurate audience measurement across TV and digital channels to deliver trusted insights to advertisers, agencies and media companies worldwide.
Legacy Data Platform Strained Trusted Media Measurement
As data volumes and analytics requirements grew, the client’s ability to deliver trusted, timely audience measurement came under pressure, eroding confidence in media planning and slowing time to market for new services and markets.
These business risks traced back to several technical roadblocks:
- Audience data was distributed across impressions, panel feeds and publisher systems, creating integration complexities.
- Inconsistent campaign reach and frequency reporting impacted confidence in media planning and measurement.
- Legacy systems struggled to support modern analytics and reporting requirements.
- Feature enhancements required lengthy development cycles due to fragile, tightly coupled code.
- Large-scale panel and census datasets demanded faster processing capabilities.
- Daily batch jobs routinely approached or exceeded SLA windows.
- Compute-intensive transformations could not be efficiently optimized using traditional SQL-based approaches.
- Monolithic pipeline architecture limited scalability and parallel processing.
- Expansion into new geographic markets and onboarding additional services required significant engineering effort.
Cloud-Native Data Platform Modernization with Azure and Databricks
Persistent partnered with the client to modernize its data platform and establish a scalable, cloud-native analytics foundation.
Platform Modernization
Persistent, migrated legacy processing frameworks to a modern Apache Spark-based architecture. We re-engineered data pipelines using Spark APIs to support both batch and near real-time processing workloads. Building reusable infrastructure patterns enabled deployment across the U.S., Europe and other global markets.
Cloud-Native Data Engineering
The team leveraged Azure, Databricks, Kubernetes (AKS), Apache Airflow and containerized services to create a scalable and resilient platform. Ensured consistent execution and processing across on-premises environments, AKS clusters and Databricks workspaces. Implemented orchestration improvements for re-execution, lineage tracking, monitoring and SLA management.
Advanced Analytics Enablement
Persistent introduced multi-resolution analytics capabilities across TV and digital measurement platforms.
Our expertise managed and developed a reusable Spark framework to accelerate implementation of new business rules and data transformations. Additionally, we delivered standardized campaign validation dashboards to support consistent cross-channel audience measurement.
Established an automated data quality framework to improve data accuracy and governance for the client.
Technology Stack
- Cloud Platform
- Microsoft Azure
- Data & Analytics
- Azure Databricks
- Apache Spark
- Containerization & Orchestration
- Azure Kubernetes Service (AKS)
- Apache Airflow
Faster, Scalable Audience Analytics with Higher Data Quality
With Persistent, the client was able to:
- Accelerate data pipeline execution by 50%, consistently meeting SLA windows for daily panel and census processing
- Double (2x) data processing capacity to absorb growing panel and census datasets
- Release new features and market expansions 40% faster, improving time to market for a competitive edge
- Improve data quality detection by 30%, strengthening confidence in campaign reach and frequency reporting
