AWS Business Intelligence Blog

The future of dashboarding: Prime Video’s migration journey to HAQM QuickSight

Prime Video is a premium streaming service that offers customers a vast collection of TV shows and movies—all with the ease of finding what they love to watch in one place. We offer customers thousands of popular movies and TV shows, from original and exclusive content to exciting live sports events. We also offer our members the opportunity to subscribe to add-on channels that they can cancel at any time and to rent or buy new release movies and TV box sets on the Prime Video Store.

Prime Video is a fast-paced growth business—available in over 240 countries and territories worldwide. The teams work in a dynamic environment where innovating on behalf of our customers is at the heart of everything we do. As a global entertainment destination, Prime Video relies heavily on data analytics to inform strategic decisions, optimize content offerings, and enhance user experiences. Prime Video’s central data engineering team, responsible for providing a governed business data layer and analytics platforms, faced significant challenges with their existing business intelligence (BI) infrastructure. These challenges led to a search for a more efficient, scalable, and cost-effective solution.

In this post, we explore how Prime Video’s central data engineering team used HAQM QuickSight to revolutionize their BI capabilities, streamline operations, and drive data-driven decision-making across the organization.

The challenge: Outgrowing shared infrastructure

Prior to adopting QuickSight, Prime Video primarily used a legacy BI server for dashboarding and reporting, hosted on HAQM Elastic Compute Cloud (HAQM EC2) instances. This setup, initially shared with two other organizations within HAQM, had expanded over time to serve 13 different organizations within HAQM. Although this arrangement seemed beneficial at first, it soon became apparent that the shared infrastructure model was causing more problems than it solved.

The main issues stemmed from the inability to partition resources effectively among the various organizations. This meant that a single resource-intensive dashboard from one group could degrade performance for other users, regardless of their own usage patterns. Additionally, the lack of clear ownership for server administration led to inconsistent management and delayed security patches.

Because the team evaluated their options, they realized that simply setting up a dedicated server for Prime Video wouldn’t address the root causes of their problems. Such a solution would still require manual patching, upgrades, and hardware management, essentially postponing rather than solving the underlying issues.

Choosing QuickSight: A strategic decision

After careful consideration, the team chose QuickSight as their new BI platform. This decision was driven by several key factors:

  • Cloud-centered architecture – QuickSight is a serverless, fully managed service, which aligned with Prime Video’s cloud-first strategy and AWS technology adoption goals.
  • Auto scaling and high availability – QuickSight is able to automatically scale resources based on demand. Combined with its built-in redundancy, it provided consistent performance and reliability.
  • Cost-effectiveness – The QuickSight pricing model, which combines fixed fees for authors with low reader costs, offered significant cost savings compared to alternative solutions.
  • Enterprise-grade security – The robust security features and seamless integration of QuickSight with existing AWS services provided the necessary safeguards for sensitive data.
  • Ease of management – As a fully managed service, QuickSight eliminated the need for manual patching, upgrades, and infrastructure maintenance, freeing up valuable IT resources.

The migration journey

The transition to QuickSight was a carefully planned process that spanned over 2 years. The team began by setting up a new QuickSight enterprise account dedicated to Prime Video in September 2022. This was followed by a phased approach to migrating existing dashboards and encouraging adoption of the new platform.

One of the key strategies employed was a voluntary dashboard burndown period, during which dashboard owners were encouraged to migrate their reports to QuickSight. This was complemented by an automated process to identify and remove inactive dashboards, which was about 40% at the start, helping streamline the migration and reduce clutter.

Throughout the migration, the team worked closely with various Prime Video groups to identify and request required features from the QuickSight product team. This collaborative approach made sure that the new platform would meet the diverse needs of the organization.

By May 2024, the team had stopped all new dashboard creation in their previous BI tool and implemented a fortnightly burndown report to track migration progress. This data-driven approach helped maintain momentum and visibility throughout the project. The final phase of the migration, which involved moving the remaining dashboards and closing down the Prime Video site on the server, was completed in November 2024.

QuickSight in action: Empowering Prime Video teams

Today, QuickSight serves as the central BI platform for Prime Video, catering to a wide range of use cases across different teams, such as Prime Video Ads, HAQM Studio Analytics, MGM+ Data team, and Science and Engineering team. Let’s explore some of the key dashboards and applications that showcase the capabilities of QuickSight.

New application version dashboard

This dashboard provides a quicker read of our P0 metrics and tracks the performance of new Prime Video applications on specific living room devices against the existing (old) applications on a daily or weekly basis through key metrics such as customer impressions per session, session conversion rate, and time to playback or first activity.

Application UX launch dashboard

This dashboard supports the launch of the new Prime Video UX design and tracks the performance of the new launch vs. the previous UX on a weekly basis through key metrics, powered through customer impression and time to playback or first activity data. The goal is to provide a quicker read of our P0 metrics and give an early indication of the new UX’s performance during the first few weeks of launch.

Client dashboard

This dashboard provides visibility into key success metrics for the client teams and empowers them to track and measure the metrics more effectively with metric movement anomaly detection and allow teams to generate actionable insights.

Product metrics dashboard

This comprehensive dashboard helps Product teams monitor session-level customer impression metrics on a daily basis.

Solution overview

The following diagram illustrates the solution architecture.

QuickSight integrates with Prime Video’s existing data infrastructure, pulling data from various sources including HAQM Simple Storage Service (HAQM S3) and relational databases like PostgreSQL, HAQM Aurora, and HAQM Redshift. This flexible architecture allows teams to use their existing data assets while benefiting from the powerful analytics capabilities of QuickSight.

Security

Security is paramount in Prime Video’s QuickSight implementation. The team uses single sign-on (SSO) based on HAQM’s internal authentication system, making sure that only authorized personnel can access sensitive business data. Additionally, QuickSight offers enterprise-grade security features that provide granular control over data access and sharing.

Realizing the benefits of QuickSight

The adoption of QuickSight has yielded numerous benefits for Prime Video:

  • Improved performance and scalability – By moving away from shared infrastructure, Prime Video eliminated performance bottlenecks and provided consistent responsiveness for users.
  • Automation capabilities – Access to QuickSight APIs enables teams to build custom applications for automating tasks such as SPICE refreshes and dashboard email delivery. It also helps in delivering content to Prime Video leaders.
  • Reduced maintenance overhead – As a fully managed service, QuickSight freed up valuable IT resources previously dedicated to server maintenance and upgrades and reduced the downtime to 0%.
  • Enhanced reliability – The team no longer experiences downtime due to server upgrades or infrastructure issues, achieving continuous access to critical business intelligence.
  • Cost and time savings – QuickSight reduced annual BI costs by about 70% and lead to approximately a 90% reduction in data engineering hours, representing significant savings without compromising on capabilities.
  • Improved governance – Detailed logging features in QuickSight, combined with AWS CloudTrail, provide comprehensive metadata on dashboards, datasets, and data sources, enhancing governance efforts.
  • Granular access control – QuickSight offers more granular control over data access with row-level security features, allowing teams to organize and secure their dashboards effectively, promoting collaboration while maintaining strict access controls.
  • Generative BI – With availability of HAQM Q, QuickSight provided support for natural language interfaces for data exploration, enabling users to uncover hidden patterns and relationships in their data with ease and without the help of a BI engineer.
  • Embedding – QuickSight embedded analytics is more deeply integrated with AWS services, making it straightforward to embed analytics into web applications and portals compared to previous BI tools.
  • Automatic narratives – QuickSight can automatically generate written narratives to explain visualizations, a feature not present in previous BI tools.

Looking ahead: The future of BI at Prime Video

QuickSight is revolutionizing data analysis and insights generation at Prime Video, and teams are excited to solve new use cases using QuickSight. The Catalog team, for instance, is using the integration of QuickSight with HAQM Q to provide natural language interfaces for catalog data exploration. This powerful combination of the Q&A feature in QuickSight can unlock new levels of insight and creativity in data analysis. Using HAQM Q Business, the DV-Finance team is exploring the feature of generating unified insight from structured and unstructured dataset using HAQM Q.

The successful implementation of QuickSight at Prime Video demonstrates the platform’s ability to meet the complex needs of a global streaming service while driving BI innovation. As Prime Video looks to the future, QuickSight will undoubtedly play a crucial role in delivering the flexibility, agility, and insights needed to stay ahead in the competitive world of digital entertainment.

About the Authors

Kapil Surve is a Data Engineer at Prime Video, with a strong foundation in building, optimizing, and maintaining data pipelines for high-performance analytics. He has a keen eye for designing solutions that streamline data accessibility and reliability, transforming raw data into actionable insights that drive data-driven decision-making across teams. When not working with data, Kapil is an avid cricket enthusiast who loves watching and discussing the sport. He also enjoys retreating to a quiet corner with his noise-canceling headphones to dive into his favorite playlists, finding a perfect balance between his love for music and passion for technology.

Parwinder Singh is an experienced Data Engineering Leader with a proven track record of building and leading high-performing teams to deliver robust data infrastructure and solutions. Known for a strategic approach to data management, Parwinder is passionate about driving data-driven decision-making across the organization by fostering collaboration between engineering, analytics, and business teams. Outside of work, Parwinder enjoys exploring new cuisines, or reading about the latest in tech innovation, adding a creative touch to his analytical mindset.

Maitri Shah is a Senior Technical Program Manager at HAQM QuickSight (AWS), leading enterprise analytics transformations and strategic BI migrations. She specializes in optimizing customer adoption through data-driven solutions and cross-functional collaboration and combines her creative problem-solving, both in transforming analytics programs and creating mixed media abstract art.