Introducing Advanced Scaling in HAQM EMR Managed Scaling
We are excited to announce Advanced Scaling, a new capability in HAQM EMR Managed Scaling which provides customers increased flexibility to control the performance and resource utilization of their HAQM EMR on EC2 clusters. With Advanced Scaling, customers will be able to configure the desired resource utilization or performance levels for their cluster, and HAQM EMR Managed Scaling will leverage the customers intent to intelligently scale the cluster and optimize cluster compute resources.
Customers appreciate the simplicity of HAQM EMR Managed Scaling. However, there are instances where the default HAQM EMR Managed Scaling algorithm might lead to cluster under-utilization for specific customer’s workload. For instance, clusters running multiple tasks of relatively short duration (task runtime of 10 seconds or less), HAQM EMR Managed Scaling by default scales up the cluster aggressively and conservatively scale it down to avoid negative impact to job run times. While this is the right approach for SLA-sensitive workloads, it might not be optimal for cost sensitive workloads. With Advanced Scaling, customer can now configure HAQM EMR Managed Scaling behavior suitable for their workload type and we will apply tailored optimization to intelligently add or remove nodes from the clusters.
To get started with Advanced Scaling, you can set the ScalingStrategy and UtilizationPerformanceIndex parameters either when creating a new Managed Scaling policy, or updating an existing Managed Scaling policy. Advanced Scaling is available with HAQM EMR release 7.0 and later and is available in all regions where HAQM EMR Managed Scaling is available. For more details, please refer to our Advanced Scaling documentation.