Upgrade HAQM OpenSearch Service Domains to the Latest Instance Types to Optimize Your Costs

Voiced by HAQM Polly

September 8, 2021: HAQM Elasticsearch Service has been renamed to HAQM OpenSearch Service. See details.

In AWS, we show no respect for the status quo (in a respectful way). We push things forward by listening to your needs and build the platform where you can find the broadest range of services and deepest functionality within these services, e.g. 400+ HAQM EC2 instances are available as of today for virtually every business need.

HAQM OpenSearch Service is a fully managed service that makes it easy for you to deploy, secure, and run elasticsearch cost effectively at scale.   You can find HAQM OpenSearch Service instances that are for general purpose as well as optimized for compute, memory, and storage.  You are encouraged to keep up to date with and upgrade to our latest generation of HAQM OpenSearch Service instances for better performance and cost efficiency. If your workload trends can let you leverage our HAQM OpenSearch Service Reserved Instances, you can gain even more savings.

AWS Reserved Instances (RI) offer you significant savings as compared to On-Demand Instance usage costs. The savings is in exchange for committing to a 1- or 3-year term of usage. Reservation models are offered for several AWS Service among which is HAQM OpenSearch Service.  Since reservations means commitment for 1- or 3-year term of usage we usually recommend to our customers to leverage this purchase option in case of stable usage with not much fluctuation and no major change in their architecture or direction in near future. Depending on the options selected during your HAQM OpenSearch Service RI purchase, you can achieve savings between 18 and 52% compared to On-Demand price.

Getting started with HAQM OpenSearch Service RI purchase recommendations

If you are wondering whether your HAQM OpenSearch Service workloads meet the criteria of RI usage, AWS Cost Explorer is a great tool to get started.  Go to your AWS Cost Explorer and click on Recommendations on the left menu. From here select “Elasticsearch” as the recommendation type.  You will then land on a page, where Cost Explorer provides you the HAQM OpenSearch Service RI purchase recommendations, based on your past 30 days of usage, as seen on the screenshot below. Having a clear understanding of what you should buy is key to maximizing your reservation-related savings.

You can from here select various parameters or filter on specific account(s) to fine-tune the given recommendations. For a complete understanding of this feature you can read our documentation.

Scope of our HAQM OpenSearch Service RI purchase recommendations

While reviewing the Cost Explorer Recommendations, you may notice that we do not give you recommendations for all your HAQM OpenSearch Service instance types, even if you are using all your clusters 100% of the time. Reason behind this is you can only buy RI on our newer instances generations. For a complete list of the instance types you can reserve, please have a look at our pricing page.

If you are using other instance types than the ones listed on our pricing page for RI, then you will not see recommendations in Cost Explorer.

In order to leverage HAQM OpenSearch Service RI purchase options for cost optimization as well as benefit from better performance, we highly recommend upgrading your HAQM OpenSearch Service domains to our latest instances types.  To do this, you need to first understand which domains you need to upgrade.

How do I identify the HAQM OpenSearch Service domains I should upgrade?

You can use AWS Config advanced queries to list all your HAQM OpenSearch Service clusters you want to upgrade and apply an Aggregator to extend the query scope to multiple accounts and supported Regions. Here’s an example AWS Config query:

This query will retrieve all HAQM OpenSearch Service domains for which the Data node(s) instance type cannot be reserved

SELECT
  configuration.domainName,
  resourceType,
  accountId,
  configuration.elasticsearchClusterConfig.instanceType,
  configuration.elasticsearchClusterConfig.dedicatedMasterType,
  awsRegion
WHERE
  resourceType = 'AWS::Elasticsearch::Domain'
  AND configuration.elasticsearchClusterConfig.instanceType NOT LIKE 't3.%'
  AND configuration.elasticsearchClusterConfig.instanceType NOT LIKE 'm5.%'
  AND configuration.elasticsearchClusterConfig.instanceType NOT LIKE 'm4.%'
  AND configuration.elasticsearchClusterConfig.instanceType NOT LIKE 'c4.%'
  AND configuration.elasticsearchClusterConfig.instanceType NOT LIKE 'c5.%'
  AND configuration.elasticsearchClusterConfig.instanceType NOT LIKE 'r5.%'
  AND configuration.elasticsearchClusterConfig.instanceType NOT LIKE 'r4.%'
  AND configuration.elasticsearchClusterConfig.instanceType NOT LIKE 'i3.%'

This other query is complementary to the previous one as it will retrieve all HAQM OpenSearch Service domains for which the Master node(s) instance type cannot be reserved

SELECT
  configuration.domainName,
  resourceType,
  accountId,
  configuration.elasticsearchClusterConfig.instanceType,
  configuration.elasticsearchClusterConfig.dedicatedMasterType,
  awsRegion
WHERE
  resourceType = 'AWS::Elasticsearch::Domain'
  AND configuration.elasticsearchClusterConfig.dedicatedMasterType NOT LIKE 't3.%'
  AND configuration.elasticsearchClusterConfig.dedicatedMasterType NOT LIKE 'm5.%'
  AND configuration.elasticsearchClusterConfig.dedicatedMasterType NOT LIKE 'm4.%'
  AND configuration.elasticsearchClusterConfig.dedicatedMasterType NOT LIKE 'c4.%'
  AND configuration.elasticsearchClusterConfig.dedicatedMasterType NOT LIKE 'c5.%'
  AND configuration.elasticsearchClusterConfig.dedicatedMasterType NOT LIKE 'r5.%'
  AND configuration.elasticsearchClusterConfig.dedicatedMasterType NOT LIKE 'r4.%'
  AND configuration.elasticsearchClusterConfig.dedicatedMasterType NOT LIKE 'i3.%'

Please do visit our HAQM OpenSearch Service pricing page regularly and update the queries accordingly, as we continuously update the instance types eligible for reservation.

Changing Instance type of your HAQM OpenSearch Service domain(s)

To change the instance type, you need to perform a configuration change on your domain. HAQM OpenSearch Service uses a blue/green deployment process when updating domains. Blue/green typically refers to the practice of running two production environments, one live and one idle, and switching between the two, as you make software changes. In the case of HAQM OpenSearch Service, it refers to the practice of creating a new environment for domain updates and routing users to the new environment after those updates are complete. The practice minimizes downtime and maintains the original environment in the event that deployment to the new environment is unsuccessful. For a full explanation of configuration change, please refer to our documentation.

Go to HAQM OpenSearch Service console, select your “Domains” on the left menu, select your domain and click “Edit cluster configuration“.

From the edition screen, you will need to change the instance type at two places and select an instance type which is eligible for reservation purchase.

Here are the two places where you need to change the instance type:

1. Data node – Instance type
2. Master node – Instance type

Once the instance type has been changed, click the “Submit“ button at the bottom of the screen and wait for the domain status to be ”Active”.

Note: If the new instance type you would like to select is not available in the console, this is because the current version of HAQM OpenSearch Service you are using on this domain is not compatible with the targeted new instance type. Please check here for more information.

Purchase HAQM OpenSearch Service RI and start optimizing your cost

Now that your cluster has been upgraded to one of our latest instance types, you can expect Cost Explorer recommendation tool to give you recommendations for the cluster on which you changed the instance type.

Get back to Cost Explorer recommendations as explained earlier in this article. Depending on how many days you’ve used HAQM OpenSearch Service, for instance, you can select “Based on the past: 7 Days” options, if you recently made the instance type change.

One you know what you need to purchase, you can use these to proceed to purchase.

To purchase HAQM OpenSearch Service RI:

1. On the Reserved Instances page in the HAQM OpenSearch Service console, choose Order reserved instance.
2. Purchase your reservations by following the instructions at HAQM OpenSearch Service Reserved Instances in the HAQM OpenSearch Service Developer Guide.

You will be able to view your reserved instances utilization along with the savings generated in your reservations utilization report in Cost Explorer in the next few days.

In this blog, we’ve shared the benefits to upgrade your HAQM OpenSearch Service instances to the latest generations, how you can identify the instances of older generations and configure your HAQM OpenSearch Service domain changes to upgrade the instance types, and how you can receive recommendations and purchase your HAQM OpenSearch Service Reserved Instances. As mentioned in the blog, please visit the HAQM OpenSearch Service pricing page regularly, as we continuously update the instance types eligible for reservation.