Enable smart text analytics using HAQM OpenSearch Service and HAQM Comprehend

September 8, 2021: HAQM Elasticsearch Service has been renamed to HAQM OpenSearch Service. See details.

We’re excited to announce an end-to-end solution that leverages natural language processing to analyze and visualize unstructured text in your HAQM OpenSearch Service domain with HAQM Comprehend in the AWS Cloud. You can deploy this solution in minutes with an AWS CloudFormation template and visualize your data in a Kibana dashboard.

HAQM OpenSearch Service is a fully managed service that delivers Elasticsearch’s easy-to-use APIs and real-time capabilities along with the availability, scalability, and security required by production workloads. HAQM Comprehend is a fully managed natural language processing (NLP) service that enables text analytics to extract insights from the content of documents. Customers can now leverage HAQM OpenSearch and HAQM Comprehend to index and analyze unstructured text, and deploy a pre-configured Kibana dashboard to visualize extracted entities, key phrases, syntax, and sentiment from their documents.

As an example, a company might have large volumes of online customer feedback or transcribed customer calls. With this solution, you can visualize a time series of the sentiment of customer contacts, analyze a word cloud of the entities or key phrases in those contacts, search contacts for a specific product by sentiment, and much more. In this blog post, let’s look at an example Kibana dashboard that you can deploy to draw insights from your text data with HAQM ES and HAQM Comprehend. For detailed instructions, please visit the solution implementation guide.

This solution uses AWS CloudFormation to automate the deployment on the AWS Cloud. You can learn more about the solution by clicking this link and download the template here:

You can use this template to launch the solution and all associated components. Deploying this solution with the default parameters builds the following environment in the AWS Cloud.

The default configuration deploys HAQM API Gateway, AWS Lambda, HAQM OpenSearch Service, and AWS Identity and Access Management roles and policies, but you can also customize the template based on your specific network needs. Once the solution is deployed, you get a fully compatible HAQM ES RESTful API that you can use to ingest documents to HAQM ES and automatically tag the documents with NLP-based text analytics from HAQM Comprehend. You can then use the pre-configured Kibana dashboard to visualize these insights. In the example below, the entity dashboard below shows the word cloud for commercial items, organizations, people, locations, events, and titles from news content.

The sentiment dashboard below shows the sentiment over time, total counts of each sentiment and the top documents with positive and negative sentiment from unstructured text.

The Kibana dashboard is interactive and user-friendly, allowing you to dive deep into your unstructured text data. Try this solution now:

• AWS Solution overview
• Implementation guide
• AWS CloudFormation template

This solution is available in all Regions where HAQM ES and HAQM Comprehend is available. Please refer to the AWS Region Table for more information about HAQM OpenSearch Service and HAQM Comprehend availability.

About the Author

Sameer Karnik is a Sr. Product Manager leading product for HAQM Comprehend, AWS’s natural language processing service.