AWS Contact Center

Analyze HAQM Connect Chatbot performance using Contact Trace Record, HAQM Lex logs, HAQM Athena and HAQM QuickSight – Part 4

Note: This is the fourth blog in the HAQM Connect reporting blog series. If you haven’t read “Analyze HAQM Connect Contact Trace Record with HAQM Athena and HAQM QuickSight–Part 1”, we strongly recommend you do before proceeding further.

Introduction

Many organizations want the ability to generate chatbot performance reports for their digital customer experience offering. HAQM Lex is the natural language understanding (NLP) engine for chatbots used by HAQM Connect. Relating HAQM Lex logs to the Contact Trace Record (CTR) and conversation logs, these organizations can identify chatbot issues, tune the NLP engine, understand customer sentiments, and improve the chatbot performance.
While CTR captures transactional metrics, such as hold time, wait time, and agent interaction time, and more, HAQM Lex conversation logs capture caller’s utterance, NLP confidence score, sentiment score, intent name, and more.
In this blog, you will learn how to enable HAQM Lex conversation logs, relate it to the CTR logs, and generate reports using HAQM QuickSight.

Overview

Fig 1–High level architecture diagram

In the above architecture, HAQM Kinesis Streams HAQM Connect CTRs as raw data to an HAQM Simple Storage Service (S3) bucket. The CTR captures the events associated with a contact call from your contact center. HAQM Connect delivers contact records at least once. It may deliver more CTR records for the same call, such as new information arriving after initial delivery.
HAQM Lex delivers text and audio logs for the conversation. For this blog, we will focus on the text logs with its associated metadata. HAQM Lex delivers conversational logs to S3 for each caller utterance. The conversation log is identified by a session ID, used to troubleshoot a distinct caller-chatbot session.
AWS Glue Catalog has the table definitions for the CTR and HAQM Lex output files. You can relate CTR and HAQM Lex logs by Contact Id and run queries using HAQM Athena. For visualization, HAQM QuickSight is used.
This solution uses a parameter named “Project Name” that defines the solution domain. You can deploy this solution multiple times with different project names in order to maintain isolation. Since it will be used to name fundamental components, such as an S3 bucket, the project name must be unique globally.

Pre-requisites

To follow along with the solution presented in this blog post, you must understand the following AWS services and features:

Walkthrough

There are two options to deploy based on the level of analyses:

  • Option 1—detailed analyses: This solution allows you to query HAQM Lex conversation logs with CTR entities, such as queues and agents.
  • Option 2—basic analyses: This solution allows you to analyze the HAQM Lex conversation logs only.

Option 1 deployment steps

Follow the blog “Analyze HAQM Connect Contact Trace Record with HAQM Athena and HAQM QuickSight–Part 1” to deploy and validate the solution until Athena. For data visualization in QuickSight, follow the steps in this blog.

Note the “ProjectName” used while deploying the CloudFormation template in the first blog, as the same name must be used in this step to continue to build on top of the same deployment. You can use a project name like “acathenaquicksight” but the name must be unique or CloudFormation will return an error while creating the S3 bucket. If an error happens, you can redeploy with a different project name.

  1. Sign in to the AWS Management Console.
  2. Download the CloudFormation template from here into your local machine.
  3. In your preferred Region, create a CloudFormation stack using the template file downloaded in step 2
  4. The deployment requires the following parameters:
    1. Under “CreateANewGlueDatabaseAndS3BucketForTheProject” select the “Use existing…” option
    2. Use the same project name as given while deploying “Analyze HAQM Connect Contact Trace Record with HAQM Athena and HAQM QuickSight-Part 1” blog.
  5. Keep the defaults on the remaining options page and select Create Stack
  6. To associate HAQM Connect Contact ID with the HAQM Lex conversation logs, add the “ContactId” (Destination key) as the session attribute in the Contact Flow for each “Get customer input” node.
  7. Provide the HAQM Lex Bot Alias in the Get customer input node.

Option 2 deployment steps

  1. Sign in to the AWS Management Console.
  2. Download the CloudFormation template from here into your local machine.
  3. In your preferred Region, create a CloudFormation stack using the template file downloaded in step 2
  4. The deployment requires the following parameters:
    1. Under “CreateANewGlueDatabaseAndS3BucketForTheProject” select the “Create a new…” option
    2. Provide a project name that will be used to create resources within AWS
  5. Keep the defaults on the remaining options page and select Create Stack

Steps Common to “Option 1” and “Option 2”

For HAQM Lex V1

  1. Navigate to bot’s setting tab and click on “Conversation logs”. Then click on the setting icon (cog wheel) for the bot Alias.
  2. Enable the Text logs and select Log group name and IAM role from the drop-down (both the names should contain the project name and the bot’s name provided in the CloudFormation Template (CFT) parameters).
  3. Now, build and publish the bot.

For HAQM Lex V2

  1. Navigate to the bot Aliases and click on the Alias name.
  2. Scroll down to the conversation logs section and click on Manage conversation logs
  3. Enable the Text logs and select the Log group name (It will contain the project name and the bot name provided in the CFT parameters)
  4. Scroll down and click on Save

Validation

Generate data by initiating a chat session and interacting with HAQM Lex chatbot. Within a few minutes, HAQM Lex logs data appear in the storage bucket named as your project name under the folder “lex-base” as a “.parquet” file.

  1. Log in to HAQM Athena, under the Tables you should see “connect_lex_<bot name>”.
  2. To query the data with Athena, click on three dots besides the ‘connect_chat’ table, then click on “Preview Table”
  3. Athena displays the query results when you run it.
  4. If you are deploying with option 1, create HAQM Athena view by executing the query under the CloudFormation output.

Visualize in HAQM QuickSight

Using Athena to query the data allows you to visualize the results directly with HAQM QuickSight. Before connecting HAQM QuickSight to Athena, you must grant HAQM QuickSight access to Athena and its associated S3 bucket. For more information, see managing HAQM QuickSight permissions to AWS resources. After granting the access permissions, you can create a new dataset in HAQM QuickSight based on the Athena table that was created.

  1. Navigate to QuickSight and select ‘Analyses’.
  2. Click on “New Dataset, then select “Athena” and give the data source a name.

In the example below, the name chosen is ‘connect-lex-ctr-visualization’. Afterwards, click on Create data source.

    1. Choose the name of the database and the source view/table based on the following deployment option, then click select:
      • If you are deploying with option 1, select view name “<ProjectName>_<BotName>_Lex_with_Ctr_AverageSentimentAndCtrDataPerContact”
      • If you are deploying with option 2, select the table name”connect_lex_<botname>”
      • Note: Create views within HAQM Athena to build enhanced visualization. For more details refer to this blog post.
    2. Finally, click ‘Visualize’ to complete the data-set creation.
    3. In the next screen, you should see the chat data like below

You can use HAQM QuickSight to add other chat records visualizations. Build enhanced visualizations, by creating views within HAQM Athena, check this blog post for more details. In addition, you can build dashboards that continuously monitor your connect instances and share those dashboards.

Clean up

In order to remove the resources created by the stack, perform the following:

  1. Delete the CloudFormation stacks created while implementing this blog.
  2. Delete the object and the S3 bucket created from the CloudFormation template. The Bucket name should contain the project name.
  3. Delete the Glue database created from the CloudFormation template.
  4. Delete the Athena view created for option-1 deployment.

Conclusion

In this blog, you learned how you can enable HAQM Lex logs, stream it to HAQM S3 using HAQM Kinesis Data Firehose. This blog also demonstrates how to relate CTR and HAQM Lex Logs, and generate reports on HAQM QuickSight. To build enhanced visualizations, by creating views within HAQM Athena, check this blog post for more details. To visualize your data in Tableau and Power BI, follow Building AWS data lake visualizations with HAQM Athena and Tableau and Using the HAQM Athena Power BI connector blog, respectively.

For more analytics capabilities on HAQM Connect data sources, refer the following blogs in the HAQM Connect reporting blog series:

Author Bio

Mehmet Demir is a Senior Solutions Architect at HAQM Web Services (AWS) based in Toronto, Canada. He helps customers in building well-architected solutions that support business innovation.
Ankur Taunk is a Senior Specialist Solutions Architect at AWS. He helps customer achieve their desired business outcomes in the Contact Center space leveraging HAQM Connect.