HAQM Chime SDK for Telemedicine Solution

Introduction

To meet the needs of the emerging wave of digital healthcare, our global healthcare customers have been building telehealth solutions that comply with healthcare regulations, such as HIPAA and GDPR, in order to deliver higher quality service and better patient outcomes. For example, telehealth visits have grown to over 30 times higher than before the COVID pandemic. In addition to ambulatory care visits, behavioral healthcare providers, such as Talkspace, can also innovate securely on  AWS in order to make online mental health consulting accessible from anywhere, thereby revolutionizing mental health care for the digital age.

HAQM Chime SDK has been a critical service powering these virtual patient care and communications by global healthcare companies. And now, HAQM Chime SDK has released two new features: media capture pipelines and live transcription, which  can be utilized to enhance HAQM Chime SDK meetings. We have previously demonstrated how to create a media capture pipeline in order to securely deliver meeting recordings to an HAQM S3 bucket. This post will further extend the HAQM Chime SDK chat application with live transcription and media capture functions.

HAQM Chime SDK chat messages are persistent and will be processed by HAQM Kinesis Data Stream. The chat serves as a waiting room scenario in this telemedicine solution. The chat messages can trigger a Lambda function that invokes the HAQM Lex bot to greet the attendee and conduct basic interactions before the healthcare provider arrives, such as self-check-in and future appointment scheduling. The live transcriptions of HAQM Chime meetings entail medical transcription analysis powered by HAQM Comprehend Medical. Besides the live transcription feature of HAQM Chime SDK, HAQM Transcribe and Transcribe Medical can be integrated with other telehealth platforms, such as automatic Zoom transcriptions.

Telemedicine Solution Deployment

Below is the overview diagram:

Here are step-by-step instructions for how to deploy this solution to your AWS account in the us-east-1 region.

1. Create an HAQM Lex V1 Bot by importing the zip file downloaded here. Build and publish the chat bot with alias as ‘demo’, and note the bot name and version, which will be utilized by the backend message moderator Lambda function.
2. Create an HAQM Cloud9 environment with m5.large (8GiB RAM) instance type (recommended). The other parameters, such as environment type, platform, and networking, can be set by default values.
3. Open the Cloud9 IDE console and install the missing dependency in the terminal:
   npm install -g jq
4.  Check out this GitHub code repository, go to televisit-demo folder, and run the deploy.sh script:

git clone http://github.com/aws-samples/amazon-chime-sdk.git

cd amazon-chime-sdk/apps/televisit-demo

./deploy.sh

If you see the prompts during the frontend CloudFormation deployment, then you just need to input the HAQM S3 bucket name that hosts the HTML and bundled Javascript file in addition to default values for other parameters. Alternatively, you can deploy the backend and frontend manually by following the READMEs in each folder.

In the backend deployment, two CloudFormation parameters in the template, namely LexBotName and LexBotAlias, should take corresponding values from step 1. The backend template will create several API Gateway endpoints handled by Lambda functions. The frontend stack will configure and build the React web application. It will also create a private S3 bucket as the origin for a CloudFront distribution, and it will upload the bundled Javascript file and HTML file to that S3 bucket. Once the deployment is finished, you should be able to access the telemedicine webpage through the CloudFront domain:

Figure 2. CloudFormation frontend stack outputs

Similar to an existing HAQM Chime SDK chat meeting application, you must register a new user on this web application, and then confirm the new user registration on AWS console for HAQM Cognito User Pool:

Figure 3. Confirm new user registration on Cognito User

Once confirmed, you can sign in to the telemedicine application through that user. After you log into the webportal, you can add a new channel to interact with Chatbot via HAQM Chime SDK chat, and then start a new meeting with live transcription and media capture functions. The Github code repository has some screenshots for the demo.

Note that we have configured the HAQM Chime SDK live transcription by using the HAQM Transcribe Primary Care use case in this demo, as you can see in the StartTranscriptionApiFunction Lambda function:

const response = await chime.startMeetingTranscription({
      "MeetingId": meetingId,
      "TranscriptionConfiguration": {
        "EngineTranscribeMedicalSettings": {
          "Region": "us-east-1",
          "LanguageCode": "en-US",
          "Specialty": "PRIMARYCARE",
          "Type": "CONVERSATION"
        }
      }
    }).promise();

HAQM Transcribe Medical supports primary care and specialty care areas, such as cardiology, neurology, obstetrics,gynecology, pediatrics, oncology, radiology, and urology, and only for English language. Alternatively, you can utilize HAQM Transcribe for other languages.

response = await chime.startMeetingTranscription({
        "MeetingId": meetingId,
        "TranscriptionConfiguration": {
          "EngineTranscribeSettings": {
            "Region": "us-east-1",
            "LanguageCode": <LanguageCode>
          }
        }
      }).promise();

The S3 bucket utilized for capturing meeting recordings have public access blocked and server-side encryption enabled. In addition to streaming live transcription, you can also start batch transcription jobs by using the captured media files. To do so, follow this blog post to concatenate the chunked audio files by using the Lambda function with FFmpeg library layer.

Clean up

• Delete HAQM Lex bot and Cloud9 instance on AWS console.
• Delete both the frontend and backend CloudFormation stack on the AWS console or via AWS CLI. If you see errors to delete the S3 bucket or Custom Resource, then empty the S3 bucket and try to delete the stack again.
• Finally, you will also need to delete the HAQM Chime app instance created during backend CloudFormation deployment by using the following AWS CLI command:

aws chime delete-app-instance --app-instance-arn <appInstanceArn FROM STEP 4>

Conclusion

This post has demonstrated how to create a web application for virtual telehealth visits between patients and clinicians. The new features of HAQM Chime SDK including live transcription and media capture, make it easier to automate the clinical notes analysis and generation. Further intelligent SOAP clinical notes automation will let clinicians focus on patient conversation as well as reduce clinician fatigue from writing and editing clinical summaries. Please note that this demo intends to show some of the possibilities when integrating HAQM Chime SDK and other managed AI services. When you build your telemedicine solutions on AWS, you should include additional security services in a production deployment, including HAQM Macie for detecting sensitive data in S3 buckets and AWS WAF for API Gateway.

To find the latest developments to this solution, check out HAQM Chime SDK applications GitHub repository.