Build smart chat apps with HAQM machine learning APIs and the PubNub ChatEngine

This is a guest blog post by Adam Bavosa, Open Source Software Engineer at PubNub. In their own words, “PubNub is a leading provider of real-time APIs for building chat, device control and real-time mapping apps that scale globally.”

The demand for chat apps is growing wildly. From social apps to business collaboration, in-game chat to customer support, every app needs chat. Chat apps like Telegram are making mainstream news and generating widespread interest.

Historically, building a scalable, feature-rich chat app was surprisingly difficult. Not anymore. PubNub ChatEngine has integrated HAQM machine learning APIs, which makes building intelligent, scalable chat apps super easy for mobile and web.

In this blog post, you’ll learn how to spin up the PubNub ChatEngine and add HAQM Translate and HAQM Polly for cross-lingual, speech-enabled chat apps that you can embed anywhere.

HAQM’s new machine learning (ML) APIs make chat applications more powerful. And when paired with pre-built PubNub integrations, the HAQM ML APIs can be easily added to any ChatEngine-powered chat app. You can use this blog post as a quickstart for using the new HAQM ML APIs and PubNub ChatEngine to build and deploy any kind of smart chat app quickly, whether for 1:1 chat or chatrooms that can each support over 1 million people, with enterprise-grade security and reliability.

The PubNub BLOCKS Catalog contains three blocks for HAQM ML APIs:

• HAQM Translate: Natural and fluent language translation
• HAQM Polly: Turn text into lifelike speech using deep learning
• HAQM Comprehend: Discover insights and relationships in text

A fourth block HAQM Lex (for adding conversational interfaces to your applications) will be added soon.

The following walk-through shows you how to get ChatEngine running, and demonstrates how to enable HAQM Translate and HAQM Polly within your chat app. The pattern is similar for the other HAQM APIs like HAQM Comprehend and coming soon (at the time of this blog post)… HAQM Lex, with those integrations (i.e., blocks) also available in the BLOCKS catalog.

Components covered

You will be building an app using the new HAQM ML APIs using the following components:

1. Browser tab with CodePen with built-in Chat
2. PubNub API Keys
3. HAQM Translate and HAQM Polly API Key

Getting Started with PubNub ChatEngine

PubNub ChatEngine makes building enterprise-ready chat apps easy. The PubNub ChatEngine allows for any UI to be integrated, so the UI example in this blog post is just that… a simple example.

Step 1: Create a free PubNub account

The best way to start is to create a free PubNub account. When you’re done, you should see your PubNub dashboard. Leave the portal open in a browser tab. If you already have a PubNub account, simply log in and leave the tab open.

Step 2: Create a ChatEngine instance in your PubNub account

Navigate to the PubNub ChatEngine Quickstart Tutorial page. In section 1 of the tutorial called “PubNub Keys,” you should see a   button. Press it. It will create a new PubNub keyset and configure it for use with ChatEngine. After a few seconds, you should see a success message followed by some code containing your new keyset:

Step 3: Create a copy of the ChatEngine Client with CodePen

Click here to launch a new CodePen window with ChatEngine pre-installed. Once CodePen is launched, you have to do two things:

• In CodePen, replace JavaScript lines 3-6 with your PubNub publish and subscribe keys. (That’s the code that appears after you pressed the Setup button earlier).

• Save your CodePen URL so it doesn’t get lost!
  

Congratulations! So far so good. Next you will implement HAQM APIs into your CodePen.

You now have a functioning ChatEngine app that you can begin tweaking. Open a new browser instance in Incognito mode to the same URL as your CodePen. Now you can chat with yourself, naturally. 

Later on (outside of this tutorial) you may want to dig into what you can do with ChatEngine, including Android and iOS support, complete the quickstart guide and also check out the ChatEngine docs.

Adding HAQM Translate to your PubNub ChatEngine instance

To demonstrate how simple it is to add HAQM APIs to PubNub ChatEngine, let’s start with HAQM Translate. There are only a few new concepts and details you need to understand before continuing the code tutorial:

• PubNub Functions allow you to intercept chat messages and add inline functionality. We’ll use PubNub Functions to call the HAQM Translate API and replace the source language with the target language.
• Note that your free PubNub key only allows 3 Functions to run simultaneously. Since ChatEngine already uses two Functions on its own, you can only experiment with one HAQM API at a time in your free account. Simply contact PubNub to get your account configured for more.

Step 1a: HAQM API Keys

You’ll also need your HAQM access and secret keys. If you have not already done so, create an AWS account and get an HAQM Translate and HAQM Polly access key and secret key (find out how here). Once you have your keys you will be able to add them into the AWS secure vault in your function.

Step 1b: Clone the HAQM Translate Block into Your PubNub account

1. First, ensure you’re still logged into PubNub. Then, open a new tab and go to the HAQM Translate Block. Scroll down the page, and click the “Try It Now” blue button.
   Important: on the next page, make sure you select “ChatEngine App” in the App dropdown:
2. Now click “Get Started”. This will take you to the PubNub Functions page, allowing you to edit your Functions code and deploy it.
   

Step 2: Configure your HAQM Translate PubNub Function 

Here, you will add your HAQM keys into the PubNub Functions Vault so the PubNub Function can successfully call the HAQM APIs. Then, you will also select the Source and Target languages for your Function.

• Click the “My Secrets” button on the left side of the PubNub Functions page. Add two new secrets: your HAQM access key and your HAQM secret key. The key names must be “AWS_access_key” and “AWS_secret_key”. When complete, your secrets screens should look like this:
  
  Important: do not delete the “secretKey” already in the vault. This is needed for ChatEngine and is not related to the HAQM APIs.
  

Step 3: Launch Your PubNub Function and Translate to Your Heart’s Content

There’s one final step.  You need to bind your Function to all PubNub “channels” to ensure that chat messages coming through any channel are translated. On the left side of the Functions page (right above the “My Secrets” button, find the Channel setting and change the value to a wildcard: *

Now, deploy your PubNub Function by clicking the red “Save” button, and then click the “Start module” button. After a few seconds, the PubNub Function will be deployed globally!

Now, go back to your existing ChatEngine app in CodePen and type in the source language. You should see the translations happening instantly.

Note: You can change your Source and Target language for HAQM Translate by changing the language codes on lines 9 and 10 of the JavaScript in CodePen.

Turn Chat messages into lifelike speech with HAQM Polly 

HAQM Polly creates audio, spoken language for any chat message sent through PubNub ChatEngine. This next section will show you how to deploy the HAQM Polly block, and also modify your ChatEngine client code to support audio files and spoken word.

Note: Your free PubNub account may require you to stop your HAQM Translate PubNub Function prior to deploying HAQM Polly. So remember to click the Stop Module button on your HAQM Translate function prior to starting this section of the guide. 

Step 1: Create your HAQM Polly PubNub Function

This tutorial has made some tweaks to the existing HAQM Polly Function in the BLOCKS Catalog. So, this time, instead of just cloning the existing block, we’re going to create our own PubNub Function from scratch. While the previous HAQM Translate Function used a “Before Publish” Event to intercept the chat message in-flight, this time we’ll use a PubNub REST endpoint Function, called “On Request” in the PubNub Portal.

1. Go to the PubNub Admin Dashboard, click on your ChatEngine App, then the FUNCTIONS tab on the left. You should see the ChatEngine module, click it and then click create to instantiate a new event handler.
   
2. Now, create the new module called “Polly”, and inside the module create a function also called Polly. (For clarity, a Module is a collection of Functions).
   
3. Click “Create” to add a new Function, and be sure to select the “On-Request” function type. Specify “polly-endpoint” for the URI Path:
   
4. Next we’ll copy the edited source code for the HAQM Polly integration, found in polly-pfunc.js, from the GitHub repository. Paste the code into the editor.The reason you are creating a Function Endpoint this time is the Chat App is setup to invoke a Voice MP3 Render request out-of-band from the original message. This is a great reason to use a Function Endpoint rather than an On-Before or On-After event handler.

Step 2: Configure your HAQM Polly PubNub Function

You should have already added your AWS access key and AWS secret key into the Vault in Step 2 in the HAQM Translate section. If you haven’t already done so, go ahead and add those two secret keys into the vault now.

Important: As before, do not delete the “secretKey” already in the vault. This is needed for ChatEngine and is not related to the HAQM APIs.

Step 3: Configure your ChatEngine client in CodePen 

Now, Start the Module by clicking the button at the top right. This button deploys the new code globally in just a few seconds. Our ChatEngine frontend is ready to go.

Before closing the Functions editor, click the COPY URL button. This copies the URL for the REST endpoint for the service.

In your CodePen file, paste this URL in the JavaScript to set the pollyFunctionURI (line 7).

var pollyFunctionURI = 'http://pubsub.pubnub.com/v1/blocks/sub-key/__SUB_KEY__/__URI__';

That’s it!  In CodePen, type a chat message. Turn up the volume, and click the HAQM Polly icon next to the chat message. You will hear the chat message spoken out loud!

Note for users who started this tutorial in the middle: If you haven’t started this tutorial from the top you may not have entered your API Keys into the CodePen. To get your API Keys and paste them into the CodePen JavaScript, visit the Key Info page in the PubNub portal for your PubNub ChatEngine app. Copy the Publish and Subscribe keys from the portal and paste them near the top of your CodePen.

HAQM Polly UI audio

The HAQM Polly audio data is played in the browser by using base64 encoded mp3 data URIs and the HTML audio element. The audio data is retrieved from the API only once when an HAQM Polly button is pressed. The data URI is stored on the dataset attribute of the chat log button element. There is one audio element that has its play feature triggered by the button click. In JavaScript, the URI is either fetched via AJAX, or subsequently it is moved from the button dataset to the audio source parameter.

ChatEngine features and plugins

You may have looked further at the CodePen application code. The extended features of ChatEngine plugins is used by the CodePen. You can also implement chat log history fetch, typing indicator, and the markdown plugin. ChatEngine and all of its plugins are open source on PubNub’s GitHub and available for import through npm.

Your ChatEngine code is tied to your PubNub Publish and Subscribe keys, and the chat channel. So, every other browser that launches this UI will see the same chat log as you.

There you have it! Now you can make chat apps smarter using the power of HAQM machine learning and PubNub ChatEngine. Using API integrations between ML services on AWS and PubNub, it is now possible for any developer to easily enhance their chat apps with powerful language translation, text-to-speech, sentiment analysis, and (coming soon!) chatbot functionality.

And with GDPR and SOC 2 Type II certifications using the PubNub network and ChatEngine, the largest enterprises can quickly deploy intelligent chat and collaboration functionality that meet any scaling, security, or reliability requirements.

About the author

Adam Bavosa is an Open Source Software Engineer at PubNub with interests including JavaScript, Blockchain and API development.