Amazon Lex sample Voice API application

Use this sample application to connect to a Lex connector for voice interaction with a Lex Bot, including real time transcripts and sentiment analysis.

Amazon Lex access

In order to get started, you will need to have an AWS account, as well as a bot on Amazon Lex. After your bot is configured, you will need to obtain your AWS key and secret.

To find your Access Key and Secret Access Key:

• Log in to your AWS Management Console.
• Click on your user name at the top right of the page.
• Click on the Security Credentials link from the drop-down menu.
• Find the Access Credentials section, and copy the latest Access Key ID.
• Click on the Show link in the same row, and copy the Secret Access Key.

You will also need your bot name, which can be found in the Settings -> General on your Amazon Lex bot page, as well as the Alias of the bot, which is located in Settings -> Alias.

About this sample application

This sample client application makes use of Vonage Voice API to establish voice calls and set up websockets connections to stream audio to the Lex connector.

The connector posts back in real time transcripts and optionally sentiment scores (if enabled in Lex Bot), via a webhook call back to this Vonage Voice API sample client application.

Once this application will be running, you test as follows:

• From a phone, call in to the phone number linked to your application (as explained below),
  
• This phone call will have a corresponding Websocket call leg for 2-way audio streaming with the connector,
  
• Transcript and sentiment scores will be received by this application in real time,
  
• When the caller hangs up, both phone call leg and Websocket leg will be automatically terminated.

The first talk action is a simple way to start the call: Lex expects the user to speak first, so we need to start the conversation as one would in a phone call, with the answerer greeting the caller. You can customise this text to fit your use case.

You should look at the range of voices available on Nexmo and on Lex to select the same voice, so that it feels natural for the caller. (There is some overlap in the choice of voices available from both Nexmo and Lex.)

The next action is connect: this makes the call connect to the WebSocket endpoint, specifically the Lex Connector WebSocket.

The parameter sensitivity allows you to set the VAD (Voice Activity Detection) sensitivity from the most sensitive (value = 0) to the least sensitive (value = 3), this is an integer value.

The path portion of the uri is the same as the path to the PostContent endpoint within Lex but with your server host address, e.g. xxxxx.ngrok.io. Therefore you should set your BOTNAME, ALIAS and USER details as part of this URI. You can get these details from your AWS Console after you set up a new instance of Lex.

Within the headers section of the endpoint you must supply your aws_key and aws_secret that will be used to connect to Lex.

The eventUrl is where Nexmo will send events regarding the connection to the Lex Connector so that your application can be aware of the start and end of a session.

Set up the connector server - Public hostname and port

First set up a Lex connector server from https://github.com/nexmo-se/lex-connector.

Default local (not public!) connector server port is: 5000.

If you plan to test using Local deployment with ngrok for both the connector application and this sample application, you may set up multiple ngrok tunnels.

For the next steps, you will need:

• The Lex connector server's public hostname and if necessary public port,
  e.g. xxxxxxxx.ngrok.io, xxxxxxxx.herokuapp.com (as LEX_CONNECTOR_SERVER, no port is necessary with ngrok or Heroku as public hostname)

Sample Voice API application public hostname and port

Default local (not public!) sample application port is: 8000.

If you plan to test using Local deployment with ngrok for both this sample application and the connector application, you may set up multiple ngrok tunnels.

For the next steps, you will need:

• The server's public hostname and if necessary public port on where this application is running,
  e.g. yyyyyyyy.ngrok.io, yyyyyyyy.herokuapp.com (as host), no port is necessary with ngrok or Heroku as public hostname.

Set up your Vonage Voice API client application credentials and phone number

Log in to your or sign up for a Vonage APIs account.

Go to Your applications, access an existing application or + Create a new application.

Under Capabilities section (click on [Edit] if you do not see this section):

Enable Voice

• Under Event URL, select HTTP POST, and enter https://<host>:<port>/event (replace <host> and <port> with the public host name and if necessary public port of the server where your application is running)
  
• Under Answer URL, leave HTTP GET, and enter https://<host>:<port>/answer (replace <host> and <port> with the actual value as mentioned above)
  
• Click on [Generate public and private key] if you did not yet create or want new ones, save the private.key file in this application folder.
  IMPORTANT: Do not forget to click on [Save changes] at the bottom of the screen if you have created a new key set.
  
• Link a phone number to this application if none has been linked to the application.

Please take note of your application ID and the linked number (as they are needed in the very next section.)

For the next steps, you will need:

• Your application ID (as APP_ID),
  
• The phone number linked to your application (your phone will call that number),
  
• Your Vonage API key (as API_KEY)
  
• Your Vonage API secret, not signature secret, (as API_SECRET)
  
• The Lex connector server public hostname and port (as LEX_CONNECTOR_SERVER)
  

Running Lex sample Voice API application

You may select one of the following 2 types of deployments.

Local deployment

To run your own instance locally you'll need an up-to-date version of Node.js (we tested with version 14.3.0).

Copy the .env.example file over to a new file called .env:

cp .env.example .env

Edit .env file, and set the 8 parameter values:
API_KEY=
API_SECRET=
APP_ID=
LEX_CONNECTOR_SERVER=
AWS_KEY=
AWS_SECRET=
BOT_NAME=
ALIAS=

Install dependencies once:

npm install

Launch the applicatiom:

node lex-client

Command Line Heroku deployment

Copy the .env.example file over to a new file called .env:

cp .env.example .env

Edit .env file, and set the 8 parameter values:
API_KEY=
API_SECRET=
APP_ID=
LEX_CONNECTOR_SERVER=
AWS_KEY=
AWS_SECRET=
BOT_NAME=
ALIAS=

If you do not yet have a local git repository, create one:

git init
git add .
git commit -am "initial"

Deploy this client application to Heroku from the command line using the Heroku CLI:

heroku create myappname

On your Heroku dashboard where your client application page is shown, click on Settings button, add the following Config Vars and set them with their respective values:
API_KEY
API_SECRET
APP_ID
LEX_CONNECTOR_SERVER
AWS_KEY
AWS_SECRET
BOT_NAME
ALIAS

git push heroku master

On your Heroku dashboard where your application page is shown, click on Open App button, that hostname is the one to be used under your corresponding Vonage Voice API application Capabilities (click on your application, then [Edit]).
See more details in above section Set up your Vonage Voice API client application credentials and phone number.