Telnyx & Symbl.ai: CI Enabled Transfers On Calls

Avatar photoby Eric Giannini | 09-14-2021

Objective: Telnyx Voice API Initiates Conversationally Intelligent Transfers to Live Operators with Symbl.ai’s APIs in a Python Flask App with an IVR/AVA

Concepts from the Technology

Telnyx, a developer centric CPaaS platform, provides Python telephony developers the ability to program Voice APIs for call routing. Adding Symbl.ai’s Conversation Intelligence APIs empowers developers to transform call routing into omnichannel, contextual, artificially intelligent transfers, bridging the gaps from conversational intelligence to conversation intelligence.

Synopsis:

Your customer calls an telephony API programmed by Telnyx to route calls for an Integrated Virtual Assistant (IVR)/ Automated Virtual Assistant (AVA). After the customer’s call is received by Telnyx’s Voice API, Symbl.ai’s Conversation Intelligence API platform triggers automated speech recognition on the customer. Symbl.ai analyzes the customer’s messages to determine whether or not the customer responses display a rise or fall in disposition. If the disposition falls below a preset threshold, the logic of the application automatically transfers the customer away from a IVR/AVA to a live operator, creating a persistent, omnichannel, multi-member, artificially intelligent conversation.

In your demo application you build a call routing app with Flask in Python with Telnyx’s Voice APIs. After building the call routing application, you add Symbl.ai’s Conversation Intelligence APIs to enable your call routing application to extend beyond mere automated speech recognition to analyze sentiments on messages.

Application Requirements:

  • Python3
  • Flask
  • ngrok (i.e., a secure introspectable tunnel to localhost) for Webhooks
  • Telnyx account
  • Symbl.ai account

Disclaimer :

Telnyx and Symbl.ai present the demo application as a finished product. If you would like to follow the guide to recreate or gain inspiration for implementing your own intelligent transfers, then you have to be prepared to make use of not only multiple accounts but several technologies simultaneously. These technologies are at time protocols such as PSTN or at others such things as ngrok, a secure introspectable tunnel to localhost. The demo is not for the faint of heart.

Telnyx

Sign up with Telnyx

To summarize, you need to checkout the following sections of the codebase to get up and running with your integration of Symbl.ai:

  • Telnyx Call Control Basics
  • Server and Webhook Setup
  • Receiving and Interpreting Webhooks
  • Call Commands
  • Client State
  • Building the IVR
  • Creating the IVR
  • Answering the Incoming Call

To configure those steps, you need to complete the following guide or simply download the codebase to get up and running:

Symbl.ai

Sign Up with Symbl.ai

How to Get Up and Running with Symbl.ai

Register for an account at Symbl (i.e., https://platform.symbl.ai/). Grab both your appId and your appSecret.

With both of these values you are ready to create a special file called `symbl.conf` where both your appId and appSecret are stored for access whenever you make a call with one of the Python SDK’s convenience methods. Create the file. Store the file wherever your .py application is executed. Symbl.ai’s Python SDK is configured to access file automatically.

Symbl.ai’s Python SDK is designed to enable developers to mobile the capabilities of their voice, video, broadcast, or message real-time microservices into a fully fledged system of up to date analysis. Designed according to a Singleton design pattern, the Python SDK operates on an instantiated instance of a `connection_object` through which access to Symbl.ai’s Conversation APIs is enabled by an inferred object, allowing you to access any or all of Symbl.ai’s convenience methods for programmable intelligence from a single object.

Adding Symbl.ai

The next step in your application is to add Symbl.ai. Symbl.ai’s Python SDK provides you with a library to import directly into your application.

`import symbl`

Connecting to the PSTN Call

After adding those credentials to your app, your next step is to configure your application to connect to the Telnyx Voice API’s PSTN call with the IVR/AVR speaking to the agent. Symbl.ai’s Python SDK provides you with a convenient way to connect right out of the box:

```python phoneNumber = "" #Telnyx Voice API phone number. Can be found in invitation emailId = "" #Telnyx emailId ```

After declaring these variables, the next step is to add the convenience method for starting the PSTN connection:

``` connection_object = symbl.Telephony.start_pstn( phone_number=phoneNumber, dtmf = ",,{}#,,{}#".format(meetingId, password) #do not change these variables ) ```

Although the `dtmf` is configured for Zoom, your Telnyx Voice API call does not require a `dtmf`. You may ignore these variables or add a hashtag.

To integrate the Publicly Switched Telephone Network (PSTN) connection directly with the Telnyx app, you refactor the code for handling the `phoneNumber`

Subscribing to Events

After establishing the PSTN connection, you program your app to receive live updates on the conversation’s events such as the generation of transcripts, contextual insights such as action-items, follow-ups, or questions, or topics.

The Symbl.ai Python SDK provides a convenience method called  `connection_object.subscribe` that listens to the events of a live call subscribing to updates on those live events in real-time.

It takes a dictionary parameter, where the key can be an event and it’s value can be a callback function that should be executed on the occurrence of that event.

```Python  connection_object.subscribe({     'transcript_response': lambda response: print('printing the first response ' + str(response)),      'insight_response': lambda response: print('printing the first response ' + str(response))     }     ) print(connection_object) ```

Conversation

After the conversation is established, the Subscription API begins to provide you with real-time updates on the conversation. Since the conversation is already established, a `conversationId` is generated. To assign that `conversationId` to a variable, you make a call to Symbl.ai’s API through the Python SDK’s convenience method for a `conversationId`:

```python get_conversation_id() ```

Sentiment Analysis

You make an external API call to the Conversation API after Symbl.ai’s Python SDK provides you with a `conversationId`. With Symbl.ai’s Python SDK you make a call through its convenience method with a parameter {} for live updates on the results of sentiment analysis.

```python def sentiment():        sentiment = connection_object.conversation.get_messages(parameters={'sentiment': True})            for message in sentiment.messages:            if (message.sentiment.suggested == "negative" and has_transfer_happened == False):                trigger_phrase = message.text   # Phrase that triggered a negative response                print("Message that triggered transfer:" + trigger_phrase)                has_transfer_happened == True;                return ("Negative")    i = 0    has_transfer_happened = False   # To make sure we send only 1 transfer command    while i < 30:        sentiment()        if sentiment() == "Negative":            transfer()            print("Initiated Transfer")            break        else:            time.sleep(1)            i = i + 1 ```

Transfer Logic

In your current demo you program the logic of your application to respond to a rapid decline in sentiments by triggering an automatic transfer with Telnyx’s native Call Commands in its Voice API. You configure the transfer for your Telnyx Voice API calls in the following way:

```python # our transfer command def transfer():    connection_id = os.getenv('CONNECTION_ID')    client_state = "outbound agent transfer"    encoded_client_state = base64.b64encode(client_state.encode('ascii'))    client_state_str = str(encoded_client_state, 'utf-8')    telnyx.Call.create(        connection_id=connection_id,        to = transfer_number,        from_ = conference_number,        client_state = client_state_str    ) ```

With the transfer out of the way, the next step is to configure a method for programming the logic of programmable intelligence into your application:

```python   def sentiment():        sentiment = connection_object.conversation.get_messages(parameters={'sentiment': True})            for message in sentiment.messages:            if (message.sentiment.suggested == "negative" and has_transfer_happened == False):                trigger_phrase = message.text   # Phrase that triggered a negative response                print("Message that triggered transfer:" + trigger_phrase)                has_transfer_happened == True;                return ("Negative") ```

Since your application is a demo application, you programm the intelligence into your application in a straightforward manner. If the analysis of the speaker’s sentiments drop below a threshold, the logic of your programmable intelligence immediately triggers the transfer. To manage the logic of the transfer, create a while loop in the following way:

```python    i = 0    has_transfer_happened = False   # To make sure we send only 1 transfer command    while i < 30:        sentiment()        if sentiment() == "Negative":            transfer()            print("Initiated Transfer")            break        else:            time.sleep(1)            i = i + 1 ```

Deployment

To deploy your application to local host make use of ngrok. This application is served on the port defined in the runtime environment (or in the .env file). Be sure to launch ngrok for that port

./ngrok http 8000

Conclusion

You have successfully programmed conversation intelligence into your conversational intelligence. Congratulations! Your Computer Telephony Interface experience or, CTIX, is digitally transformed with an edge from the algorithmic revolution’s Artificial Intelligence.

You program your application to respond to a rapid incline / decline in the sentiment of the call. If an IVR/AVR, for instance, mentions words like ‘special promotion’, ‘discount’, ‘free’, these phrases may elicit a rapid incline in the speaker’s disposition towards the IVR/AVR bot, triggering app logic for an automatic transfer to a live agent to seal the deal. On the other hand, words from the speaker like “operator, operator, operator”, or “I don’t want to speak to a robot”, or “f***ing robot”, may elicit a rapid decline in the speaker’s disposition, triggering an automatic transfer to ensure customer satisfaction. Either way you are free as a developer to program the logic of your application to respond to the change in sentiments accordingly.

What’s next?

With Symbl.ai you have the power to connect, visualize or transfer your conversation data. The next step is to extend your programmable intelligence beyond merely automatically intelligent transfers to a persistent, omnichannel, contextual, artificially intelligent calling experience or Computer Telephony System. After onboarding the agent, your Symbl.ai `conversationId` doesn’t change. In addition, Symbl.ai’s Tracker’s API is a feature extension for adding detection of words directly.

Imagine, for instance, that you are an administrator with a Computer Telephony Interface. You have a batch of recorded calls where the rapid decline in sentiments over messages caused your application’s logic to transfer calls automatically to a live operator. You would like to know what topics correspond to the topics within which the rapid decline of sentiments occurs, the frequency or the time from start to transfer. With Symbl.ai’s Python SDK you are empowered to transform your analysis of calls on an end to end basis, both providing the Conversation Intelligence with which to program intelligent transfers as well as program an entire interface for analyzing the calls straight out of the box.

How to Refine Audio with Symbl.ai

As a Conversation Intelligence Analyst, you refine your audio Your analysis of calls with a number of refinements through the Media Enhance API by Dolby.io. Jayson DeLancey, the Developer Relations Manager at Dolby.io, produced a Postman Public Workspace as a guide for developers to embark upon the process of analyzing recorded streams with Symbl.ai. A public workspace empowers developers to collaborate when building a project with a convenient Graphical User Interface (GUI) for making RESTful API requests.

Called  Transcribe Media with Symbl.ai, the public workspace is available here: https://www.postman.com/dolbyio/workspace/dolby-io-community/overview. To use the public workspace, you’ll need to:

  1. Fork the collection into your own workspace
  2. Fork the environment to add your API Keys for both Dolby.io and Symbl.ai accounts.

Within the Public Workspace, you navigate from folder to folder for what might be a typical series of steps for extending beyond mere automatic speech recognition to analyses.

The Dolby.io Media Enhance API is designed to improve the sound of your media. Our algorithms analyze your media and make corrections to quickly improve and produce professional quality audio. By using this API you can automate typically labor intensive and specialized media processing tasks to improve:

  • Speech leveling
  • Noise reduction
  • Loudness correction
  • Speech isolation
  • Sibilance reduction
  • Plosive reduction
  • Dynamic equalization
  • Tone shaping

With these features, developers may refine their analyses of wildly connected calls in a way that exposes a recorded stream to one of many enhancements for overall improved analyses.

​​Time Series Analysis

A simple example of your ability to transform a call into an analysis with Symbl.ai’s Python SDK is to map the time stamped data from the sentiments to topics for the sake of performing a time series analysis with Numpy, Matplotlib or Pandas.

Join Our Community

Symbl.ai invites developers to reach out to us via email at [email protected], join our Slack channels, fork our Postman Public Workspace, or `git clone` our repos at Symbl.ai’s GitHub.

Avatar photo
Eric Giannini
Lead Developer Evangelist & Advocate