Building Real-Time Engagement Applications with Agora and Symbl.ai

by Neeraj Chaudhary | 11-23-2021

Developers building the bleeding edge of voice, video, chat, or broadcast experiences with Agora are now in a position to take its Software-Defined Real-Time Network (SD-RTN) to the next level of real-time engagement (RTE) with Symbl.ai‘s Conversation Intelligence platform.

Developers building on Agora can extend their RTE applications to truly understand natural human conversations across video, audio and messaging channels with Symbl.ai. Through comprehensive APIs, developers can easily integrate with Symbl.ai and access the full suite of conversation intelligence capabilities:

  • Intelligent speech recognition
  • Generate contextual insights such as sentiment and custom intents
  • Auto-generate and classify topics, questions, follow ups, entities and action items
  • Access advanced conversation analytics

Combining Symbl.ai and Agora enables developers to easily augment and enhance RTE applications by adding live captioning, real-time or post-call coaching, compliance, content moderation, intelligence-driven search and more.

Symbl.ai’s proprietary AI/ML technology combines the power of contextual understanding of human language and deep learning models trained with diverse conversation data to truly understand natural conversations, beyond keywords or pre-set dictionaries. Symbl.ai is domain agnostic and requires no upfront training, eliminating the need to build, maintain and train ML models – empowering developers to easily and securely integrate AI and ML with their RTE apps.

The next section will walk you through the steps to deploy a multi-party video-conferencing application that demonstrates Symbl’s Real-time APIs in combination with the Agora SDK.

Pre-requisites

Features

  • Live Closed Captioning
  • Real-time Transcription
  • Real-time Insights: Questions, Action-Items and Follow-ups
  • Real-time Topics with sentiments
  • Video conferencing with real-time video and audio
  • Enable/Disable camera
  • Mute/unmute mic
  • Screen sharing

Browser Support

This application is supported only on Google Chrome and Firefox.

Credentials

Get your Symbl credentials (App Id and App Secret) from the Symbl Platform Console.

Get your Agora credentials (App Id) from the Agora Platform Console. See here for more information on how to do that.

Setup the Database

  • Download and install PostgreSQL. You can follow these steps to install PostgreSQL.
  • Create a database with the name of your choice. See here for more information.
  • Note the username, password and database name that you have created.

Setup the Backend

1. Clone the repo.

2. Navigate to the Symbl-Powered-Agora-Backend-master directory and open the config.json file.

3. Add your Symbl App Id and App Secret values in the respective fields below:

"SYMBL_APPID": ""  "SYMBL_SECRET": ""

4. Open the file models/db.go and replace the following line (19) within the CreateDB function:

db, err := gorm.Open("postgres", os.Getenv("PG_DB_DETAILS"))

with your PostgreSQL database user, password, host and database name as described below:

db, err := gorm.Open("postgres","postgres://<user>:<password>@<host>/<db_name>?sslmode=disable")

This sample application uses GORM for connecting to the PostgreSQL database and you can learn more about it here.

Run the Backend server

1. Navigate to the Symbl-Powered-Agora-Backend-master directory and run the following command:

go run server.go

Your backend server should be running on port 8080 and you should see a log message similar to the following:

{"level":"debug","time":"9999-99-99T99:99:99-07:00","message":"Backend server running on port: 8080"}

You can also navigate to http://localhost:8080 to make sure the server is up and running without any issues. You should see a sample web page as shown below.

Setup the Frontend

1. Open the file config.json under the folder Symbl-Powered-Agora-Master and provide your Agora project name, display name and App Id in the respective fields below:

"projectName": ""  "displayName": ""  "AppID": ""

2. Add the Backend URL in respective field below:

"backEndURL": "http://localhost:8080"

Run the Frontend

Navigate to the Symbl-Powered-Agora-master directory and run the following command:

npm install

The command above will install all the necessary frontend dependencies.

Run the following command to start the frontend application:

npm run webM

Your frontend server should be running on port 3000 (http://localhost:3000). You should see the web application ready to be used.

Testing the Application

With your backend and frontend servers up and running, navigate to http://localhost:3000, click on the Create a meeting button, enter a room name and click the Create a meeting button again.

When the meeting URL is created, click on the Enter Meeting (as host) button to enter the meeting.

Select your camera, microphone and type your display name before clicking on the Join Room button.

Conclusion

This application allows you to join an Agora video conference meeting with Symbl Transcripts and Insights, Topics and Sentiments enabled and displayed on the screen in real-time.

Community

Symbl.ai‘s invites developers to reach out to us via email at [email protected], join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai’s GitHub.

Neeraj Chaudhary