Blog

How To/ Product
Use Case
Company
Partnership/ Customer
Thought Leadership

Categories

Why and How to Perform an Automatic Speech Recognition (ASR) Evaluation

Why use an ASR evaluation?

An ASR Evaluation can help developers troubleshoot speech recognition issues and improve performance. In addition, an ASR Evaluation can help you identify commonly misrecognized words resulting in a better customer experience. 

Metrics from an ASR evaluation

This utility can perform an evaluation of the results generated by any Speech to Text (STT) or Automatic Speech Recognition (ASR) System.

You will be able to calculate these metrics:

  • Word Error Rate (WER), which is the most common metric for measuring the performance of a Speech Recognition or Machine translation system
  • Levenshtein Distance calculated to the word level
  • Number of Word level insertions, deletions, and mismatches between the original and generated file
  • Number of Phrase level insertions, deletions and mismatches between the original and generated file
  • Text Comparison to visualize the differences (color highlights)
  • Overall statistics for the original and generated files (bytes, characters, words, new lines etc.)

Installation:

What to expect

The simplest way to run your first evaluation is to pass your original and generated options to asr-eval command. The original file is the human-generated plain text file with the original transcript for reference. The generated file is also plain text but contains the generated transcript from the STT/ASR system.

For more information, visit the Speech Recognition Evaluation Library on GitHub.

Next steps

We hope this was useful as you explore the benefits of conversational intelligence in your own products. If you have not already taken advantage, we have free trial credits so you can try Symbl’s Platform today.

Learn more about our conversational intelligence solutions by visiting our developer documentation.