Can You Hear Me Now? What MOS Scores Really Tell You About the Quality of Your Calls
November 24, 2012
When PBX and Telco vendors are asked about Voice Quality on their networks, they often respond by quoting “MOS” scores, or using the term “Toll Quality.” Few of these vendors take the time to explain what these terms mean while many sales people have only memorized them as industry buzzwords and can’t explain them at all.
MOS = Mean Opinion Score
Have you ever answered a survey that asked, “On a scale of 1 to 5, where 5 is ‘Excellent’, how satisfied were you with our customer service?” If you have, you’ve halfway there to understanding MOS Scores. Imagine inviting 100 people, one at a time into a quiet room. In this room, they all listen to an identical recording of someone speaking one sentence. After listening, they’re asked to fill out a survey form rating the voice quality of what they heard.
The survey card would look something like this:
On a scale of 1 to 5, how would you rate the sound quality of what you heard? (circle one)
- Unsatisfactory – Very annoying, distortion is objectionable.
- Poor – Annoying distortion, but not objectionable.
- Fair – Perceptible distortion, slightly annoying.
- Good – Slight perceptible level of distortion, but not annoying.
- Excellent – Imperceptible level of distortion.
Your group of 100 people might include an audiophile and symphony conductor (very fussy) as well as a retired aircraft engine repairman with some hearing loss (just happy to understand the sentence). To adjust for differences in what people consider annoying or objectionable, we’ll average all of their scores together. Statisticians call this average number a ‘Mean’ Score.
Congratulations, you have now generated a MOS score for the voice quality of that sample recording!
So, what numbers are “good enough” for a telephone call? As with many questions, the answer is, “It Depends.” In this case, it depends on the expectations of your users. Do they expect their calls to sound like a digital fiber-optic landline or are they okay with a rural cell phone connection?
The term “Toll Quality” was created to describe the voice quality of an average (properly working) traditional land line (or “Toll”) telephone. Toll Quality lines usually score about a 4.0 during typical MOS tests. Coincidentally, this is also the term I use to describe my favorite chocolate-chip cookies.
Drawbacks & Limitations of MOS
Although many vendors equate high MOS scores with good voice quality, it is important to realize that MOS scores have some limitations and do not completely describe all aspects of call quality.
Since MOS scores are one-way recordings, they do not measure how much delay there is on a call. A caller might be on a satellite link with a half-second one-way delay, causing callers to talk at the same time, wait a long time to see if the other side wants to talk, or resort to saying “Over” at the end of each sentence, like an old-time two-way radio operator. They might be very annoyed, but this would not show up in a standard recorded MOS score.
Another call quality factor frequently not captured in a MOS score is overall loudness. If you’ve ever complained that your television commercials were blasted at double the volume of the program you were watching or had a conference call with one LOUD caller, you’ve heard a problem that wouldn’t show up in a MOS score. (Remember, the MOS test is conducted in a very quiet room, and only one recording is played.) There are some variations of the MOS survey that measure Loudness Preference (MOSLP) or Listening Effort (MOSLE), but these use separate questions and scales than the base MOS numbers typically quoted by vendors.
While MOS Scores may provide some indication of how happy (or unhappy) your callers are, it still frequently only confirms something you already found out from the angry mob outside your office. Like the “Check Engine” light on your car, it indicates that you may have a problem, but does little to diagnose what the underlying problem is, and what piece of your wiring or equipment is the root cause of their frustrations.
To troubleshoot the source of bad voice quality, you need to find the “impairment” causing the problem–which we hope to cover in future articles.
Are you an advanced user who wants to completely “geek out” on MOS testing methods? See the ITU Standards Document here: http://www.itu.int/rec/T-REC-P.800-199608-I/en
Do you have feedback, comments, or suggestions for future topics? Want to see more (or less) articles like this? Comment below!
Image Credits: Moss = Microsoft Office Clip Art Online; Idiot Light = Wikipedia Commons.