Wednesday, March 11, 2020

Bug in UCDialplans North American Rulesets


I was troubleshooting an issue where people couldn't dial US/CA toll-free numbers in Teams the other day, when using Direct Routing. Initially, the root issue wasn't that easy to solve, due to the lack of a Voice Routing Test Case tester like we have in Skype for Business. Normally, one of the first things I do when troubleshooting issues like this, is to make sure the user is actually ALLOWED to make such phone calls, based on their assigned voice policy. Teams doesn't have an equivalent to the SfB voice route tester. If it did, I would have found it much sooner.

The first thing that I did was look at the user's call history in the Teams Admin Centre. Sure enough, every toll-free call made by the user failed with the below error logged in Teams:

"The destination failed because it doesn't exist"



No information about this error came up in a search. Reviewing logs from the SBC showed the call never got to the SBC. Client logs didn't provide any usable information. Finally, I looked deeper into the applied dialplan. At this point, I saw the problem, and it was me.

Of course, I had applied a ruleset generated by UCDialplans.com. Everybody was automatically assigned the global voice policy, which was set to allow dialing only within the US. This utilizes a feature in UCDialplans.com where you can decide exactly what constitutes a "National" call.

If you're not familiar with the intricacies of the North American dialplan (NANPA), just know that the US, Canada and 26 Caribbean countries share the country code +1.  Each country has its own set of one or more area codes. There is no way to distinguish whether a number is in the US, Canada or a Caribbean country just by looking at it.

UCDialplans.com can configure your National voice policy so that a "National" call can include any number that starts with +1, limit to US/CA or limit it to US OR CA. When creating the ruleset, UCDialplans.com has a drop-down option where you decide what you want to "Treat as National".


If you select "In-Country Only" (which is the default option), UCDialplans.com will create a national voice route that only includes the area codes for the selected country. The current US example is below, which explicitly details every area code in the US, and is derived programmatically from the current list of area codes:

^\+1(?!(900|976))(20[^04]|21[^1]|22[^1267]|23[149]|24[08]|25[12346]|26[0279]|27[0269]|30[^06]|31[^1]|32[0135]|33[^358]|34[0167]|35[12]|36[014]|38[056]|40[^03]|41[^168]|42[345]|43[0245]|44[0235]|46[39]|47[0589]|48[04]|50[^06]|51[^149]|53[0149]|54[01]|55[19]|56[12347]|57[01345]|58[056]|60[^04]|61[^13]|62[03689]|63[016]|64[016]|65[017]|66[01279]|67[018]|68[0124]|70[^059]|71[^01]|72[04567]|73[1247]|74[037]|75[47]|76[02359]|77[^1678]|78[1567]|80[^079]|81[^19]|83[0128]|84[3578]|85[^1235]|86[02345]|87[028]|90[^025]|91[^1]|92[0589]|93[^235]|94[0179]|95[12469]|97[^4567]|98[0459]|281|458|520|828)[2-9]\d{6}

For the US/Canada option, rather than explicitly including every area code, it EXCLUDES Caribbean area codes, which is a much smaller list. Similarly, for US/Canada/Caribbean there is no need to exclude any area code (except premium), so the list is even smaller.

In an effort to reduce route selection time and processing overhead, the National voice policy doesn't explicitly include PSTN usages for local dialing, UNLESS least-cost routing is being used. For the US/Canada and US/Canada/Caribbean route options, toll-free numbers are captured by the route pattern because they're NOT in the exclusion list of Caribbean area codes. Unfortunately, I didn't account for toll-free numbers in the US or Canada Only route options, which means that toll-free numbers can't be dialed if you use the default options.

You could fix this by adding the Local PSTN usage to your voice policies, but that adds unnecessary overhead. A better option is to explicitly include toll-free numbers in the computed route.

^\+1(?!(900|976))(20[^04]|21[^1]|22[^1267]|23[149]|24[08]|25[12346]|26[0279]|27[0269]|30[^06]|31[^1]|32[0135]|33[^358]|34[0167]|35[12]|36[014]|38[056]|40[^03]|41[^168]|42[345]|43[0245]|44[0235]|46[39]|47[0589]|48[04]|50[^06]|51[^149]|53[0149]|54[01]|55[19]|56[12347]|57[01345]|58[056]|60[^04]|61[^13]|62[03689]|63[016]|64[016]|65[017]|66[01279]|67[018]|68[0124]|70[^059]|71[^01]|72[04567]|73[1247]|74[037]|75[47]|76[02359]|77[^1678]|78[1567]|80[^079]|81[^19]|83[0128]|84[3578]|85[^1235]|86[02345]|87[028]|90[^025]|91[^1]|92[0589]|93[^235]|94[0179]|95[12469]|97[^4567]|98[0459]|281|458|520|828|8(00|33|44|55|66|77|88))[2-9]\d{6}

The latest version of UCDialplans.com does this now, but if you've previously applied a UCDialplans.com ruleset using the "In-Country Only" option, you can either manually update your National voice route, or you can run one of the below scripts, which will seek out any US/CA-National voice route and apply the fix. If you use any of the other options, you aren't affected.

How can I tell which option was used?

Easily. The voice route for either of the other options are MUCH smaller in size. If your voice routes look like this, you are not affected.

US/Canada: (?!(24[26]|26[48]|284|345|441|473|649|664|721|[67]58|767|784|8[024]9|86[89]|876|900|976))[2-9]\d\d[2-9]\d{6}

US/Canada/Caribbean: (?!(900|976))[2-9]\d\d[2-9]\d{6}


The Fix is In

The below PS commands will append tollfree prefixes to the problematic voice route. Run whichever is appropriate for your environment, and BACKUP before you run this.

Skype for Business Fix

$Routes = Get-CsVoiceRoute | Where {($_.Identity -eq 'US-National' -or $_.Identity -eq 'CA-National') -and $_.NumberPattern -notlike '*[2-9]\d\d[2-9]\d{6}' -and $_.NumberPattern -notlike '*8(00|33|44|55|66|77|88)*'}; ForEach ($Route in $Routes) {Set-CsVoiceRoute -Identity $Route.Identity -NumberPattern $Route.NumberPattern.replace(")[2-9]\d{6}","|8(00|33|44|55|66|77|88))[2-9]\d{6}")}

Microsoft Teams Fix

$Routes = Get-CsOnlineVoiceRoute | Where {($_.Identity -eq 'US-National' -or $_.Identity -eq 'CA-National') -and $_.NumberPattern -notlike '*[2-9]\d\d[2-9]\d{6}' -and $_.NumberPattern -notlike '*8(00|33|44|55|66|77|88)*'}; ForEach ($Route in $Routes) {Set-CsOnlineVoiceRoute -Identity $Route.Identity -NumberPattern $Route.NumberPattern.replace(")[2-9]\d{6}","|8(00|33|44|55|66|77|88))[2-9]\d{6}")}


My apologies to anyone caught up by this bug. If you see me in the streets, feel free to slap me.


Monday, March 9, 2020

"Have you tried turning it off and back on again?" for MS Teams

One of the most powerful tools in a support professional's arsenal is the tried-tested-and-true method of restarting a computer to resolve strange issues. I can't tell you how many times this fixed the oddest issues over the years. Why does it work? You might as well ask where socks go once placed in the clothes dryer.

I'm excited/sad to report that the same method works in Microsoft Teams to resolve strange issues that defy explanation. I'm not talking about client-side issues. I'm talking about issues at the Office 365 end of things. How does that work when there are no administrator-accessible servers to restart? You simply toggle the relevant setting off and back on again.

We use Direct Routing for our PSTN services, and for the most part it works great. In the past few weeks, I've had some of our users experience strange issues with telephony in Teams.
  1. A few users were unable to receive any PSTN calls. Troubleshooting at the SBC would show 487 Request Terminated - Phone number not found.
  2. Another user's outbound PSTN calls were showing as Anonymous/Private. There weren't any caller ID policies being applied that would account for the behaviour.
For each of those issues, I verified that things were indeed setup properly in Teams via PowerShell. Get-CsOnlineUser showed that their LineURI and OnpremLineURI settings were correct, and Enterprise Voice was enabled. Get-CsOnlineVoiceUser showed they were all licensed, PSTNConnectivity was OnPremises and their Number was correct.

In short, there was nothing on the Teams side of things that indicated an issue. I decided to try the "Have you turned it off and back on again?" trick, which consisted of:
Set-CsUser -Identity UserID@contoso.com -EnterpriseVoiceEnabled:$FALSE
Set-CsUser -Identity UserID@contoso.com -EnterpriseVoiceEnabled:$TRUE

I waited patiently all of 10 seconds between commands. I then instructed the users to log off and back onto Teams. Lo and behold, in each case, the problem went away.

Yeah, I know. I don't get it either, but whatever...it works.