Showing posts with label regex. Show all posts
Showing posts with label regex. Show all posts

Tuesday, April 25, 2017

Tenant Dial Plans in Skype for Business Online

As others have noted, all Skype for Business Online tenants that are capable of PSTN calling are now able to create their own custom dial plans. This allows administrators to create custom normalization rules for local and extension dialing and other cases the default normalization rules don't handle.

Dial plans work a bit differently in Skype Online than they do in on-premise deployments. In on-prem deployments, dial plans are completely independent of one another. Users assigned either a specific site or user-level dial plan will only use the normalization rules contained within that dial plan. Normalization rules assigned at the global level do not apply.

In Skype Online, users can be assigned to either a tenant global dial plan or a tenant user dial plan. As with Skype for Business on-prem, users will get normalization rules from the global tenant dial plan, unless they are explicitly assigned to a user tenant dial plan. There is no inheritance from tenant global to tenant user dial plans.

However, users WILL inherit dial plan normalization rules from a Skype Online global dial plan specific to the country where their phone numbers are hosted. This is the default dial plan that all users are assigned to in the absence of a tenant global or user dial plan.

If you were to run Get-CsDialPlan on your tenant, you will see dial plans for most of the countries in the world (220 to be exact. 12 short of what's in the Skype Optimizer, but who's counting?). These dial plans are pretty basic, accounting for national and international dialing prefixes but not much else.  The US and UK dial plan normalization rules are shown below:

Name
Priority
Pattern
Translation
US Intl Dialing
0
^011(\d+)$
+$1
US Long Distance
1
^1(\d+)$
+1$1
US Default
2
^(\d+)$
+1$1
US Extension Rule
3
^((\+)?(\d+))(;)?(ext|extn|EXT|EXTN|x|X)(=)?(\d+)$
$1;ext=$7
GB Intl Dialing
0
^00(\d+)$
+$1
GB Long Distance
1
^0(\d+)$
+44$1
GB Default
2
^(\d+)$
+44$1
GB Extension Rule
3
^((\+)?(\d+))(;)?(ext|extn|EXT|EXTN|x|X)(=)?(\d+)$
$1;ext=$7

The rules are quite rudimentary, and don't allow for capturing and fixing user-initiated dialing errors such as overdialing, incorrect adding of national prefixes for international calls, or local calls. The extension rule is quite interesting because it appears to have been written by someone who really doesn't know how to use regular expressions. Unless your extension rule matches the very specific cases of ext, extn, EXT, EXT, x or X, then it won't work. If they used my standard for pattern and translation, it would be both simpler and would capture pretty much ANY style of writing phone numbers with extensions:
Pattern: ^\+?(\d+)\D+(\d+)$
Translation: +$1;ext=$2
If you look at the dial rules for other countries, you'll see the only difference between them is the long distance and international dialing prefix and the country code.  I get that it can be tricky to maintain dial rules for 230+ countries, but hey if one simple Canadian dude can do it, couldn't Microsoft???

Aaaanyways, the global dial plan for your given country is applied along with any tenant level dial plans.  It appears that the tenant dial plans are applied first, followed by the global dial plan for your country.

This means you can use the Skype Optimizer to create more nuanced normalization rules that will work better than the default ones.  You can't disable the global dial plan, so if a user dials a number not covered by your tenant dial plans, then the global ones will apply. So, even if someone tries to dial a completely nonsensical number such as 00000023423422348987987998770, it would get captured by the global Default normalization rule and would allow the user to dial that number. Of course, it wouldn't get far, but I would think that if you can prevent users from dialing invalid numbers at the client level, it would keep Skype Online servers from having to use precious system resources to parse that number and notify the user it is invalid.

Thankfully, this means the Skype Optimizer is still relevant in the Skype Online world. You can still use it to create dial plans and normalization rules that will make the user dialing experience better, including support for:

Just select the desired options for your country, press the Generate Rules button, and run the script. It will ask for your credentials for your Skype Online tenant and will then ask if you want to create a tenant global or tenant user dial plan. It will then apply said normalization rules to your tenant. That's all it takes!

Buy now! Operators are standing by!





Wednesday, August 10, 2016

North American Call Authorization Tricks

Almost every country in the world has a dedicated country code assigned to it for telephony purposes (see countrycode.org for a full list). There is one very large exception to this rule and that is countries in North America (US and Canada) and a good portion of the Caribbean.  These countries all share the same country code of +1 and are collectively part of the North American Numbering Plan (NANP), which is managed by the North American Numbering Plan Association (NANPA).

A NANP number consists of the country code +1, a 3-digit area code, and a 7-digit subscriber number, often formatted like +1 (212) 222-3333.  You'll sometimes see the +1 omitted from the number on business cards and websites that don't think globally.

Dialing numbers in North America is quite different from most places in the world.  In the rest of the world, if you want to dial a long-distance or national number, you have to dial a national access code.  In a lot of countries, this number is 0.  However, in NANP countries, you have to dial 1 for long-distance/national numbers. That's right, you have to dial the actual NANP country code as the long-distance access code, which is unique among all other countries.  Incidentally, the international access code is 011 (in many other countries, its 00).
"Connect me with the German Chancellor immediately! I have an idea for a new album!"
If you want to dial Canada from the US, or the US from Canada, or any of the Caribbean countries in NANP, you don't dial the international access code 011, you just dial the number as you would any other long-distance number.  So, if I want to dial Toronto, Canada from Vancouver, Canada I would dial +1 416 555 1111.  If I wanted to dial New York City from Vancouver, I'd dial +1 212 555 1111.

It gets even better.  NANP area codes are allocated by a trio of over-caffeinated spider monkeys who press numbers on a giant keypad at random to assign a new area code (well, that's how it appears).  There is no logical separation between area codes between any of the countries.  For example, area code 646 is in New York, USA, 647 is in Ontario, Canada, and 649 is in the Turks & Caicos (Caribbean island nation). Click here for a current list of area code allocations in NANP.

These days, calls between the US and Canada are often priced identically to a domestic long-distance call.  But calls to the NANP Caribbean countries are usually priced much higher, akin to dialing some very remote international destinations.  Continuing the previous example, a US user that has signed up for Microsoft's Skype for Business Online PSTN Calling Plan can expect to pay $0.013 per minute  (just over 1 cent a minute) to call area code 647 in Ontario, but would end up paying $0.583 (almost 45x more expensive) to $0.741 per minute to call area code 649 in the Turks & Caicos.

Since it is nigh impossible for a regular person to distinguish a Caribbean country from US/Canada based solely on the area code, it is easy for people to accidentally incur high phone charges when calling these countries.
He had an Apple watch long before anybody else. True trendsetter, even if it took 30 years.
Skype for Business administrators often want to prevent certain users from dialing these Caribbean countries, and it can be easily done with some fancy routing rules.  Assuming you've already configured voice polices for national and international access, then you just have to modify the National level voice route to block Caribbean destinations.  You could do some research to determine the Caribbean area codes, and create your own regular expression to block them, but I figure I should do SOMETHING useful with this post and provide you with that information.

This Skype for Business route number pattern will block all Caribbean countries that are part of NANP, except for the US Virgin Islands and Puerto Rico (which are often not priced much differently than US numbers, when calling from the US):
\+1(?!(24[26]|26[48]|284|345|441|473|649|664|721|758|767|784|8[024]9|86[89]|876|900|976))[2-9]\d\d[2-9]\d{6}$
If you want to also block US Virgin Islands and Puerto Rico, then use this one:
^\+1(?!(24[26]|26[48]|284|34[05]|441|473|649|664|721|758|767|78[47]|8[024]9|86[89]|876|900|939|976))[2-9]\d\d[2-9]\d{6}$
You may note that I've also made sure to exclude premium area codes like 900 and 976.

To make sure that people assigned an International voice policy can dial those Caribbean countries, your international route should include a pattern for North American numbers and look something like this:
^\+((1(?!(900|976))[2-9]\d\d[2-9]\d{6})|([2-9]\d{6,14}))$
If you REALLY wanted to get draconian and block calls to Canada from the US, and vice versa, you could use the following rules:

Block Canadian and Caribbean calls:
^\+1(?!(900|976))(20[^04]|21[^1]|22[4589]|23[149]|24[08]|25[12346]|26[0279]|27[026]|30[^06]|31[^1]|32[0135]|33[^2358]|34[067]|35[12]|36[01]|38[056]|40[^03]|41[^168]|42[345]|43[0245]|44[023]|47[0589]|48[04]|50[^06]|51[^149]|53[0149]|54[01]|55[19]|56[12347]|57[01345]|58[056]|60[^04]|61[^13]|62[03689]|63[016]|64[16]|65[017]|66[01279]|67[018]|68[124]|70[^059]|71[^01]|72[0457]|73[1247]|74[037]|75[47]|76[02359]|77[^1678]|78[1567]|80[^079]|81[^19]|83[012]|84[3578]|85[^1235]|86[02345]|87[028]|90[^025]|91[^1]|92[0589]|93[16789]|94[0179]|95[12469]|97[^4567]|98[0459]|281|458|469|520|828)[2-9]\d{6}$
Block American and Caribbean calls:
^\+1(?!(900|976))(41[68]|43[178]|51[49]|58[17]|70[59]|78[02]|90[25]|204|226|236|249|250|289|306|343|365|403|450|506|548|579|604|613|639|647|778|807|819|825|867|873)[2-9]\d{6}$
The above rules are formatted slightly differently from the ones that block Caribbean countries.  The Caribbean rules show which area codes to BLOCK.  The other rules show which area codes to ALLOW.  Of course, I didn't roll these rules by hand, since there are tens of dozens of area codes allocated to each country, and do change from time to time.   I used the super handy-dandy Lync Optimizer.  As you might expect, these options are available in the Lync Optimizer via the "Treat as National" option (only shows for North American dial rules).  You can select one of the following options with regards to dialing other countries within NANP:
  1. In-Country Only - Treat all calls to NANP countries other than your own as international, even though users don't have to dial 011 to reach them. As such, users will have to be a member of a voice policy that allows international dialing.
  2. US/Canada - Treat calls to anywhere in US/Canada as national calls, excluding the Caribbean. To dial Caribbean countries, users will have to be a member of a voice policy that allows international dialing. Not available for Caribbean rulesets.
  3. US/Canada/Caribbean - Treats calls to anywhere in NANPA (US/Canada/Caribbean) as national calls (along with the potentially higher call costs).

If you are creating a ruleset for a Caribbean country that uses +1 as the country code, you won't be able to select US/Canada, since this would make dialing within that Caribbean country difficult.

Selecting the Simple Ruleset option prevents usage of this feature, and will default to US/Canada/Caribbean. You won't be able to control how people dial other NANP countries when there is only a single simple routing rule.

So, there you go.  A lesson in the finer details of dialing numbers in North America that the rest of the world might not be aware of. Try to contain your excitement.

Wednesday, January 21, 2015

Adventures in Normalization

One of the things that differentiate Lync from its competitors is its reliance on 3rd party peripherals like deskphones. This is often touted as a Good Thing, since you're not locked into a single vendor for phones. Phones from single-vendor solutions are often more expensive than an equivalent Lync phone simply because you can't get them from anywhere else, so competition is low and prices are higher.

Since Lync phones are provided by different vendors, there is naturally a lot of variation in the specific details on how things work. Microsoft has alleviated this somewhat by instituting a Lync-qualified devices program.  This is a fairly thorough program that puts devices through their paces to ensure they function properly within Lync. 

However rigorous the program, little details slip through the cracks. One of the details that I've noticed is how different phones (and even the Lync client) handle regular expressions in different ways. For anyone familiar with the minutiae (thanks Word of the Day Calendar) of Lync Enterprise Voice, regular expressions are used for phone number normalization and routing. 

All Lync-qualified devices handle basic regex in the same manner. For most people, this is not an issue, but sometimes you need to use lesser-known regex patterns to accomplish certain goals. When you try to get fancy with regex, things sometimes break down with disparate results between clients and phones. 

Over the years, I've heard about and personally experienced situations where Lync Optimizer-generated regex doesn't work on specific phones. The affected vendors have eventually fixed these issues, but I figured that I should do some hardcore 60 Minutes style investigation into just how well different devices handle some of the more esoteric regex patterns out there. 

To do this, I used my wide variety of Lync deskphones I have available for testing in my home lab.  (You should hear what it sounds like when someone phones me.  Everything is ringing and flashing, and I usually go "AAAAhhhhh" and run out of the room.)  I used a Polycom VVX 600, a Polycom CX600, a Snom 760, and an old LG-Nortel "Tanjay" series phone.  My AudioCodes 440HD wasn't connecting to our Lync server at the time, so it was left out of testing. Every phone was updated with the latest firmware available at testing time. 

For the first round of the Lync Normalization Olympics, I decided to keep it simple, using some of the rule formats commonly used by the Lync Optimizer.  I input the normalization rules into Lync via the Control Panel, and I used the "Create voice routing test case information" to see how Lync Control Panel would interpret the regex, and included that in the results too. For all the rulesets, I used a unique "identifier" for each rule to allow me to setup a series that I could test all at once.  

Commas

First off, I used a regex rule that included a comma, which I had heard caused normalization issues with one particular vendor. In the below example, I'm looking for a pattern of numbers that starts with 999 followed by 10 or 11 digits. The 999 gets stripped and puts a + in its place.

Pattern: ^999(\d{10,11})$
Translation: +$1
Input number: 99915552223333 then 9995552223333
Expected output: +15552223333 then +5552223333

Control Panel
Lync Client
Tanjay Phone
Polycom CX
Polycom VVX
Snom
+15552223333
+15552223333
+15552223333
+15552223333
+15552223333
+15552223333
+5552223333
+5552223333
+5552223333
+5552223333
+5552223333
+5552223333
Everything passed that one with flying colours, so rumours of a phone type not handling commas seem to no longer apply.

Dealing with Extensions, Part I

Another common rule used in the Optimizer is a rule to accept any dialed number, but strip out any extensions and leave it as an externally routable E.164 number. This rule was meant mostly for click-to-dial from the Lync client, but I did hear about a particular vendor's phone not being able to parse that particular rule. In the next example, I'm looking for a pattern of numbers that starts with 998 followed by 11 digits, followed by some regex that would strip anything else beyond the 11 digits. The 998 gets stripped and puts a + in its place.

Pattern: ^998(\d{11})\d*(\D+\d+)?$
Translation: +$1
Input number: 99815552223333
Expected output: +15552223333

Control Panel
Lync Client
Tanjay Phone
Polycom CX
Polycom VVX
Snom
+15552223333
+15552223333
+15552223333
+15552223333
+15552223333
+15552223333
Everything passed that one with flying colours, so rumours of a phone type not parsing those rules seem to no longer apply.

Dealing with Extensions, Part II

The next rule test is a variation on the first example above, using a slightly different regex pattern to strip any extension. This format was used a long time ago in the Lync Optimizer, but was replaced with the "cleaner" looking version above.  One person told me he used this format to get around an issue with a particular phone vendor's issue parsing the previous example's format.

Pattern: ^997(\d{11})(\s*\S*)*?$
Translation: +$1
Input number: 99715552223333
Expected output: +15552223333

Control Panel
Lync Client
Tanjay Phone
Polycom CX
Polycom VVX
Snom
+15552223333
+15552223333
+15552223333
+15552223333
+15552223333
+15552223333
Again, no issues on any platform.

Extensions with Optional Site Code

In companies with multiple sites that provide its users with extension dialing, its often necessary to use site codes to properly differentiate one site from another. This is especially true if there are overlapping extension ranges originating from legacy PBXs. So, for a company using 4-digit extensions, they may prepend a site code to ensure uniqueness. So, Site #101 could use 1015xxx for their extension range, and Site #102 could use 1025xxx for their extension range.

One particular company wanted to ensure that users within any given site could dial their own users by extension without the prepended site code, but still allow them to enter the site code if they wanted to. I did this in one rule by using a bit of fancy regex. In the below example, a user should be able to enter the extension with or without the 996 "site code".

Pattern: ^(?:996)?(5\d{3})$$
Translation: +1555222$1
Input number: 9965000 then 5000
Expected output: +15552225000


Control Panel
Lync Client
Tanjay Phone
Polycom CX
Polycom VVX
Snom
+15552225000
+15552225000
+15552225000
+15552225000
+15552225000
+15552225000
+15552225000
+15552225000
+15552225000
+15552225000
+15552225000
+15552225000
Sigh, once again, no issues on any platform.

OK, boooooring. All the phones passed the basic regex tests without issue. Not at all "exciting". I use "exciting" in quotes, because how exciting can the topic of normalization be?

Multiple Replacement Strings

Sometimes you need to use multiple replacement strings in a rule (ie $1, $2 etc). Rumour has it some phones have issues with this.

Pattern: ^996(\d{3})000(\d{7})$
Translation: +1$1$2
Input number: 9965550002223333
Expected output: +15552223333

Control Panel
Lync Client
Tanjay Phone
Polycom CX
Polycom VVX
Snom
+15552223333
+15552223333
+15552223333
+15552223333
+15552223333
+15552223333
Running out of ways to say "No issues".


Advanced Normalization Tests 

To really gauge how well a phones' regex engine functions, we have to throw at it some regex that would only be used in very specific circumstances. 

For example, let's say I have a need to translate a set of digits into a pattern where those digits are buried within other numbers.  For example, I want to take 92xx and translate it to +155592xx999.  So, 9201 translates to +15559201999.  If you try to create a pattern like ^(92\d{2})$ and translate it to +1555$1999, there is ambiguity on whether you mean +1555 $1 999 or +1555 $19 99. 

Normally, I would follow the .NET regex reference which has different ways of dealing with this, but none of them work in the Lync client.  I did stumble upon one that did work, even though it isn’t a valid .NET expression.  Not only that, but testing with other phones gave different results most of the time.

I tried various patterns, and plugged in 9100, 9200, 9300 etc to see the results in both Control Panel and Lync softclient and desk phones. The results proved interesting.

Pattern: ^(91\d{2})$
Translation: +1555$1999
Input number: 9100
Expected output: Expect failure, but want to see how various devices handle this situation.

Control Panel
Lync Client
Tanjay Phone
Polycom CX
Polycom VVX
Snom
+1555$1999
+155599
+15559100999
+15559100999
No match
+1555
Seems as though the old Tanjay and CX-series phone work in this situation, but nothing else. No big deal, because I wouldn't expect any consistent behaviour here.


Pattern: ^(92\d{2})$
Translation: +1555${1}999
Input number: 9200
Expected output: +15559200999

Control Panel
Lync Client
Tanjay Phone
Polycom CX
Polycom VVX
Snom
+15559200999
No match
+15559200999
+15559200999
+15559200999
No match
Using ${1} is a valid way to prevent any ambiguity between $1 or $19 in this example. Sadly only the Lync client and the Snom don't respect this valid regex.


Pattern: ^(93\d{2})$
Translation: +1555$_999
Input number: 9300
Expected output: +15559300999

Control Panel
Lync Client
Tanjay Phone
Polycom CX
Polycom VVX
Snom
+15559300999
No match
No match
No match
+15559300999
No match
Using $_ is a way to substitute the entire string. Not ideal code-wise, but wanted to see if it would work.  Only the Control Panel and Polycom VVX phones properly parsed it.


Pattern: ^(?<1>94\d{2})$
Translation: +1555${1}999
Input number: 9400
Expected output: +15559400999

Control Panel
Lync Client
Tanjay Phone
Polycom CX
Polycom VVX
Snom
+15559400999
No match
No match
No match
+15559400999
No match
This is a valid .NET regex method to use named replacement strings instead of numeric. So, I could have easily used ^(?<DigItDude>94\d{2})$ -->  +1555${DigItDude}999.  Again, only the Control Panel and Polycom VVX phones properly parsed it.


Pattern: ^(95\d{2})$
Translation: +1555($1)999
Input number: 9500
Expected output: +15559500999

Control Panel
Lync Client
Tanjay Phone
Polycom CX
Polycom VVX
Snom
+1555(9500)999
+15559500999
+15559500999
+15559500999
+15559500999
+1555(9500)999
This is actually not a valid .NET regex method. Surprisingly, all the devices except for the Snom managed to parse it correctly.

Conclusions

As long as your phone number normalization requirements don't require any digit replacement buried within other digits, you can safely use just about any Lync-qualified device.  However, if you know in advance that you have such a requirement (and its very unlikely that you would), I would avoid Snom phones at this time, simply because they couldn't handle any of the advanced regex I threw at them in my attempts to solve that specific issue.

The way I see it, if I build a regex pattern in Lync Control Panel, I should be able to trust that every single device will parse that regex pattern the same way.  I find it rather annoying that there isn't any rigorous checking to see if vendors adhere to .NET regex standards. On that front, I feel Microsoft needs to be more diligent with their phone qualification process.  Its these little details that can lead to frustration for anybody deploying Lync Enterprise Voice.

If you got to the end of this post and are still not asleep.....congratulations!  This post has been entered in the "Most Boring Tech Post of the Year" for 2015. Yes, I know its early, but I think this is an early front-runner for the win.