Undocumented behaviour with dialplans on MS Teams

Who am I talking to? Eartha Kitt? Sorry, wrong number. I was trying to call my car.

I've been spending a lot of time recently messing around with MS Teams dialplans thanks to the recent release of SBC-specific translation rules. For the most part, the overall structure is the same as with Skype for Business on-prem. Some PowerShell commands are different, but most of what you already know is usable in Teams.

The biggest issue with Teams is that there hasn't been a UI for managing your Teams Enterprise Voice layout. This can be an issue if you are big into Direct Routing.  Thankfully, Microsoft recently made available what is hopefully the first round of UI enhancements to the Teams Admin Center. You can now create and manage dialplans in the UI. Microsoft has dutifully updated their documentation to help people navigate these new features.

However, there are some undocumented features that open up some interesting scenarios. Along with these are some extremely frustrating bugs that you should be aware of.

The Hidden Feature

In Skype for Business on-prem, there is a global dialplan that applies to users when there isn't an appropriate site or user-level dialplan. In the Teams documentation, the tenant Global dialplan serves a similar purpose, but there are additional features outside of what can be done on-prem. 

In Teams, Microsoft has published a series of very simple pre-defined dialplans called service country dialplans that apply to nearly every country in the world. You can see these by running Get-CsDialPlan and Get-CsVoiceNormalizationRule. They do the bare minimum, and don't do any real validation. These dialplans cannot be modified. 

The documentation goes on to say that you can define normalization rules in the tenant global dialplan or any number of tenant user dialplans. If a user tenant dialplan isn't assigned to a user, they will be assigned an effective dialplan of whatever is in the tenant global dialplan and the default service country dialplan for that user. Similarly, if a user is assigned to a tenant user dialplan, their effective dialplan will consist of the contents of the assigned tenant user dialplan along with the user's service country dialplan. The tenant dialplans have higher priority than the service country ones. 

The hidden feature is that if you have defined any normalization rules in the tenant global dialplan, those normalization rules will appear at the top of the list of any new tenant user dialplans created afterwards. This can be very useful in large organizations with many sites, but have a common set of extension dial rules. You simply have to define the rules once in the tenant global dialplan, and they will show up in every new tenant user dialplan created from that point forward. This applies to dialplans created in both PowerShell and the Teams Admin Center.

Unfortunately, there isn't a real link between the tenant global and tenant user dialplans. They are simply copies of the original. If you make a change to the tenant global dialplan, it won't be reflected in all your other dialplans. As with Skype for Business, you're still stuck with using PowerShell to manage bulk changes. Overall, it might make deploying new dialplans just a little bit simpler if you choose to take advantage of this undocumented feature.

The Bugs

Unfortunately, there are also a number of annoying bugs/oversights with the new UI. Some are cosmetic, but some will force you to re-create your dialplan if you accidentally encounter one.

Description Discrepancies

If you've used UCDialplans.com to create your dialplans, you may have noticed that I make extensive use of the Description field to let people know where the rules came along with a copyright notice. I even put in linebreaks to make it readable in the SfB Control Panel UI. These descriptions are rather lengthy and go well beyond 255 characters (sorry, not sorry).

Here are my grievances with the Description field in the MAIN dialplan UI:

1. The Teams admin center dialplan UI doesn't respect the linebreaks and just renders a wall of text.
2. The UI throws an error that you can only have up to 255 characters, when PowerShell allows a much higher number.
3. The UI also throws errors about "invalid characters", such as !@#$%^*()=/\[]{}:;?<>+' Again, PowerShell has no issue with any of these characters.

What makes this even more odd, is that the UI for the individual normalization rules exhibits NONE of these issues. It renders the linebreaks properly, and doesn't complain when the description is longer than 255 characters and includes "special" characters.

The External Dialing Prefix Bug

This is a big bug, and am quite surprised it made it through QA. For a bit of background, you can configure Teams so that users can dial an external access prefix (like 9), before dialing an external number. For users coming from a legacy PBX, this is undoubtedly a boon and doesn't force them to retrain years of muscle memory. It also makes off-hook dialing more reliable. For an extended explanation of this feature, go waaay back to one of my posts from 2011 on the topic. I haven't tested it myself, but the details should still apply to Teams.

Most customers don't bother with this feature, and have been happily creating dialplans in Teams/Skype for Business Online for years via PowerShell without defining an external dialing prefix. However, if you create a new dialplan or edit an existing one in the UI, you will not be able to save your changes unless you set an external dialing prefix.

Also, this seems to have found itself into the PowerShell commands themselves, but not completely. You can create a new dialplan with New-CsTenantDialplan without an external dialing prefix, and you can make changes to an existing dialplan with Set-CsTenantDialplan, but if you try to remove an external dialing prefix via PowerShell by setting it to either $NULL or "", you will be denied.

As long as you don't set a prefix, you should be fine. Don't do what I did and set a prefix in the UI to see what happens, and expect to be able to remove it in PowerShell. This forced me to redo my entire dialplan from scratch. Thankfully, UCDialplans.com took care of that in a minute.

Spelling Mistakes

A minor quibble, but one that will force me to keep a close eye on (boo hoo me). The New/Get/Set-CsOnlinePSTNGateway command has options for setting inbound and outbound translation rules:

• InboundPstnNumberTranslationRules
• InboundTeamsNumberTranslationRules
• OutboundPstnNumberTranslationRules
• OutbundTeamsNumberTranslationRules

Do you see the problem?

Someone's spellcheck didn't work.

Conclusion

I'm trying to get this in front of the right people at Microsoft to get them to fix. Hopefully, this half of my post will have a limited shelf life. Have any of you experienced any of these issues or come across any others? Let me know in the comments.

Tenant Dial Plans in Skype for Business Online

As others have noted, all Skype for Business Online tenants that are capable of PSTN calling are now able to create their own custom dial plans. This allows administrators to create custom normalization rules for local and extension dialing and other cases the default normalization rules don't handle.

Dial plans work a bit differently in Skype Online than they do in on-premise deployments. In on-prem deployments, dial plans are completely independent of one another. Users assigned either a specific site or user-level dial plan will only use the normalization rules contained within that dial plan. Normalization rules assigned at the global level do not apply.

In Skype Online, users can be assigned to either a tenant global dial plan or a tenant user dial plan. As with Skype for Business on-prem, users will get normalization rules from the global tenant dial plan, unless they are explicitly assigned to a user tenant dial plan. There is no inheritance from tenant global to tenant user dial plans.

However, users WILL inherit dial plan normalization rules from a Skype Online global dial plan specific to the country where their phone numbers are hosted. This is the default dial plan that all users are assigned to in the absence of a tenant global or user dial plan.

If you were to run Get-CsDialPlan on your tenant, you will see dial plans for most of the countries in the world (220 to be exact. 12 short of what's in the Skype Optimizer, but who's counting?). These dial plans are pretty basic, accounting for national and international dialing prefixes but not much else.  The US and UK dial plan normalization rules are shown below:

Name	Priority	Pattern	Translation
US Intl Dialing	0	^011(\d+)$	+$1
US Long Distance	1	^1(\d+)$	+1$1
US Default	2	^(\d+)$	+1$1
US Extension Rule	3	^((\+)?(\d+))(;)?(ext|extn|EXT|EXTN|x|X)(=)?(\d+)$	$1;ext=$7
GB Intl Dialing	0	^00(\d+)$	+$1
GB Long Distance	1	^0(\d+)$	+44$1
GB Default	2	^(\d+)$	+44$1
GB Extension Rule	3	^((\+)?(\d+))(;)?(ext|extn|EXT|EXTN|x|X)(=)?(\d+)$	$1;ext=$7

The rules are quite rudimentary, and don't allow for capturing and fixing user-initiated dialing errors such as overdialing, incorrect adding of national prefixes for international calls, or local calls. The extension rule is quite interesting because it appears to have been written by someone who really doesn't know how to use regular expressions. Unless your extension rule matches the very specific cases of ext, extn, EXT, EXT, x or X, then it won't work. If they used my standard for pattern and translation, it would be both simpler and would capture pretty much ANY style of writing phone numbers with extensions:

Pattern: ^\+?(\d+)\D+(\d+)$
Translation: +$1;ext=$2

If you look at the dial rules for other countries, you'll see the only difference between them is the long distance and international dialing prefix and the country code.  I get that it can be tricky to maintain dial rules for 230+ countries, but hey if one simple Canadian dude can do it, couldn't Microsoft???

Aaaanyways, the global dial plan for your given country is applied along with any tenant level dial plans.  It appears that the tenant dial plans are applied first, followed by the global dial plan for your country.

This means you can use the Skype Optimizer to create more nuanced normalization rules that will work better than the default ones.  You can't disable the global dial plan, so if a user dials a number not covered by your tenant dial plans, then the global ones will apply. So, even if someone tries to dial a completely nonsensical number such as 00000023423422348987987998770, it would get captured by the global Default normalization rule and would allow the user to dial that number. Of course, it wouldn't get far, but I would think that if you can prevent users from dialing invalid numbers at the client level, it would keep Skype Online servers from having to use precious system resources to parse that number and notify the user it is invalid.

Thankfully, this means the Skype Optimizer is still relevant in the Skype Online world. You can still use it to create dial plans and normalization rules that will make the user dialing experience better, including support for:

• Overdialing
• Local number dialing patterns
• Extensions
• Removing national prefixes from internationally dialed numbers
• Slices, dices and makes julienne fries

Just select the desired options for your country, press the Generate Rules button, and run the script. It will ask for your credentials for your Skype Online tenant and will then ask if you want to create a tenant global or tenant user dial plan. It will then apply said normalization rules to your tenant. That's all it takes!

Buy now! Operators are standing by!

Lync/S4B Client Can Normalize Numbers with Plus Sign?!?!?

One of my central tenets of number normalization in Lync/Skype for Business has always been:

"Lync/Skype for Business will not attempt to normalize a number that already has a plus sign at the beginning"

There are signs that this is no longer true, at least for customers running Skype for Business Server 2015 and the Skype for Business 2016 client (and even the Lync 2010 client). I was messing around with normalization rules last night and realized that my Skype for Business 2016 client (version 16.0.7030.1021 32-bit) WAS normalizing numbers that started with a plus sign.  I later validated this with a Lync 2010 client against the same Skype for Business 2016 server.  And a commenter below said he's pretty sure he did this with Lync Server 2013.  So this means that either it has ALWAYS worked this way and I never realized it, or its something relatively new on the server-side.

I should clarify here and state that the normalization rule in question was specifically constructed to include a plus sign in the normalization pattern (as highlighted below):

^\+?(?:011)?(1|7|2[07]|3[0-46]|39\d|4[013-9]|5[1-8]|6[0-6]|8[1246]|9[0-58]|2[1-689]\d|3[578]\d|42|5[09]\d|6[789]\d|8[035789]\d|9[679]\d)(?:0)?(\d{6,14})(\D+\d+)?$

This DOES NOT mean that the Skype for Business client will suddenly normalize all numbers that start with a plus. If your normalization rules do not include a plus sign, it will not normalize a number with a plus sign.

This has all sorts of ramifications, all of them positive. That means that dealing with improperly formatted click-to-dial numbers in Internet Explorer is much simpler.  We can now fix this with a simple normalization rule, instead of doing it in a trunk translation rule or route.

This also means that dialing contact phone numbers from Outlook can also be much more reliable, especially if an extension is entered.  Before, if a user entered a contact phone number with an extension in Outlook using the "Phone number builder", it would format the number like "+1 (212) 555-1212 x 345".  If you clicked this number, Lync would parse the "x" as a "9" and the number would come out in Lync like "+121255512129345" and would likely fail.

I have no idea how long this new behaviour has been available.  It works on a Skype for Business Server 2015 environment with both Skype for Business 2016 clients and Lync 2010 clients.  I don't have a Lync 2013 or older environment to test with.  I tried to see if this behaviour extended to server-side normalization scenarios with no luck. I tried to change the destination phone number on an incoming call that started with a plus sign.  It didn't work, which implies that this is a client-only thing. The MSPL workaround is still required.

If this is only new to me, then I apologize for wasting everybody's time.  I've incorporated my new findings into the Lync Optimizer (as you would expect), because it does allow for several dialing scenarios to work better than before.

If anybody has any information on this or can test other versions of Lync server/client, drop me a line.

UPDATE (2016-Aug-24): Updated to clarify that a normalization rule has to explicitly look for a plus sign for this to work. Original post implied that any valid, matching normalization rule would be applied to a number that had a plus sign on the front.  Also made changes based on further observations with older clients.

Using the Lync Optimizer to Configure a Single Multi-Country SIP Trunk

As SIP trunking becomes more reliable and trusted, companies are taking advantage of the ability to consolidate standard PSTN phone lines from multiple office locations into a single SIP trunk.  Most of the larger SIP providers can provide phone numbers for multiple countries, allowing for even further consolidation of telephony resources.

Skype for Business is very well suited to this sort of consolidation.  Typically, larger companies deploy large centralized Enterprise Edition pools in a few central datacentres that serve offices spread across entire continents.  These centralized pools in turn connect to a SIP trunk over a trusted, secure connection.

As mentioned, these SIP trunks can provide phone numbers for multiple countries, and price those calls as if they originated from those countries.  So, if a company has a SIP trunk in London, UK, they can give out phone numbers to other countries.  For example, users could have Germany-based phone numbers served out of the London SIP trunk, and calls from those numbers would be priced as if they originated from Germany. If a German user placed a call to another German phone number, they would be charged as a local or national call, even though the SIP trunk may physically reside outside of Germany.  Conversely, if that same German user called a London phone number, the call would be charged as an international call.

Setting up Skype for Business dial plans and voice policies in this situation can get tricky, but fortunately I've come up with a very clean and easy to execute plan on how this can be accomplished using the Lync (Skype) Dialing Rule Optimizer.

Let's use an example of a company with a central Skype for Business deployment in London, UK with a single SIP trunk assigned to that location.  This company has offices in the following locations:

• London, UK
• Berlin, Germany
• Paris, France

All user phone numbers for all three locations are hosted out of the London SIP trunk.

The company has the following requirements:

• Users must be able to dial as they are accustomed to in their respective country
• Must be able to limit dialing for certain groups to the local, national or international level for their respective country

Let's assume this is a greenfield deployment, and nothing has been done yet in terms of Enterprise Voice configuration outside of getting the London SIP trunk connected to Skype for Business.

The first thing to do is to generate rulesets for each of the three locations using the Dialing Rule Optimizer.

First, London:

Then Berlin:

Note that I selected the "Force English Rulenames", which I did just because I can't read German or French and would prefer English rule names and descriptions throughout my Skype for Business deployment.

And finally Paris...

The resulting .PS1 rulesets are then copied to one of the Skype for Business servers. 

First, the London ruleset is applied to the deployment, creating user-level dial plans and voice policies when prompted.  When complete, there will be a single London dial plan and 3 voice policies. All routes use the single SIP trunk (as you would expect, since its the only one available).

Dial Plan page after running London ruleset

Voice Policy page after running London ruleset

In this state, we can easily assign a UK dial plan and voice policies to UK users as appropriate.

Next, run the rulesets for Berlin and Paris.  This is done for multiple reasons. Firstly, it will create German and French dial plans, so users in those countries can dial numbers as they do in those countries.  Secondly, it will allow administrators to assign country-specific local, national and international policies.  Someone assigned to a German national policy will only be able to dial phone numbers in Germany and nowhere else, regardless of the physical location of the SIP trunk.

When running the rulesets, make sure NOT to select the option to use least-cost routing. Least-cost routing only works with multiple SIP trunks from different providers. With a single SIP trunk, as in this example, implementing least-cost routing adds unnecessary PSTN usages to voice policies that ultimately don't accomplish anything.  If we make a call from a German number to London using a London route, the call will still be billed as an international call from Germany, as the SIP provider will see the German number as the source and bill accordingly.

To make things easier and faster to run, I recommend the use of the multiple PowerShell command-line switches available with rulesets.  It makes applying multiple rulesets faster and less prone to errors.  For the rulesets in this example, I used the following switches (using the Paris ruleset as an example):

 .\FR-Paris-Lync.ps1 -SiteID 1 -DialPlanType user -LeastCostRouting:$FALSE -OverwriteSiteVoicePolicy:$FALSE -LocationBasedRouting:$FALSE -PSTNGateway gatewayname -MediationPool mediationpoolname

For a full listing of the available command-line options, use the command:

Get-Help .\FR-Paris-Lync.ps1 -Full

Once done, the dial plans and voice policies will look as below.

Dial Plan page after running Berlin and Paris rulesets

Voice Policy page after running Berlin and Paris rulesets

This might seem more complicated than absolutely necessary, and you would be right. You could certainly just use London-based rules for all locations, but it would limit your options severely.

Setting up things in this way will allow users to dial numbers exactly as they do at home, and also allows administrators very granular control over dialing capabilities.  Common area phones in Paris can be limited to local dialing in Paris.  First level helpdesk employees in Germany can be limited to dialing only German numbers.

This shows how easy it is to use the Dialing Rule Optimizer to quickly setup flexible dial plans and voice policies in even the largest voice deployments.  If you have any questions about this method, please leave a comment.

Dealing with Trunk Prefixes in Phone Numbers in Skype for Business

First of all, I spend far too much time thinking about dial rules.

For those who don't know, a trunk prefix is a number (or numbers) that typically has to be dialled prior to dialling a phone number within the subscriber's country, but outside the subscriber's home area code.  The trunk prefix is a way to signal the telephone network that the dialed number is a long-distance or national-level call.

For example, the trunk prefix for UK national calls is 0.  If I were sitting in Liverpool and wanted to dial someone in London, I would have to dial 0, then the area code for London (20), then the subscriber phone number (say 3456 7890).  For example, 02034567890.

Not every country uses a trunk prefix, but those that do generally use 0 as a trunk prefix. The actual national trunk prefix usage breakdown (based on my research) is below:

• 101 countries don't use a trunk prefix at all
• 95 countries use 0
• 26 countries use 1 (these are countries that are part of NANPA, like US/Canada and many Caribbean countries)
• Russia and 5 other former USSR states use 8
• Mexico uses 01
• Hungary uses 06

NANPA countries are unique in that the country code 1 is also used as a sort of trunk prefix for long-distance calls. Italy is another unique case in that they don't have a national trunk prefix, but they DO use 0 as the beginning of the area code, which can confuse people.

Trunk prefixes are NOT used outside of the country. Continuing my earlier example, dialling that same London, UK phone number from the US would start with my international call prefix (011), then the UK country code (44), the London area code (20), and finally the subscriber number (3456 7890), resulting in 011442034567890.  If that US person tried to insert the UK national call prefix in the number (0114402034567890), it would fail to connect because trunk prefixes are not used outside the home country.

Now, the UK user shouldn't be expected to know the international trunk prefix for countries other than his own, so when presenting their number on business cards, web pages or emails, they should use the internationally recognized standard for phone number presentation, which is (DRUMROLL)......E.164!!!!  Regular readers of my blog and anyone who knows Lync/Skype for Business should be intimately familiar with E.164 formatted numbers.  If you need a refresher, look back at a very old (from 2010!), but still useful blog post on the subject.

The UK user should present their number like this:

+44 20 3456 7890

However, in many cases the UK user presents their number with the trunk prefix, like this:

+44 (0) 20 3456 7890

or

+44 (0) 203 456 7890

People within the UK understand what the (0) represents, but people outside the UK would assume the 0 is part of the phone number and try to dial it as shown, resulting in call failure. This particular practice is very common in the UK, France and to a lesser extent, Australia. There are likely others (please let me know). Below are some screenshots from various webpages from GLOBAL companies that exhibit this problem:

Astra Zeneca

BAE Systems

Rolls Royce

Since this blog sadly only reaches a very small percentage of the world's population, there's no way I can expect everyone to get their act together and format their phone numbers properly.  Others have ranted about this exact practice before, and it hasn't seemed to help.

As you may be aware, when you're using Internet Explorer with the Skype for Business/Lync client plug-in, many phone numbers on web pages can be directly dialled by clicking the wee little Skype/Lync icon beside the phone number on the right.  You can see this on the Astra Zeneca and Rolls Royce page samples above. You can click those icons, and it will dial the phone number as its presented on the page.  Clicking the number for Astra Zeneca's media department will give you this:

This is not going to work for anybody, since the trunk prefix 0 will be dialled as part of the number and will fail. You might be thinking that you could write a normalization rule that could deal with this, but Lync/Skype for Business will not normalize a number that has a + in front, because it assumes that any number starting with a + is already normalized.  No matter what you do, a phone number that already has the + can't be modified at the source.

If you're anywhere other than the UK, your international routes probably aren't so strict that it would "know" that putting the UK trunk prefix in the number is wrong, so Lync/SfB will happily send the call on through, only to get dumped by your PSTN carrier as an invalid number.

What you CAN do, is create a trunk translation rule that will catch any instances where a 0 is immediately after the country code and strip it before sending it to the PSTN.  There are only a few countries where 0 is actually a part of the phone number (land line numbers in Italy and mobile numbers in the Republic of the Congo), so the regular expression we create can deal with this.

The below rule should work nicely:

Pattern: ^\+(1|7|2[07]|3[0-46]|39\d|4[013-9]|5[1-8]|6[0-6]|8[1246]|9[0-58]|2[1235689]\d|24[013-9]|242\d|3[578]\d|42|5[09]\d|6[789]\d|8[035789]\d|9[679]\d)(?:0)?(\d{6,14})(;ext=\d+)?$

Translation: +$1$2

The first part of the expression with all the digits is the regex for every possible country code in the world. The (?:0) means that if 0 is present immediately after the country code, it will be dropped. Then we accept anything from 6 to 14 other digits.  We separate out Italy (39\d), Republic of the Congo (242\d) to allow for 0 in the cases where they are allowed in those countries.

If you are located in a country that is guilty of the crime of not using E.164 formatted numbers everywhere (I'm looking at you, United Kingdom), then there is the additional step of modifying the appropriate voice routes to allow for numbers coming through the system with a 0 in the wrong spot.  Using UK's national route as an example:

Old route pattern: ^\+44(1[1-9]\d{7,8}|2[03489]\d{8}|3[0347]\d{8}|5[56]\d{8}|8((4[2-5]|70)\d{7}|45464\d))$
New route pattern: ^\+440?(1[1-9]\d{7,8}|2[03489]\d{8}|3[0347]\d{8}|5[56]\d{8}|8((4[2-5]|70)\d{7}|45464\d))$

Note the presence of the 0? after the +44, which indicates that 0 may or may not be present for the route pattern to match. As of right now, the Optimizer will create modified routes only for countries that I'm aware of that flaunt the E.164 rules on a regular basis:

• UK
• France
• Australia

If there are others, please let me know, either directly or via blog comments.

All the above logic has been incorporated into the Lync/Skype4B Dialing Rule Optimizer, so if you are using that tool for creating your EV dialplans/routing, then you'll be covered.

Testing Inbound PSTN Connectivity Directly from Lync/Skype for Business

File this one under "It's obvious once you think about it for even a second"...

I was recently attempting to get a SIP trunk working to a new Sonus gateway located in Event Zero's head office in Brisbane, Australia.  I wanted to test inbound calls to the gateway by calling one of our response group workflows.

Normally, I would pick up my mobile or home phone and dial the number to test, but since I was dialing Australia, and I'm a cheap bastard, I didn't want to do that.  It was also the middle of the night in Australia, so I couldn't get someone local to do it for me.  If I dialed the number from Skype for Business, then it would do a reverse-number lookup, find an internal match and route me to the response group internally without going through the PSTN, which wouldn't achieve my goal.

My goal was to route a call to our Australian response group via Skype for Business via the PSTN in the easiest, laziest, least-expensive-to-my-own-wallet way possible (this is sounding like an exam question).  Pick one of the following answers:

1. Suck it up princess and eat the $2.05 it will cost to call Australia for a few seconds.
2. Wake up someone in Australia at 2 in the morning, claiming its an emergency.
3. Convince yourself that since outbound calls work fine, inbound should too.
4. Use a trunk translation rule to route a dummy phone number to the correct destination.
5. Give up and start drinking, because its Friday afternoon dammit!

The correct answer is #4 (although the judges will also accept #5).

If you've ever been at one of Doug Lawty's Enterprise Voice sessions at LyncConf or Ignite, you have probably seen his infamous diagram showing how a PSTN call works its way through the system.  To save you the pain, here the relevant details in a very small nutshell:

1. Dialed number gets normalized to E.164
2. Skype for Business does a reverse number lookup to see if there's an internal user or service that is assigned that number. 
3. If there's a match, routes the call internally directly to the user/service.
4. If there isn't an internal match, it finds a valid route through a PSTN trunk
5. It applies any outbound number translation to make the number conform to local standards and sends it on its merry way out the gateway to the PSTN.

Since reverse number lookup happens before handing off the call to the routing engine, we can simply call a dummy PSTN number of our choosing, and once it reaches the trunk translation rule stage, change that dummy number back to the proper number assigned to the Response Group workflow.

In the above example, all I do is create a trunk translation rule to change the non-existent North American phone number +1 (555) 999-0000, and translate it to the proper number.  Once the call has progressed to the point where outbound translation is happening, Skype for Business won't be doing another reverse number lookup, so it will continue to route that number out to the PSTN, at which point it should route to the correct destination (which could be right back where it came from). 

So, a simple solution to overcome my unwillingness to waste money on stuff I don't have to.

Selective Caller ID Blocking in Lync/Skype for Business

Once in a while, a customer asks how they can block caller ID on outgoing calls in Lync/Skype for Business. If they're interested in blocking caller ID on all calls, then the answer has always been simple: use the Alternate Caller ID field in Voice Routes.

Easy enough, right? But what if administrators want to give users the ability to selectively block caller ID at the individual call level?  When on the traditional PSTN, users in most places can enter a code prior to dialing the number if they want to block their caller ID.  In most North American markets, this code is usually *67.  So, a user could dial *6715552224444 and their caller ID would be blocked for that call.

In Lync/Skype for Business, you can replicate this functionality but it takes some work. Since the caller ID blocking feature is done at the route level, you will need to have a separate route for calls with an alternate caller ID.  For calls to use that route, you will need some unique signifier that would not occur in normal day-to-day dialing.

Since the caller ID block code you use on the PSTN should be unique, you can incorporate that into your dialplan, routes and trunk translation rules.  When I first tried this, I wanted to use *67 in my dialplan, but the Lync client didn't like the * so much. It threw up the virtual equivalent of giving me the finger and refused to work, so I had to go about it differently.  Thankfully, things have improved recently so that the Skype for Business client will happily accept the * (or # if you want).

My earlier version of this blog post simply put the *67 in front of the number and sent it through the system, stripping the *67 just before sending off to the PSTN.  Kevin Bingham from Cerium Networks suggested it might work by adding a custom parameter suffix, such as ;callerid=blocked (custom parameters are allowed by RFC3966).

So, after a bit of trial and error, I managed to make it work. This method is much nicer as it conforms to the standard RFCs and is fully E.164 compliant. Also, when someone types *67 and a number, it normalizes without showing the parameter suffix, which looks nicer to the user.

Here's a simplified example using the North American dialplan. First, you'll need a normalization rule in your dial plan that will accept the caller ID block code.

Pattern: ^\*67(\d{10})$
Translation Rule: +1$1;callerid=blocked

This will take any number that starts with *67 and is 10-digits long and translate it to +1 then the 10-digit number, followed by ;callerid=blocked. The client won't see this part of the normalization string.

Then you will need a special route that will accept numbers with that pattern and will apply the alternate caller ID:

Make sure you assign the appropriate PSTN gateway to the route and add the voice route to the appropriate PSTN Usages.

Finally, you will have to create a trunk translation rule on the selected trunk to strip the ;callerid=blocked from the dialed number before sending it out to the PSTN.

I've tested this using the latest Skype for Business server and client (as of August 2015), and the 5.4 firmware on a VVX 600 as well as a CX600 deskphone.  Let me know if this doesn't work on an older platform. I originally attempted this several years ago, and the Lync client (or server...can't recall), didn't deal with the *67 so well.

Incidentally, you might have issues with the alternate caller ID not being sent.  I've seen this on our own SIP provider.  Checking the SIP traces, I can see that I'm presenting the alternate caller ID as a P-Asserted-Identity, which is the way it should work, so my SIP provider seems to be ignoring it.

The example above used "callerid=blocked" as the custom parameter. You can name your custom parameter whatever you want as long as it is in lower case and uses the same format as ;parametername=value. Custom parameters have to also be placed after any ;ext= parameter.

As you might expect, I've incorporated this functionality into the Lync Optimizer.  Try it out and let me know how it works for you.

Dealing with Overdialing in Lync/Skype for Business

If you've never heard the term "overdialing", you're not alone. I hadn't heard of the term either until Greig Sheridan mentioned the term to me one day. But after reading a few definitions on the web, I realized that I knew about it, but didn't know it had a name.  Granted, the definitions I found were all from Australian sites, so maybe its an Aussie thing. My Australian exposure has gone up dramatically since joining Event Zero back in February, so maybe it's to be expected, along with the increased incidents of me saying things like "mate", "throw another shrimp on the barbie", and "a dingo ate my baby."

Anyways..."overdialing" can be thought of as what happens when people try to dial a phone number with more digits than are actually allowed by the PSTN carrier.  This often pops up with numbers that are presented as words, like 1-800-BUYSTUFF, which has 1 too many digits/letters for the North American dial plan (all numbers are 11-digits long, including the 1).

A somewhat little known fact is that you can type the above example exactly like that in Lync/SfB and it will automagically translate the letters to the corresponding numbers and show the number 1-800-289-78333.  Again, notice its got one too many digits and hasn't normalized (note the lack of + sign at the beginning of the number).

The reason for this is because most typical normalization rules only match a very specific number of digits.  Take for example, this typical North American normalization rule and translation pattern for North American national numbers:

^1?(\d{10}) --> +1$1

For the uninitiated, this normalization rule will accept any 10-digit number, with or without a leading 1, and will translate it to a valid 11-digit North American national number.  So, if you dial 5552223333 or 15552223333, it will normalize it to +15552223333.

But if you dial one or more extra digits, it will no longer match the normalization rule and will just show the digits you typed. If you typed 55522233331, and try to dial that number in Lync/Skype for Business, it may or may not fail depending on how you created your route patterns. For example, if you've structured your route pattern to accept numbers without a leading plus sign, it would route through Lync/S4B (ie.  ^\+?\d+$ would accept any number of digits with or without a plus).  I don't advise this, because its messy and makes advanced routing decisions harder. Plus it goes against the general advice of E.164 EVERYWHERE.

If you allowed a number to get through to the PSTN with too many digits, the carrier will reject any extra digits and dial the number properly, but for systems like Lync/Skype for Business, you should take care to ensure you deal with overdialed numbers seamlessly.

To make sure any overdialed numbers are normalized correctly, a simple extra few characters added to the end of a normalization rule will fix the problem nicely. Using the first example, we simply add a \d* to signify "any number of digits", followed by a $ to signify the end of the string.  So, ultimately we have:

^1?(\d{10})\d*$ --> +1$1

Now, when we dial that same number with too many digits, we get the following properly formatted normalized result:

In the end, a simple change to an otherwise generic normalization rules can end up making your users' lives just a little bit easier.

Thanks to the aforementioned Greig Sheridan, who suggested I write a blog post around this esoteric topic. I owe you a Foster's, mate!

Address Book Normalization in Skype for Business

A constant problem with Enterprise Voice in Lync is the fact that phone numbers stored in Active Directory often are not stored in the E.164 format that Lync requires for proper functionality.  If the numbers aren't recognizable, then the Lync client won't show phone numbers like office, mobile or home numbers to other users. The best thing to do is to update all phone numbers in AD to conform to E.164 standards, but this is often not practical for many reasons.

In Lync Server 2013 and older, you could use a special oddly named text file called the Company_Phone_Number_Normalization_Rules.txt to normalize AD phone numbers to E.164 standards as they are being added to the Lync Address Book.  This saved administrators a lot of work, especially when they are unable to change the core numbers in AD.  However, the fact that you had to create this text file and place it in a certain folder on every Lync server was not the most effective way to manage address book normalization.

Thankfully, things have changed in Skype for Business to finally make address book normalization part of a new suite of PowerShell commands, *-CsAddressBookNormalizationConfiguration and *-CsAddressBookNormalizationRule (where * is the typical Get, Set, New).  The Company_Phone_Number_Normalization_Rules.txt is no longer necessary and is ignored in Skype for Business.

When you first install Skype for Business, it is populated by a default set of address book normalization rules, which you can see by typing:

(Get-CsAddressBookNormalizationConfiguration).AddressBookNormalizationRules

You will get back the default set of rules stored in the Global config as below:

Description :
Pattern     : E164
Translation : null
Name        : Generic_E164

Description :
Pattern     : \++(\d+)[Xx]+(\d{1,15})
Translation : +$1;ext=$2
Name        : Generic_WithExtension

Description :
Pattern     : \++(\d+)ext[=]?(\d{1,15})
Translation : +$1;ext=$2
Name        : Generic_WithLongExtension

Description :
Pattern     : \++(\d+)EXT[=]?(\d{1,15})
Translation : +$1;ext=$2
Name        : Generic_WithLongExtensionUpper

Description :
Pattern     : \++(\d+)
Translation : +$1
Name        : Generic_All

In my opinion, these rules are not well-constructed.  Unless there is some hidden feature I'm not aware of, the E164 rule does absolutely nothing. (UPDATE 2015-Aug-13: Hany Elkady noted in the comments below that removing the E164 rule seemed to stop AD phone numbers already stored in E.164 format from appearing in the address book.  I personally haven't seen this occur but you might want to leave that rule in place.)  The other rules are oddly formatted. You can do the same in a much more flexible way with fewer rules. You can add/delete/modify rules to this by using:

 New/Remove/Set-CsAddressBookNormalizationRule

To start with a clean slate, you might want to get rid of the default rules, by running Get-CsAddressBookNormalizationRule | Where {$_.Name -ne 'Generic_E164'} | Remove-CsAddressBookNormalizationRule.

For my typical North American address book file, I add the following rules (assumes North American numbers are stored as either 10 or 11 digits without the +):

New-CsAddressBookNormalizationRule -Parent Global -Name 'NANPA-10-Digit' -Pattern '1?([2-9]\d{9})' -Translation '+1$1' -Priority 0

New-CsAddressBookNormalizationRule -Parent Global -Name 'NANPA-10-Digit-Ext' -Pattern '1?([2-9]\d{9})\D+(\d+)' -Translation '+1$1;ext=$2' -Priority 1

For a more global approach, you might use something like:

New-CsAddressBookNormalizationRule -Parent Global -Name 'Global' -Pattern '((1[2-9]\d\d[2-9]\d{6})|([2-9]\d{6,14}))' -Translation '+$1' -Priority 0 

New-CsAddressBookNormalizationRule -Parent Global -Name 'Global-Ext' -Pattern '((1[2-9]\d\d[2-9]\d{6})|([2-9]\d{6,14}))\D+(\d+)' -Translation '+$1;ext=$2' -Priority 1

The rules you add will be determined by the format phone numbers are stored in Active Directory. It appears that the normalization rule engine hasn't changed since the older versions of Lync, which means that you will still have to enter your normalization rules exactly as you did with the old Company_Phone_Number_Normalization_Rules.txt.

As with older versions of Lync, you can force an update of the address book by running Update-CsAddressBook.  Any phone numbers that aren't normalized are stored in the Invalid_AD_Phone_Numbers.txt file in your address book share under \x-WebServices-x\ABFiles\00000000-0000-0000-0000-000000000000\00000000-0000-0000-0000-000000000000.

You can import existing rules from the Company_Phone_Number_Normalization_Rules.txt file by using the command Import-CsCompanyPhoneNormalizationRules.  The RTM upgrade process really should take care of this for you, but it doesn't appear to (at least for me, but I first did Skype for Business months ago).

With this post, I hope to never again have to type the oh-so-memorable filename Company_Phone_Number_Normalization_Rules.txt.

Large Scale Extension Dialing in Lync

I was recently working a pretty large Enterprise Voice deployment that had more than a hundred sites located around the world. Every site used DIDs for their users phone numbers, but wanted to be able to reach every other site using a 3-digit site code, plus the last 4 digits of the user's phone number (total 7 digits). This would be pretty easy except for the following wrinkle: they wanted people within any one site to be able to dial just the last 4-digits of other people within that site, rather than the 7-digit "global" extension or full DID. To complicate matters, multiple sites had overlaps with the last 4 digits of their numbers.

Unfortunately, this meant that every single site would require their own dedicated Lync dial plan, with only one rule that is different from the other sites. Management of these sites can be a nightmare, especially when you consider what has to happen when a new site gets added. Not only do you have to add the other sites extension normalization rule to the new site, but every other site has to have their dial plans updated with the new site's extension normalization rule. With many sites, this can become difficult to manage.

Thankfully, Powershell can take a daunting task such as this and make it much easier. I've developed a process to manage large numbers of extension ranges that is scalable and easy to replicate at other sites. Of course, this being me, the core of it is based around Lync Optimizer generated rulesets, and the associated scripts assume you follow the same naming convention. So, without further ado....

For this particular company, I decided on a naming convention for 7-digit extension normalization rules that follow the format CountryID-StateProv-City-7Ext0x (eg. US-TX-Dallas-7Ext01, US-TX-Dallas-7Ext02 etc). 4-digit extension rules for specific sites use the format CountryID-StateProv-City-4Ext0x (eg. US-TX-Dallas-4Ext01).  Every site's dial plan would end up with a large number of 7Ext normalization rules, and a single 4Ext normalization rule specific to that site.

I kept a repository of 7-digit extension dialing rules for all sites, organized alphabetically in a manually created dialplan called ZZ-ExtensionList. This dialplan was only used for the scripts I used later to populate new site lists.

Each site used the Lync Dialing Rule Optimizer to generate rulesets for each site. I used the Extensions entry page to generate the local site's 7Ext0x normalization rule.  For this particular company, since we were using a 3-digit site code and the last 4-digits of the user's DID, the extension entry page looked something like this:

For the above example, the 3-digit site code is 350, and user's phone numbers are in the range +442055662000 to +442055662099.  The # of Ext Digits in DID field tells the Optimizer that only the last 4 digits of the 7-digit extension is actually part of the DID.

I ran the Optimizer generated script against the deployment as per usual, which generated the standard ruleset for the given site. Once complete, I opened the ZZ-ExtensionList dial plan and added the newly generated 7Ext0x rule(s) to the list, making sure to maintain alphabetical order.

I then ran the custom script AddExtensionsToDP.ps1 (see below) with the switch -DialPlan set to the name of the newly added dialplan (ie .\AddExtensionsToDP.ps1 -DialPlan UK-London) This script does two things. First, it adds all the extension dialing rules from ZZ-ExtensionList into the new dialplan. Secondly, it modifies the 7Ext0x rule(s) specific to the new site into a 4Ext0x rule(s). So, for the above example:

Original Pattern/Translation: ^350(20\d{2})$          -->    +44205566$1
New Pattern/Translation:       ^(?:350)?(20\d{2})$  -->    +44205566$1

The new pattern allows users within the site to either dial 20xx or 35020xx for other users within the site.

AddExtensionsToDP.ps1

# This script adds all the extension normalization rules from ZZ-ExtensionList to the selected dialplan # Assumes you've already generated the new dialplan via the Lync Optimizer (www.ucdialplans.com) # Then run UpdateNormRules.ps1 to add the new normalization rule to every other site [CmdletBinding(SupportsShouldProcess = $True)] param( [parameter(ValueFromPipeline = $false, ValueFromPipelineByPropertyName = $true, Mandatory = $true)] [string]$Dialplan ) $NormRuleList = Get-CSVoiceNormalizationRule | Where {$_.Identity -like 'Tag:ZZ-ExtensionList/*' -and $_.Identity -notlike "Tag:ZZ-ExtensionList/$DialPlan-7Ext*"} | Sort-Object Priority ForEach ($NormRule in $NormRuleList) { New-CSVoiceNormalizationRule -Parent $DialPlan -Name $NormRule.Name -Description $NormRule.Description -Pattern $NormRule.Pattern -Translation $NormRule.Translation -Priority $NormRule.Priority -IsInternalExtension:$TRUE } # Replace the 7Ext rule for the target site with its own custom Ext rule $7ExtRules = Get-CsVoiceNormalizationRule | Where {$_.Identity -like "Tag:$DialPlan/$DialPlan-7Ext*"} | Sort-Object Priority -Descending ForEach($7ExtRule in $7ExtRules) { $RuleName = $7ExtRule.Name $NewRuleName = $RuleName -replace "7Ext", "4Ext" $NormPattern = $7ExtRule.Pattern $NewNormPattern = $NormPattern -replace "\^(\d{3})", '^(?:$1)?' New-CSVoiceNormalizationRule -Parent $DialPlan -Name $NewRuleName -Description $7ExtRule.Description -Pattern $NewNormPattern -Translation $7ExtRule.Translation -Priority 0 -IsInternalExtension:$TRUE Remove-CSVoiceNormalizationRule $DialPlan/$RuleName }

Once that script was done, then I ran the UpdateNormRules.ps1 script (shown below) with the -NormRuleBase switch to specify the name of the dial plan created at the beginning. (ie. .\UpdateNormRules.ps1 -NormRuleBase UK-London). This script adds the normalization rules for the new site to all the other site-specific dialplans, in the proper alphabetical order.

UpdateNormRules.ps1

# This script is designed to be run after implementing a new dialplan for a site. # It will insert any extension normalization rule for that site into the dialplan for every other site in alphabetical order # Prior to running this script, insert the new normalization rule(s) in the ZZ-ExtensionList dialplan in the appropriate order. Make sure you name the rule(s) as per the following: # 2-Letter countrycode-State/Prov-City-7Ext0# (where # starts at 1), ie US-TX-Dallas-7Ext01 # # This script will copy the normalization rule to every other dialplan in the same order [CmdletBinding(SupportsShouldProcess = $True)] param( [parameter(ValueFromPipeline = $false, ValueFromPipelineByPropertyName = $true, Mandatory = $true)] [string]$NormRuleBase ) # Find the priority of the first norm rule we want to insert $NormRuleListRev = Get-CSVoiceNormalizationRule | Where {$_.Identity -like "Tag:ZZ-ExtensionList/$NormRuleBase*"} | Sort-Object Priority -Descending ForEach ($NormRulePriority in $NormRuleListRev) { $BasePriority = $NormRulePriority.Priority } $NormRulePrior = (Get-CSVoiceNormalizationRule | Where {$_.Identity -like "Tag:ZZ-ExtensionList/*" -and $_.Priority -eq $BasePriority-1}).Name $NormRulePriorPrefix = $NormRulePrior.substring(0,$NormRulePrior.LastIndexOfAny("-")) $NormRulePriorSuffix = $NormRulePrior.substring($NormRulePrior.Length-5,5) $NormRulePrior = $NormRulePriorPrefix + "*" + $NormRulePriorSuffix $DPList = Get-CsDialPlan | Where {$_.Identity -like '*-*' -and $_.Identity -ne 'Tag:ZZ-ExtensionList' -and $_.Identity -ne "Tag:$NormRuleBase" -and $_.Identity -notlike 'Service:*'} Foreach ($DP in $DPList) { $DPName = $DP.Identity $Priority = (Get-CSVoiceNormalizationRule | Where {$_.Identity -like "$DPName/$NormRulePrior"}).Priority + 1 Write-Host $DP.Identity $Priority If ($Priority -ne $NULL) { ForEach ($NormRule in $NormRuleListRev) { New-CsVoiceNormalizationRule -Name $NormRule.Name -Parent $DPName -Description $NormRule.Description -Pattern $NormRule.Pattern -Translation $NormRule.Translation -IsInternalExtension:$TRUE -Priority $Priority } } }

Once complete, the new site had all the existing site-level extension normalization rules for every other site, and all existing sites also had the new site's extension normalization rule inserted into their dial plans. This ensured that all sites could dial any other site using their site code and extension.

So, to summarize the workflow:

1. Manually create an empty dialplan called ZZ-ExtensionList
2. Generate a Lync Optimizer dialplan for your selected area, populating the Extensions using the appropriate format for your deployment. Make sure the Suffix is set to 7Ext01 (and 02 etc).
3. Run the Lync Optimizer generated ruleset against your deployment.
4. Copy the -7Ext0x rule(s) to the ZZ-ExtensionList dialplan ensuring proper alphabetical order.
5. Run the AddExtensionsToDP.ps1 script to add all the extension rules in ZZ-ExtensionList to the new dialplan.
6. Run the UpdateNormRules.ps1 to update all other dial plans with the newly created extension normalization rule.
7. Repeat steps 2-6 as necessary.

Rinse and repeat for each new dial plan you want to add to your environment. This process along with the attached scripts should make managing large dial plans much simpler. As always, be cautious when running these scripts, as I can't be responsible for any issues this causes.  Always make a backup of your existing Lync EV environment before running these or any scripts provided by strange people who pretend they look like David Hasselhoff.

Service Number Formatting in Lync

I recently had the distinct pleasure of engaging the Lync community via Twitter to ask how many people normalized their service and emergency numbers to include a plus sign (+). I asked this question because I had come across a couple of SIP providers in both the US and internationally that expected E.164 (global numbers starting with a plus sign) phone numbers for all except for emergency and service numbers. To put it another way, they wanted a plus in front of all phone numbers except for emergency/service numbers.

My very informal poll results were as follows:

Use + for everything: 6 votes
No + for emergency/service numbers: 4 votes
It depends: 1 vote

A "lively" discussion ensued between @ucomsgeek and @CAnthonyCaragol (the C stands for "Corky", in case you were wondering) on the validity of using a + sign in front of numbers that are not dialable from outside the originating country. The discussion got a little crazy and we very nearly came to blows on several occasions. @ucomsgeek had to be admitted to the hospital with what can only be described as an "RFC induced heart palpitation".

I had traditionally used a + in front of everything, adhering to my mantra of "E.164, E.164, E.164".  This included service numbers like 911 in North America, or 112 in Europe, so they would get normalized to +911 and +112.  My experiences with SIP providers not wanting a + in front of those numbers made me think, which is a rare occurrence, and is usually followed by being distracted by shiny things. If SIP providers don't want a + in front of those types of numbers, there must be a good reason for it.

So, whenever I'm faced with the great unknown, I turned to the trusty Interweb to find answers. I didn't have to look far. Phone number formatting is governed by the E.164 standard published by the Telecommunication Standardization Sector of the International Telecommunication Union or ITU-T for short (which is a good thing, because having to say "the Telecommunication Standardization Sector of the International Telecommunication Union" gets tiresome).  E.164 number representation within the Tel URI format used in SIP is governed by RFC3966 published by the Internet Engineering Task Force (IETF).

Reading these documents is an enlightening and mind-expanding experience. The E.164 standard takes a lot of pages to say that an E.164 number consists of the country code, followed by area code (national destination code or NDC) and the subscriber number.

Other than that, it doesn't say anything about plus signs or anything else other than just boring numbers, so we turn our attention to the much more interesting and relevant RFC3966.

RFC3966 has a lot to say about how to format numbers for use in computer/telecom equipment. The first thing that struck me was that the plus sign is reserved ONLY for global numbers, meaning numbers that are globally unique and adhere to the E.164 standard described above.

Section 5.1 says:

   The 'telephone-subscriber' part of the URI indicates the number.  The
   phone number can be represented in either global (E.164) or local
   notation.  All phone numbers MUST use the global form unless they
   cannot be represented as such.  Numbers from private numbering plans,
   emergency ("911", "112"), and some directory-assistance numbers
   (e.g., "411") and other "service codes" (numbers of the form N11 in
   the United States) cannot be represented in global (E.164) form and
   need to be represented as a local number with a context.  Local
   numbers MUST be tagged with a 'phone-context'

Section 5.1.4 describes how to format a global number:

   Globally unique numbers are identified by the leading "+" character.
   Global numbers MUST be composed with the country (CC) and national
   (NSN) numbers as specified in E.123 [E.123] and E.164 [E.164].
   Globally unique numbers are unambiguous everywhere in the world and
   SHOULD be used.

So, since numbers like 911 are not global numbers, they can't be referenced with a plus sign, but have to be formatted with a context, which is described a little bit later.

Section 5.1.5 describes how to format a local number (which includes service/emergency numbers by definition of not being global numbers):

   Local numbers are unique only within a certain geographical area or a
   certain part of the telephone network, e.g., a private branch
   exchange (PBX), a state or province, a particular local exchange
   carrier, or a particular country.  URIs with local phone numbers
   should only appear in environments where all local entities can
   successfully set up the call by passing the number to the dialling
   software.  Digits needed for accessing an outside line, for example,
   are not included in local numbers.  Local numbers SHOULD NOT be used
   unless there is no way to represent the number as a global number.

Section 5.1.5 goes on to say more about how to format local numbers:

   Local numbers MUST have a 'phone-context' parameter that identifies
   the scope of their validity.  The parameter MUST be chosen to
   identify the local context within which the number is unique
   unambiguously.  Thus, the combination of the descriptor in the
   'phone-context' parameter and local number is again globally unique.
   The parameter value is defined by the assignee of the local number.
   It does NOT indicate a prefix that turns the local number into a
   global (E.164) number.

What the above says is that local numbers must use a specific parameter called "phone-context" to ensure there is no ambiguity with global numbers. A little further reading brings up 911 number formatting as an example:

   A context consisting of the initial digits of a global number does
   not imply that adding these to the local number will generate a valid
   E.164 number.  It might do so by coincidence, but this cannot be
   relied upon.  (For example, "911" should be labeled with the context
   "+1", but "+1-911" is not a valid E.164 number.)

So, for example, 911 should be formatted as tel:911;phone-context=+1 for North America, but clearly says that +1911 is not acceptable.  However, Lync doesn't support phone-context in normalization rules or routing logic (trust me, I tried), so we're kinda stuck if we want to be fully RFC3966 compliant with regards to emergency and service numbers.

We're left with two options for service/emergency numbers really, either use the plus sign or don't. Since the plus sign is supposed to be reserved for global numbers, we probably shouldn't use it for emergency/service numbers. But on the other hand, people are generally used to seeing all their dialed numbers in Lync magically get formatted with a plus sign when it matches up with a normalization rule, which gets taken as a sign that you're good to go (at least that's the way I take it).

But consider this weird little thought experiment. If you normalize 911 to +911, you could think that this represents a really short number within India (country code 91). Yes, I know that a country of more than a billion people is likely going to have more than 9 phones, but you get the idea. What if India decided you could dial the Prime Minister of India by dialing "1"? Then anybody in North America trying to have an engaging conversation with Narendra Modi would end up talking to North American emergency services instead, which would be baffling for both sides.

Another possibility is that you could inadvertently route emergency calls to a global destination. Imagine you have a Lync server in India, and you decide to use least-cost routing to save on toll costs by routing any India-bound call to that Lync server. You configure a route and PSTN usage that sends any call starting with +91 to the India Lync gateway. You inadvertently put that PSTN usage above a PSTN usage meant for local service numbers. Now any call to 911 gets routed to India, and will most likely fail.

You could say the solution to the above scenario is just to make sure that service PSTN usages are above any international PSTN usage, but I'm a fan of engineering solutions that don't rely on someone setting things in exactly the proper order to get the desired results. I'm thinking of administrator/human fault tolerance here.

Rather than say the odds of any given service number conflicting with a real global number is vanishingly small (probably true, but you can't know for certain.....at least until the Lync Optimizer has dial rules for every country in the world), I'd rather ensure with 100% certainty it won't happen, by not putting a plus in front of emergency/service numbers.

Lync will happily route numbers with or without a plus, as long as your routing logic allows for it. You could build your entire Lync Enterprise Voice deployment around not using plus signs at all if you wanted to, but I wouldn't advise it, mostly because of Lync federation. Published phone numbers from federated contacts with a plus sign would not route in a deployment built around not using the plus sign.

So, there you have it. This blog post has convinced me that not using a plus sign for non-global numbers is the way to go.  I've updated the Lync Optimizer code to follow this tweak in best practices. What are your thoughts?  Does this change your view?  Does this very long blog post on the topic of a plus sign make you fear for my sanity? Let me know in the comments.

Adventures in Normalization

One of the things that differentiate Lync from its competitors is its reliance on 3rd party peripherals like deskphones. This is often touted as a Good Thing, since you're not locked into a single vendor for phones. Phones from single-vendor solutions are often more expensive than an equivalent Lync phone simply because you can't get them from anywhere else, so competition is low and prices are higher.

Since Lync phones are provided by different vendors, there is naturally a lot of variation in the specific details on how things work. Microsoft has alleviated this somewhat by instituting a Lync-qualified devices program.  This is a fairly thorough program that puts devices through their paces to ensure they function properly within Lync. 

However rigorous the program, little details slip through the cracks. One of the details that I've noticed is how different phones (and even the Lync client) handle regular expressions in different ways. For anyone familiar with the minutiae (thanks Word of the Day Calendar) of Lync Enterprise Voice, regular expressions are used for phone number normalization and routing. 

All Lync-qualified devices handle basic regex in the same manner. For most people, this is not an issue, but sometimes you need to use lesser-known regex patterns to accomplish certain goals. When you try to get fancy with regex, things sometimes break down with disparate results between clients and phones. 

Over the years, I've heard about and personally experienced situations where Lync Optimizer-generated regex doesn't work on specific phones. The affected vendors have eventually fixed these issues, but I figured that I should do some hardcore 60 Minutes style investigation into just how well different devices handle some of the more esoteric regex patterns out there. 

To do this, I used my wide variety of Lync deskphones I have available for testing in my home lab.  (You should hear what it sounds like when someone phones me.  Everything is ringing and flashing, and I usually go "AAAAhhhhh" and run out of the room.)  I used a Polycom VVX 600, a Polycom CX600, a Snom 760, and an old LG-Nortel "Tanjay" series phone.  My AudioCodes 440HD wasn't connecting to our Lync server at the time, so it was left out of testing. Every phone was updated with the latest firmware available at testing time. 

For the first round of the Lync Normalization Olympics, I decided to keep it simple, using some of the rule formats commonly used by the Lync Optimizer.  I input the normalization rules into Lync via the Control Panel, and I used the "Create voice routing test case information" to see how Lync Control Panel would interpret the regex, and included that in the results too. For all the rulesets, I used a unique "identifier" for each rule to allow me to setup a series that I could test all at once.  

Commas

First off, I used a regex rule that included a comma, which I had heard caused normalization issues with one particular vendor. In the below example, I'm looking for a pattern of numbers that starts with 999 followed by 10 or 11 digits. The 999 gets stripped and puts a + in its place.

Pattern: ^999(\d{10,11})$

Translation: +$1
Input number: 99915552223333 then 9995552223333
Expected output: +15552223333 then +5552223333

Control Panel	Lync Client	Tanjay Phone	Polycom CX	Polycom VVX	Snom
+15552223333	+15552223333	+15552223333	+15552223333	+15552223333	+15552223333
+5552223333	+5552223333	+5552223333	+5552223333	+5552223333	+5552223333

Everything passed that one with flying colours, so rumours of a phone type not handling commas seem to no longer apply.

Dealing with Extensions, Part I

Another common rule used in the Optimizer is a rule to accept any dialed number, but strip out any extensions and leave it as an externally routable E.164 number. This rule was meant mostly for click-to-dial from the Lync client, but I did hear about a particular vendor's phone not being able to parse that particular rule. In the next example, I'm looking for a pattern of numbers that starts with 998 followed by 11 digits, followed by some regex that would strip anything else beyond the 11 digits. The 998 gets stripped and puts a + in its place.

Pattern: ^998(\d{11})\d*(\D+\d+)?$

Translation: +$1
Input number: 99815552223333
Expected output: +15552223333

Control Panel	Lync Client	Tanjay Phone	Polycom CX	Polycom VVX	Snom
+15552223333	+15552223333	+15552223333	+15552223333	+15552223333	+15552223333

Everything passed that one with flying colours, so rumours of a phone type not parsing those rules seem to no longer apply.

Dealing with Extensions, Part II

The next rule test is a variation on the first example above, using a slightly different regex pattern to strip any extension. This format was used a long time ago in the Lync Optimizer, but was replaced with the "cleaner" looking version above.  One person told me he used this format to get around an issue with a particular phone vendor's issue parsing the previous example's format.

Pattern: ^997(\d{11})(\s*\S*)*?$

Translation: +$1
Input number: 99715552223333
Expected output: +15552223333

Control Panel	Lync Client	Tanjay Phone	Polycom CX	Polycom VVX	Snom
+15552223333	+15552223333	+15552223333	+15552223333	+15552223333	+15552223333

Again, no issues on any platform.

Extensions with Optional Site Code

In companies with multiple sites that provide its users with extension dialing, its often necessary to use site codes to properly differentiate one site from another. This is especially true if there are overlapping extension ranges originating from legacy PBXs. So, for a company using 4-digit extensions, they may prepend a site code to ensure uniqueness. So, Site #101 could use 1015xxx for their extension range, and Site #102 could use 1025xxx for their extension range.

One particular company wanted to ensure that users within any given site could dial their own users by extension without the prepended site code, but still allow them to enter the site code if they wanted to. I did this in one rule by using a bit of fancy regex. In the below example, a user should be able to enter the extension with or without the 996 "site code".

Pattern: ^(?:996)?(5\d{3})$$

Translation: +1555222$1
Input number: 9965000 then 5000
Expected output: +15552225000

Control Panel	Lync Client	Tanjay Phone	Polycom CX	Polycom VVX	Snom
+15552225000	+15552225000	+15552225000	+15552225000	+15552225000	+15552225000
+15552225000	+15552225000	+15552225000	+15552225000	+15552225000	+15552225000

Sigh, once again, no issues on any platform.

OK, boooooring. All the phones passed the basic regex tests without issue. Not at all "exciting". I use "exciting" in quotes, because how exciting can the topic of normalization be?

Multiple Replacement Strings

Sometimes you need to use multiple replacement strings in a rule (ie $1, $2 etc). Rumour has it some phones have issues with this.

Pattern: ^996(\d{3})000(\d{7})$

Translation: +1$1$2
Input number: 9965550002223333
Expected output: +15552223333

Control Panel	Lync Client	Tanjay Phone	Polycom CX	Polycom VVX	Snom
+15552223333	+15552223333	+15552223333	+15552223333	+15552223333	+15552223333

Running out of ways to say "No issues".

Advanced Normalization Tests 

To really gauge how well a phones' regex engine functions, we have to throw at it some regex that would only be used in very specific circumstances. 

For example, let's say I have a need to translate a set of digits into a pattern where those digits are buried within other numbers.  For example, I want to take 92xx and translate it to +155592xx999.  So, 9201 translates to +15559201999.  If you try to create a pattern like ^(92\d{2})$ and translate it to +1555$1999, there is ambiguity on whether you mean +1555 $1 999 or +1555 $19 99. 

Normally, I would follow the .NET regex reference which has different ways of dealing with this, but none of them work in the Lync client.  I did stumble upon one that did work, even though it isn’t a valid .NET expression.  Not only that, but testing with other phones gave different results most of the time.

I tried various patterns, and plugged in 9100, 9200, 9300 etc to see the results in both Control Panel and Lync softclient and desk phones. The results proved interesting.

Pattern: ^(91\d{2})$

Translation: +1555$1999
Input number: 9100
Expected output: Expect failure, but want to see how various devices handle this situation.

Control Panel	Lync Client	Tanjay Phone	Polycom CX	Polycom VVX	Snom
+1555$1999	+155599	+15559100999	+15559100999	No match	+1555

Seems as though the old Tanjay and CX-series phone work in this situation, but nothing else. No big deal, because I wouldn't expect any consistent behaviour here.

Pattern: ^(92\d{2})$

Translation: +1555${1}999
Input number: 9200
Expected output: +15559200999

Control Panel	Lync Client	Tanjay Phone	Polycom CX	Polycom VVX	Snom
+15559200999	No match	+15559200999	+15559200999	+15559200999	No match

Using ${1} is a valid way to prevent any ambiguity between $1 or $19 in this example. Sadly only the Lync client and the Snom don't respect this valid regex.

Pattern: ^(93\d{2})$

Translation: +1555$_999
Input number: 9300
Expected output: +15559300999

Control Panel	Lync Client	Tanjay Phone	Polycom CX	Polycom VVX	Snom
+15559300999	No match	No match	No match	+15559300999	No match

Using $_ is a way to substitute the entire string. Not ideal code-wise, but wanted to see if it would work.  Only the Control Panel and Polycom VVX phones properly parsed it.

Pattern: ^(?<1>94\d{2})$

Translation: +1555${1}999
Input number: 9400
Expected output: +15559400999

Control Panel	Lync Client	Tanjay Phone	Polycom CX	Polycom VVX	Snom
+15559400999	No match	No match	No match	+15559400999	No match

This is a valid .NET regex method to use named replacement strings instead of numeric. So, I could have easily used ^(?<DigItDude>94\d{2})$ -->  +1555${DigItDude}999.  Again, only the Control Panel and Polycom VVX phones properly parsed it.

Pattern: ^(95\d{2})$

Translation: +1555($1)999
Input number: 9500
Expected output: +15559500999

Control Panel	Lync Client	Tanjay Phone	Polycom CX	Polycom VVX	Snom
+1555(9500)999	+15559500999	+15559500999	+15559500999	+15559500999	+1555(9500)999

This is actually not a valid .NET regex method. Surprisingly, all the devices except for the Snom managed to parse it correctly.

Conclusions

As long as your phone number normalization requirements don't require any digit replacement buried within other digits, you can safely use just about any Lync-qualified device.  However, if you know in advance that you have such a requirement (and its very unlikely that you would), I would avoid Snom phones at this time, simply because they couldn't handle any of the advanced regex I threw at them in my attempts to solve that specific issue.

The way I see it, if I build a regex pattern in Lync Control Panel, I should be able to trust that every single device will parse that regex pattern the same way.  I find it rather annoying that there isn't any rigorous checking to see if vendors adhere to .NET regex standards. On that front, I feel Microsoft needs to be more diligent with their phone qualification process.  Its these little details that can lead to frustration for anybody deploying Lync Enterprise Voice.

If you got to the end of this post and are still not asleep.....congratulations!  This post has been entered in the "Most Boring Tech Post of the Year" for 2015. Yes, I know its early, but I think this is an early front-runner for the win.