Location-Based Routing without Inbound Call Restrictions

The February 2013 Update for Lync Server 2013 (CU1) added location-based routing to the Lync Enterprise Voice featureset. As described on Technet and various other blogs, location-based routing was designed to function in countries where routing calls over the WAN is not allowed.

Countries like India apparently have rather strict telecom laws saying any calls that exit the country must exit via the PSTN.  For instance, imagine your company has a Lync deployment in India and Canada.  As I understand it, if a user sitting in India wanted to call a Canadian PSTN number, it couldn't save on international toll charges by routing the call to Canada via the corporate WAN and exit to the PSTN via the Canadian Lync deployment. The converse situation would also not be allowed.  Whether this is because India wants the international toll charge revenue or for monitoring purposes, I don't know.

Prior to Lync 2013 CU1, companies faced with these requirements would be OK when users never moved from their assigned location.  For instance, if a Canadian user never went to the India office, and the India users always stayed in India, they would never run afoul of the rules. If a Canadian user did travel to India, their Canadian voice policy would still apply (unless the administrator moved the user temporarily to the India Lync server).  Their calls would route over the WAN to the Canadian Lync server, which is not allowed by India telecom rules.

Location-based routing was obviously added to Lync to deal with the "India problem".  You can see this in the Technet documentation on how to setup location-based routing.  All the examples use India, and if you follow the documentation explicitly, you will have an India telecom compliant Lync deployment.  If our hypothetical Canadian user travels to the India office, their calls will use the India Lync deployment, rather than use the WAN for least cost routing.  Inbound calls to the Canadian user's Canadian phone number will go to voicemail, because letting him take the call via the WAN would run afoul of the India dial rules.

So, this got me thinking. What if we want to setup location-based routing, but not worry about whether or not inbound calls can route over the WAN?  I can think of many situations where location-based routing is desirable. Any place where you have lots of mobile people going to different offices can benefit from location-based routing, especially in environments where bandwidth back to the user's home pool is lacking.

Luckily, I'm working on a Lync 2013 deployment that has that exact scenario in mind. The company has many mobile users who travel to far-flung offices with poor WAN conditions.  I wanted to see if we could implement location-based routing for this company, but without the restrictions on inbound call routing imposed if you follow the Technet documentation to the letter.

The documentation has you setting up all your sites and subnets, and then running the following commands (as per http://technet.microsoft.com/en-us/library/jj994036.asp):

1. New-CSVoiceRoutingPolicy to create the routing policies and the PSTN usages assigned to it
2. Set-CSNetworkSite to enable location-based routing for the site via -EnableLocationBasedRouting:$TRUE, and to assign the previously created voice routing policy
3. Set-CSTrunkConfiguration to assign network sites and enable routing restrictions on the PSTN trunk, using -EnableLocationRestriction:$TRUE.
4. Set-CSVoicePolicy to enable location-based routing for the users assigned to that policy, using -PreventPSTNTollBypass:$TRUE

We setup location-based routing as per the documentation with only ONE key difference...we didn't enable location-based routing on the trunks as per the examples given.  When running Set-CSTrunkConfiguration, we left out the -EnableLocationRestriction:$TRUE setting.

Technet's documentation on this particular setting seems to indicate not setting -EnableLocationRestriction to $TRUE would make location-based routing not work. In actual fact, this is the setting that stops inbound PSTN calls from working while a user is at a site where call routing restrictions apply (like India).

The setting that I THOUGHT would have allowed inbound calls across the WAN to work was Set-CSVoicePolicy -PreventPSTNTollBypass:$FALSE.  This setting is actually the one that controls which users will get location-based routing applied to them.  Setting it to false disables location-based routing for the users of that voice policy, as indicated by the wording on the location-based routing setup page. It does not actually prevent PSTN calls from reaching users over the WAN, as the Technet documentation on the command indicates.

We've tested this setup and it works as we had hoped it would. When users go to remote sites, their outbound calls use the local PSTN egress point, but inbound calls to their PSTN phone number still routes to them properly.

So, in conclusion, if you want to enable location-based routing without inbound call restrictions, don't use Set-CSTrunkConfiguration -EnableLocationRestriction:$TRUE, or conversely, use Set-CSTrunkConfiguration -EnableLocationRestriction:$FALSE if you've already set it to $TRUE.

Fixing Call Issues in a Mitel 3300 - Lync 2013 Deployment

We recently deployed Lync Server 2013 Enterprise Voice for a customer who at the same time replaced their old Mitel PBX with a newer Mitel 3300. The customer wasn't ready to make a full commitment to Lync for voice traffic, but probably 80% of the customer's users are using Lync exclusively for voice.

The Lync infrastructure was connected to the PSTN via the Mitel 3300 PBX using direct SIP.  While things initially seemed to be fine, we were getting numerous reports of sporadic failed calls throughout each day.

Digging through the monitoring server reports (deploying monitoring server reports should be considered ESSENTIAL for any Lync Enterprise Voice deployment), we could see a clear trend for the call failures: Looking at the Failure Distribution Report, we could see the top failed call reason was Gateway side Media negotiation failed.

Clicking on the sessions number hyperlink brought us to the list of calls that failed due to that cause.  Selecting one shows the session detail report for that call:

At the bottom is the Diagnostic Report.  Clicking the Detail link on the server side finally shows the exact reason for the call failure:

10010; source="LyncFE.contoso.com"; reason="Gateway side Media negotiation failed"; component="MediationServer"; ValidationError="SDP Exchange already failed before SetAnswer: Session c line = 0.0.0.0 or ::, call hold not allowed for initial INVITE offer"; GatewayFqdn="10.1.1.2; trunk=NA-OH-Dayton-Mitel"

I did a search through my normal channels and didn't come up with much.  I saw a few references to clients who set the Lync client audio port range to 3, rather than the recommended 20-40 which caused their particular issue, but we were already using 40 as a client port range.

It seemed as though the issue was coming from the Mitel side.  I tried a few things like setting the trunk's RTCPActiveCalls and RTCPCallsonHold values to FALSE on the Lync side, but that had no effect.

We tried to contact the Mitel partner who put in the Mitel system, but they were of very little help.  I asked if I could poke around the Mitel web console to look for a potential fix. Surprisingly, the client said "Sure".  Having never seen a Mitel before, I thought that things were about to go very wrong, or at least we wouldn't get any closer to a solution.

Since the error was related to SDP, we focussed on the SDP Options tab under SIP Peer Profile on the Mitel.  The most promising option was Avoid Signalling Hold to the Peer.  It was promising because the error message from the Lync monitoring reports mentioned both SDP and hold, so.......we set it to Yes, and at the same time we set Enable Mitel Proprietary SDP to No.  Everything else we left alone.

Almost immediately, our problems came to an end. Calls to numbers that would fail every 2nd or 3rd attempt suddenly worked without error every single time. Another side benefit was that we were now getting music on hold when transferring to Mitel call queues, where we weren't before.

I was amazed that we had brought up the issues around both music on hold and the failed calls on numerous occasions, and the Mitel partner could not find a solution, while someone with zero experience with Mitel was able to find a solution in under 10 minutes of poking around.

I was feeling pretty good about fixing that issue, so I tried to solve the other remaining issue where outbound calls would only show the main office number rather than the individual Lync user's DID. However, my beginners luck ran out and I couldn't find a solution.

So, if you're having similar issues with a Mitel - Lync deployment, hopefully this will steer you in the right direction.

Inter-trunk Routing in Lync 2013

Introduction - What is Inter-trunk Routing?

Inter-trunk routing is a new feature in Lync 2013 that allows Lync Server to be the central call processor for a multi-PBX deployment.  In previous versions, Lync was only capable of being the final destination for phone calls.  If an inbound phone call couldn't be matched to a Lync user, the call would fail.  There was no way to have Lync forward calls on to another PBX, which limited the configuration options available to administrators.

Inter-trunk routing gives Lync administrators the ability to take calls in from a variety of sources and forward them on to other non-Lync destinations if there isn't a matching Lync user with the target phone number. 

The Problem

Recently, I had to deploy inter-trunk routing for a Lync deployment that had calls coming in from a variety of sources to a centralized Lync Server 2013 Enterprise Edition deployment:

• a Lync-certified SIP trunk
• a Canada-based T1 line connected to a PSTN gateway
• two US-based Mitel PBXs. 

The inter-trunk routing requirements were very specific: forward four phone numbers entering Lync from the SIP trunk to one of the US-based PBXs.  The four phone numbers were +1 (289) 555-3926, +1 (289) 555-6733, +1 (289) 555-6734 and +1 (289) 555-6780.

The Solution

Having never done Lync inter-trunk routing, I turned to the trusty Internet for answers.  I found lots of information on WHAT Lync inter-trunk routing was all about, but almost nothing on exactly HOW to configure inter-trunk routing in Lync.  I saw references to assigning PSTN usages to trunks, and saw the place in the Lync Control Panel to add it, but nothing but vague allusions to the process of actually implementing it.

UPDATE 10-June-2013: Richard Brynteson posted a more knowledgable deep-dive into inter-trunk routing mere days after I posted this. I made a slight adjustment to my process based on what I read in his blog, so its definitely worth reading.

As usual, when I come across such a situation, I assume that it must be so simple to configure that it didn't occur to anyone to actually document the HOW.  I then assume I must be a complete idiot for not knowing something so plainly obvious to every other Lync administrator on the planet. So, after much flailing around, I figured out a solution that works using inter-trunk routing.

Since I wanted to route only four specific phone numbers between trunks, I needed to make sure that I enabled inter-trunk routing for those numbers and no others.  If I allowed inter-trunk routing for too wide a range of phone numbers, we could get into a situation where calls could enter Lync, pass onto the target PBX and route back into Lync, causing a call loop.  Rather than leave it to the target PBX to ensure a call loop doesn't happen, I figured it best to ensure Lync only pass the specific numbers to the PBX.

I started by editing the site-level voice policy attached to the Lync site where the incoming SIP trunk calls came into Lync.  I created a dedicated PSTN usage/route combination for those four numbers.  The PSTN usage was called NA-ON-Cooksville-289555-MitelForward to keep with the naming convention used by the Lync Dialing Rule Optimizer.  The route was called NA-ON-Cooksville-289555-MitelForward with a route pattern of ^\+1289555(3926|67(33|34|80))$ pointing to the PBX at 1.1.1.1. 

The PSTN usage was placed at the very top of each voice policy's PSTN usage list to ensure that any Lync user that tries to dial those numbers will forward the calls to the desired PBX. For good measure, I also added the PSTN usage to the Global voice policy, although I'm not sure this is necessary.

Next, I assigned the new PSTN usage to the incoming SIP trunk under Trunk Configuration.  If the trunk doesn't exist, add it as a new pool trunk.  I then added a trunk for the PBX and added trunk translation rules to strip the number down to 5 digits, which is what the target PBX is expecting.

Once done, I committed the changes and waited a few minutes for the changes to bake to a nice golden brown.

The moment of truth came when I was able to call one of those four numbers and listen as Lync seamlessly passed the calls onto the Mitel PBX.  Great success!

Conclusion

I'd like to say it was my deep, Yoda-like knowledge of how call routing works in Lync that allowed me to think through the process of determining how inter-trunk routing works. Unfortunately, the truth is that I tried random combinations of things until I stumbled blindly onto the solution.  There's a reason why I chose that picture of the Hoff looking vacant and stunned at the same time as my avatar....it's how I look and feel much of the time.

To summarize the process:

1. Create a PSTN usage/route combination that is specific to the phone numbers required to be forwarded to the PBX.
2. Place the PSTN usage at the top of all voice policies to ensure that both inbound calls from the PSTN and calls made by internal Lync users are routed to the proper destination.
3. Assign the PSTN usage to the incoming trunk.

When complete, the whole call process should function as follows:

1. Inbound call comes into Lync via SIP trunk.
2. Lync normalizes the number into an E.164 phone number
3. Lync does a reverse number lookup to check for a Lync user with the same number
4. If no match in Lync, then it looks for a matching usage/route assigned to a trunk
5. When found, applies any outbound normalization rules and sends the call to the next hop

For a more detailed description on how the call flow will proceed, check out the Mastering Lync blog.  Richard Brynteson did a series on the nitty gritty details behind dialing behaviours in Lync. His post on Inter-trunk dialing is a deep-dive into the details that I don't get into here.

If there's anyone else out there trying to figure out how to implement Lync inter-trunk routing, I hope this helps figure things out.  As always, if you have any questions or can correct any misunderstanding I might have on the process, just leave a comment.