That did not go as planned. Doubt started to cripple down my spine.
It has to work I kept mumbling to myself, but ITSP and Cisco CUBE would hear none of it, numerous times..
Edit: I recommended using the CUBE Config Utility in conjunction with this post
SIP doesn’t usually stress me out anymore and after a dozen of expressway implementation neither does NATed VOIP. But apparently Murphy always keeps another trick up his sleeve.
The topology was pretty straight forward. Cisco CUBE was in a DMZ, behind an Internet facing FW which was doing the NAT for us.
Cisco CUBE: An unknown identity
It started off with a loud squeak, a sign of what’s about to come..
I set up the SIP Trunk from CUCM towards Cisco CUBE and from Cisco CUBE towards ITSP (Internet Telephony Service Provider) and tried to call.
First try, no luck.
Went over my configuration again. All checked out fine.
The first setback came early in the game forcing me to look under the hood.
*I didn’t want to use the customer logs so I replicated the issues in my own environment.
Debug ccsip messagesThe outcome was similar to this:
Sent:
INVITE sip:pashat@iptel.org:5060 SIP/2.0
Via: SIP/2.0/UDP 192.168.1.20:5060;branch=z9hG4bK2C21C6
Remote-Party-ID: "ucpros" <sip:1505@192.168.1.20>;party=calling;screen=yes;privacy=off
From: "ucpros" <sip:1505@iptel.org>;tag=410D9A-182
To: <sip:pashat@iptel.org>
Date: Wed, 31 Aug 2016 11:33:26 GMT
Call-ID: 90F1E4EF-6EA511E6-8015F4F4-11BF0349@192.168.1.20
Supported: timer,resource-priority,replaces,sdp-anat
Min-SE: 1800
Cisco-Guid: 3121089664-0000065536-0000000003-0167880896
User-Agent: Cisco-SIPGateway/IOS-15.5.3.S2
Allow: INVITE, OPTIONS, BYE, CANCEL, ACK, PRACK, UPDATE, REFER, SUBSCRIBE, NOTIFY, INFO, REGISTER
CSeq: 101 INVITE
Timestamp: 1472643206
Contact: <sip:1505@192.168.1.20:5060>
Call-Info: <sip:192.168.1.20:5060>;method="NOTIFY;Event=telephone-event;Duration=2000"
Expires: 180
Allow-Events: telephone-event
Max-Forwards: 68
Session-Expires: 1800
Content-Length: 0
.
.
.
Received:
SIP/2.0 604 Unknown Identity
Via: SIP/2.0/UDP 192.168.1.20:5060;branch=z9hG4bK2C21C6;received=80.1.1.1
From: "ucpros" <sip:1505@iptel.org>;tag=410D9A-182
To: <sip:pashat@iptel.org>;tag=5A36FA5D-57C6C0860003EE14-43C32700
Call-ID: 90F1E4EF-6EA511E6-8015F4F4-11BF0349@192.168.1.20
CSeq: 101 INVITE
Server: ser (3.3.0-pre1 (i386/linux))
Warning: 392 192.168.0.85:5060 "Noisy feedback tells: pid=1391 req_src_ip=192.168.0.86 req_src_port=5060 in_uri=sip:pashat@iptel.org:5060 out_uri=sip:pashat@iptel.org:5060 via_cnt==1"
Content-Length: 0Unknown identity, that’s a new one. I was expecting a digest challenge with a
407 Proxy Authentication Required but it wasn’t my day now was it..
It was about to be a long evening and i had to make it shorter.
I added the Alias command so that writing sip would invoke the command debug ccsip messages.
CUBE(config)#alias exec sip debug ccsip messagesAfter some brainstorming, googling and testing i was onto it. Apparently, ITSP expects to see your allocated DID range / username in the From field whenever you make a call.
My was showing the internal directory which I had to change to my allocated username.
From: "ucpros" <sip:1505@iptel.org>;tag=410D9A-182
Shows internal extension: wrong
From: "ucpros" <sip:ucpros@iptel.org>;tag=410D9A-182
Shows allocated username: right
From: "ucpros" <sip:08865423561@iptel.org>;tag=410D9A-182
Shows DID number from allocated range: rightUsually you would just have to update the External Phone Number Mask on the CUCM so it would match ITSP’s expectations, but if your case is a bit more complicated you would have to use SIP profiles to get the job done.
You can use the following set of commands to modify the From header
voice class sip-profiles 1
request INVITE sip-header From modify "(<.*:)(.*@)" "ucpros@"This command replaces anything that is before the @ with my ITSP username.
You would then have to attach it the proper outgoing dial-peer.
Dial-peer voice 1 voip
Description outgoing to ITSP
voice-class sip profiles 1
You can find very helpful documentation with real life examples of how to do it here.
You can also test your SIP profiles before you apply it with this great tool.
One-Way voice
After a rough start I was hoping for some improvement but success wasn’t giving itself up that easily. Unlike the setbacks, which continued pouring in, in buckets.
I could call now, but that was not enough.
Audio? One way, can’t hear a thing. Not a big surprise when you are behind NAT.
A quick debug ccsip messages revealed that voice packets are sent but none are received.
SIP BYE message indicated that PR=0 (Packets Received).
Received:
BYE sip:135601EA-57C6DA3C000A0C9F-43F35700@192.168.1.20:5060;transport=tcp SIP/2.0
Via: SIP/2.0/TCP 192.168.1.10:5060;branch=z9hG4bK2b4048cc72
From: "ucpros" <sip:100@192.168.1.10>;tag=193~caf26ba8-507b-44f8-ac22-461d3d689977-18522847
To: <sip:pashat@iptel.org>;tag=A5803C-1389
Date: Wed, 31 Aug 2016 13:23:08 GMT
Call-ID: d358b80-7c61da3c-16-a01a8c0@192.168.1.10
User-Agent: Cisco-CUCM11.0
Max-Forwards: 70
P-Asserted-Identity: "ucpros" <sip:100@192.168.1.10>
CSeq: 102 BYE
Reason: Q.850;cause=16
Session-ID: 25a321302cb9fe0e3578b2978beaa192;remote=a1fd93dd8b22cbd2ca21f6fe533ab193
P-RTP-Stat: PS=970,OS=155200,PR=0,OR=0,PL=0,JI=0,LA=0,DU=19
Content-Length: 0Red lights and yellow light bulbs were flooding my brain.
“You did configure the FW to allow SIP ALG, right?” I asked the customer.
“Sure!” he replied.
It was agreed that the customer’s FW would provide NAT support via SIP ALG (Application Layer Gateway).
The point of ALG is to check the Application layer of the packet (the SIP header in our case) and replace any IP addresses (usually LAN addresses) with it’s own address.
Something was missing, the FW wasn’t doing it’s job.
After a few sighs and clicks, the customer asked me to try again.
The second hurdle was behind me, but it was far from being the end of it.
No incoming calls
I could make outbound calls just fine but incoming calls were intermittently failing. I was baffled, I had to see what was going on the Cisco CUBE side.
Debug ccsip messages was there for the rescue once again.
My goal was to first verify that I’m getting the proper invite from ITSP.
I was. And then I wasn’t. When calls were going through I could see the proper invite reach the Cisco CUBE. When they didn’t, Cisco CUBE was desolated. There were no SIP packets coming in.
“How could it be?” I was thinking.
I had to narrow it down, find some consistencies and patterns.
I found that calls from some sources were always reaching Cisco CUBE, while others were never coming through. Who is to blame??
I had to check again. I registered my softphone (i like the free MicroSIP) to the ITSP and tried to call it. Calls were reaching it no matter what was the source.
YES!! It was my fault! A sense of hope flooded my body.
If I am to blame, surely I can fix it.
Quickly, what is the difference between Cisco CUBE and my softphone? Err.. Registration! I have to check my registration status.
CUBE#show sip-ua register statusFlat-line sound was filling my ears. I was so sure about my configuration that I missed the obvious. After all, outgoing calls and even some of the incoming ones were working well..
CUBE#show sip-ua register status
Registrar is not configuredI was missing the registrar command under SIP-UA. Outgoing calls were working because I had the authentication command in place which would pass the auth’ challenge from ITSP.
CUBE(config)sip-ua
CUBE(config-sip-ua)#credentials username ucpros password 7 0442A4334 realm iptel.org
CUBE(config-sip-ua)#registrar dns:iptel.org expires 3600CUBE#show sip-ua register status
Line peer expires(sec) reg survival P-Associ-URI
================================ ========== ============ === ======== ============
ucpros -1 23 yes normal
Registered!
The end was close, or so I thought.
CUCM: 404 Not Found
I could see the incoming INVITE messages from the ITSP but something had changed and I couldn’t quite point my finger on it. My Jabber wasn’t ringing. Now what??
Poker night was getting further away while a failed cut-off was getting closer by the minute.
I had to figure out what was wrong.
Debug ccsip messages.
Incoming INVITE. Check.
Outgoing INVITE to CUCM. Check.
200 OK from CUCM. Missing. How come?
Went through all of the CUCM incoming call flow. Flawless.
Back to Cisco CUBE. Checking out the SIP messages I’ve noticed something was wrong.
First look, nothing, looks great. The To field in the invite message to CUCM contained the right number.
Sent:
INVITE sip:UCPROS-TRUNK@192.168.1.10:5060 SIP/2.0
Via: SIP/2.0/UDP 192.168.1.20:5060;branch=z9hG4bK12C7B7
Remote-Party-ID: <sip:pashat@192.168.1.20>;party=calling;screen=no;privacy=off
From: <sip:pashat@iptel.org>;tag=BED29A-150B
To: <sip:821101@192.168.1.10>
Date: Wed, 31 Aug 2016 18:47:10 GMT
Call-ID: 28CC4870-6EE211E6-802DCA05-6CA790D5@192.168.1.20
Supported: 100rel,timer,resource-priority,replaces,sdp-anat
Min-SE: 1800
Cisco-Guid: 0684376435-1860309478-2150091269-1822920917
User-Agent: Cisco-SIPGateway/IOS-15.5.3.S2
Allow: INVITE, OPTIONS, BYE, CANCEL, ACK, PRACK, UPDATE, REFER, SUBSCRIBE, NOTIFY, INFO, REGISTER
CSeq: 101 INVITE
Timestamp: 1472669230
Contact: <sip:pashat@192.168.1.20:5060>
Call-Info: <sip:192.168.1.20:5060>;method="NOTIFY;Event=telephone-event;Duration=2000"
Expires: 180
Allow-Events: telephone-event
Max-Forwards: 2
Session-Expires: 1800
Content-Type: application/sdp
Content-Disposition: session;handling=required
Content-Length: 270
v=0
o=CiscoSystemsSIP-GW-UserAgent 3195 4421 IN IP4 192.168.1.20
s=SIP Call
c=IN IP4 192.168.1.20
t=0 0
m=audio 8020 RTP/AVP 0 8 101
c=IN IP4 192.168.1.20
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=ptime:20Yet CUCM couldn’t find it.
Received:
SIP/2.0 404 Not Found
Via: SIP/2.0/UDP 192.168.1.20:5060;branch=z9hG4bK12C7B7
From: <sip:pashat@iptel.org>;tag=BED29A-150B
To: <sip:821101@192.168.1.10>;tag=380~caf26ba8-507b-44f8-ac22-461d3d689977-18522858
Date: Wed, 31 Aug 2016 18:47:10 GMT
Call-ID: 28CC4870-6EE211E6-802DCA05-6CA790D5@192.168.1.20
CSeq: 101 INVITE
Allow-Events: presence
Reason: Q.850;cause=1
Server: Cisco-CUCM11.0
Session-ID: 00000000000000000000000000000000;remote=a1fd93dd8b22cbd2ca21f6fe533ab380
Content-Length: 0I had to take another look, something was amiss.
Right! The SIP request URI, it made no sense.
SIP-REQ-URI is the first row of the SIP header and it was showing a variation of my ITSP username while the To field had the called number. Problem is, CUCM uses the SIP-REQ-URI to route calls!
Timing couldn’t be worse but I knew what I had to do.
SIP profiles is the way to customize SIP headers in Cisco CUBE.
Not the easiest job on the planet but definitely one of the most powerful features Cisco CUBE has to offer.
My first try was to replace the SIP INVITE header.
voice class sip-profiles 2
request INVITE sip-header To copy "sip:(.*)@" u10
request INVITE sip-header SIP-Req-URI modify "UCPROS-TRUNK@(.*)" "\u10@"we would then have to associate the SIP Profile to the incoming dial peer.
voice service voip
sip
sip-profiles inbound
Dial-peer voice 2 voip
Description Incoming from ITSP
voice-class sip profiles 2 inboundBasically what I did here was to copy the called number from the To header to the SIP-REQ-URI header instead of my ITSP username.
It was working! I was exuberant. The happiest person within a hundred mile radius!
Until I wasn’t.
Test calls had strange behaviors.
They wouldn’t disconnect, reconnect or properly placed on hold. Basically any mid call feature was misbehaving in one way or another.
My patience was running on fumes now.
I was already late for the first round of bets and I really wanted the cut-off to end smoothly. No loose ends on this one.
I had to see what was happening SIP wise on the Cisco CUBE.
Debug ccsip messages revealed the problem.
Even though I changed the INVITE SIP header to reflect the proper called number to CUCM, all of the other messages that had abnormal SIP-REQ-URI were just blanks for CUCM.
It couldn’t understand BYE or any other SIP message that were coming from the ITSP.
Consequently, this one was pretty straight forward.
All i had to do was to change the INVITE to ANY on my SIP-profile to apply the changes on all of the incoming SIP messages from ITSP.
It would look something like that:
Voice class sip-profiles 2
request ANY sip-header To copy "sip:(.*)@" u10
request ANY sip-header SIP-Req-URI modify "UCPROS-TRUNK@(.*)" "\u10@"Success!! At last! What’s more, I managed to make it to the poker game before the blinds went up. Not to bad for a Tuesday night.
Epilogue
Let’s try to wrap things up here. There are few things that you would have to pay attention to when things go south on your CUBE implementation.
- First of all, debug ccsip messages is your best friend, use it. do notice that sometimes this debug won’t show you the register messages to your Registrar and you would have to use debug ccsip all. Only use the all command in a non-production router or during a maintenance window as it might crash your router.
- Check your From field in outgoing calls. If the number/user is not associated to you by the ITSP, your call will probably be rejected.
- Don’t forget to verify your registration status with show sip-ua register status.
Do not get confused by outgoing calls that are working.
Remember that you will need both the Authentication and the Credentials commands under sip-ua. - One-way or no-way audio is pretty common. It usually caused by NAT issues.
If your Cisco CUBE has only LAN interfaces, remember that someone has to replace the IP Addresses in the application layer (SIP headers), so either use SIP-Profiles or SIP-ALG on the FW. - CUCM routes calls using the SIP-REQ-URI header, not the To header.
Do verify what you receive from ITSP and change accordingly using SIP-Profiles.
In addition, there are some very good docs and tools out there, don’t be afraid to use it. - I have written a multi-part blog post on how to configure CUBE step-by-step, you can find very useful information there. Part 1 Part 2
Let me know if you have any questions or had any issues with your ITSP deployment in the comments!
Best of luck!
Build pro IOS configs. FAST.
