PART-1 covers below details

  • Hub and Spoke Configuration
  • NHRP registration
  • Flags in NHRP registration request
  • EIGRP route summarization on HUB
  • Spoke to Spoke tunnel

Before start our discussion on DMVPN Phase III, let’s discuss few pros and cons of DMVPN Phase II.

DMVPN Phase II

Pros:

  1. Hub and Spokes are configured as multipoint GRE tunnel
  2. Traffic between spokes can flow via dynamic spoke to spoke tunnel

Cons:

  1. Restriction on route summarization
  2. Limitation on NHS scalability
  3. Glean adjacency for next-hop in CEF table

DMVPN Phase III

Let’s build our DMVPN Phase III tunnels to understand the steps.

Diagram:

dmvpn-image 

1.  Hub and Spoke Configuration

HUB Config SPOKE-1 Config SPOKE-2 Config
interface Tunnel0
ip address 123.0.0.1 255.255.255.0
no ip redirects
no ip split-horizon eigrp 1
ip nhrp map multicast dynamic
ip nhrp network-id 1
ip nhrp redirect
tunnel source Ethernet0/0
tunnel mode gre multipoint
end
interface Tunnel0
ip address 123.0.0.2 255.255.255.0
no ip redirects
ip nhrp map 123.0.0.1 10.10.0.1
ip nhrp map multicast 10.10.0.1
ip nhrp network-id 1
ip nhrp nhs 123.0.0.1
ip nhrp shortcut
tunnel source Ethernet0/1
tunnel mode gre multipoint
end
interface Tunnel0
ip address 123.0.0.3 255.255.255.0
no ip redirects
ip nhrp map 123.0.0.1 10.10.0.1
ip nhrp map multicast 10.10.0.1
ip nhrp network-id 1
ip nhrp nhs 123.0.0.1
ip nhrp shortcut
tunnel source Ethernet0/2
tunnel mode gre multipoint
end

With this above configuration our DMVPN tunnels between hub and spokes should come up. Let’s see how NHRP registration happens.

2.  NHRP registration

Debug NHRP packets on SPOKE-1 Debug NHRP packets on HUB
 SPOKE-1#
*Dec 4 05:35:26.632: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
*Dec 4 05:35:26.633: NHRP: Send Registration Request via Tunnel0 vrf 0, packet size: 92
*Dec 4 05:35:26.633: src: 123.0.0.2, dst: 123.0.0.1
*Dec 4 05:35:26.633: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Dec 4 05:35:26.633: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 05:35:26.633: pktsz: 92 extoff: 52
*Dec 4 05:35:26.633: (M) flags: “unique nat “, reqid: 3
*Dec 4 05:35:26.633: src NBMA: 10.0.1.1
*Dec 4 05:35:26.633: src protocol: 123.0.0.2, dst protocol: 123.0.0.1
*Dec 4 05:35:26.633: (C-1) code: no error(0)
*Dec 4 05:35:26.633: prefix: 32, mtu: 17916, hd_time: 7200
*Dec 4 05:35:26.633: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0*Dec 4 05:35:26.637: NHRP: Receive Registration Reply via Tunnel0 vrf 0, packet size: 112
*Dec 4 05:35:26.638: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Dec 4 05:35:26.638: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 05:35:26.638: pktsz: 112 extoff: 52
*Dec 4 05:35:26.638: (M) flags: “unique nat “, reqid: 3
*Dec 4 05:35:26.638: src NBMA: 10.0.1.1
*Dec 4 05:35:26.638: src protocol: 123.0.0.2, dst protocol: 123.0.0.1
*Dec 4 05:35:26.638: (C-1) code: no error(0)
*Dec 4 05:35:26.638: prefix: 32, mtu: 17916, hd_time: 7200
*Dec 4 05:35:26.638: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0
HUB#

*Dec 4 05:35:26.634: NHRP: Receive Registration Request via Tunnel0 vrf 0, packet size: 92
*Dec 4 05:35:26.635: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Dec 4 05:35:26.635: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 05:35:26.635: pktsz: 92 extoff: 52
*Dec 4 05:35:26.635: (M) flags: “unique nat “, reqid: 3
*Dec 4 05:35:26.635: src NBMA: 10.0.1.1
*Dec 4 05:35:26.635: src protocol: 123.0.0.2, dst protocol: 123.0.0.1
*Dec 4 05:35:26.635: (C-1) code: no error(0)
*Dec 4 05:35:26.635: prefix: 32, mtu: 17916, hd_time: 7200
*Dec 4 05:35:26.635: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0
*Dec 4 05:35:26.635: NHRP: Send Registration Reply via Tunnel0 vrf 0, packet size: 112
*Dec 4 05:35:26.635: src: 123.0.0.1, dst: 123.0.0.2
*Dec 4 05:35:26.635: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Dec 4 05:35:26.635: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 05:35:26.635: pktsz: 112 extoff: 52
*Dec 4 05:35:26.635: (M) flags: “unique nat “, reqid: 3
*Dec 4 05:35:26.635: src NBMA: 10.0.1.1
*Dec 4 05:35:26.635: src protocol: 123.0.0.2, dst protocol: 123.0.0.1
*Dec 4 05:35:26.635: (C-1) code: no error(0)
*Dec 4 05:35:26.635: prefix: 32, mtu: 17916, hd_time: 7200
*Dec 4 05:35:26.635: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0

As soon as the tunnel comes up on Spoke, it sends a NHRP registration request to its NHS (HUB). The NHS responds back with a NHRP registration reply **.

**HUB almost sends the same request packet (without disturbing the M section) back to SPOKE as the acknowledgement (reply).

3.  Flags in NHRP registration request

Different fields inside the NHRP registration request

‘F’ Field – This is the fixed part of the request. It contains the overall information of the underlay protocol. Here NHRP is running over IPv4.

From our earlier debug output

 (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
shtl: 4(NSAP), sstl: 0(NSAP)
pktsz: 112 extoff: 52

       ‘M’ Field – This is the mandatory part. This contains the senders NBMA address, source protocol (tunnel IP of Spoke) and Destination protocol (tunnel IP of HUB/NHS). Using these information HUB populates its dmvpn table.

  (M) flags: “unique nat “, reqid: 3
src NBMA: 10.0.1.1
src protocol: 123.0.0.2, dst protocol: 123.0.0.1

        ‘C-1’ Field – This is the Client Information Element part. It contains the information about mtu and hold down timers configured on the sender. This field contains the remote end router’s NBMA address to the Tunnel IP address.

(C-1) code: no error(0)
prefix: 32, mtu: 17916, hd_time: 7200
addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0

Have a look at the prefix information in this field as well. This plays a big role in DMVPN phase III. We will discuss that during spoke to spoke traffic flow (I will cover in the next blog DMVPN Phase III).

Now coming back to our topology, let’s run EIGRP between Hub and Spokes and advertise the loopback interfaces.

HUB Config SPOKE-1 Config SPOKE-2 Config
router eigrp 1
network 123.0.0.1 0.0.0.0
network 192.168.100.1 0.0.0.0
network 192.168.200.1 0.0.0.0
router eigrp 1
network 123.0.0.2 0.0.0.0
network 192.168.1.1 0.0.0.0
network 192.168.2.1 0.0.0.0
router eigrp 1
network 123.0.0.3 0.0.0.0
network 192.168.10.1 0.0.0.0
network 192.168.20.1 0.0.0.0

Just think for a moment. With the above config do the spokes learn about each other networks?
Let’s check in the Spoke routing table.

SPOKE-1#show ip route eigrp

D 192.168.100.0/24 [90/27008000] via 123.0.0.1, 00:02:43, Tunnel0
D 192.168.200.0/24 [90/27008000] via 123.0.0.1, 00:02:43, Tunnel0
SPOKE-1#

SPOKE-1#ping 192.168.10.1 >> Pinging SPOKE-2 network
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.1, timeout is 2 seconds:
UUUUU
Success rate is 0 percent (0/5)
SPOKE-1#

WAIT !!!  Where are the routes from SPOKE-2.
SPOKE-1 only learned the routes from HUB. (Believe me the same story on SPOKE-2. It did not learn the routes of SPOKE-1. It has only routes from HUB.)
Why it is not learning the routes from SPOKE-2?

Yes. The answer is the split-horizon rule. The networks learned via tunnel 0 on hub are not being sent out via the same tunnel 0. That’s why spokes are not learning each other’s routes.
To fix the issue we can turn off the split-horizon rule on tunnel 0 of HUB. But wait then how it is different from DMVPN Phase II.

Let’s implement the first advantage of DMVPN Phase III over Phase II i.e the route summarization

4.  EIGRP route summarization on HUB

Tunnel config with EIGRP summarization on HUB
interface Tunnel0
ip address 123.0.0.1 255.255.255.0
no ip redirects
ip nhrp map multicast dynamic
ip nhrp network-id 1
ip nhrp redirect
ip summary-address eigrp 1 192.168.0.0 255.255.0.0
tunnel source Ethernet0/0
tunnel mode gre multipoint
end

Now check on the Spokes

SPOKE-1#show ip route eigrp

D 192.168.0.0/16 [90/27008000] via 123.0.0.1, 00:01:25, Tunnel0
SPOKE-1#

SPOKE-2#show ip route eigrp

D 192.168.0.0/16 [90/27008000] via 123.0.0.1, 00:02:00, Tunnel0
SPOKE-2#

5. Spoke to Spoke dynamic tunnel

Debug NHRP on HUB Debug NHRP on SPOKE-1
HUB#
*Dec 4 08:19:21.050: NHRP: Send Traffic Indication via Tunnel0 vrf 0, packet size: 84
*Dec 4 08:19:21.050: src: 123.0.0.1, dst: 123.0.0.2
*Dec 4 08:19:21.050: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Dec 4 08:19:21.050: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 08:19:21.050: pktsz: 84 extoff: 68
*Dec 4 08:19:21.050: (M) traffic code: redirect(0)
*Dec 4 08:19:21.050: src NBMA: 10.10.0.1
*Dec 4 08:19:21.050: src protocol: 123.0.0.1, dst protocol: 123.0.0.2
*Dec 4 08:19:21.050: Contents of nhrp traffic indication packet:
*Dec 4 08:19:21.051: 45 00 00 64 00 14 00 00 FE 01 76 D9 7B 00 00 02
*Dec 4 08:19:21.051: C0 A8 0A 01 08 00 77 5E 00 04 00
*Dec 4 08:19:21.054: NHRP: Send Traffic Indication via Tunnel0 vrf 0, packet size: 84
*Dec 4 08:19:21.054: src: 123.0.0.1, dst: 192.168.10.1
*Dec 4 08:19:21.054: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Dec 4 08:19:21.054: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 08:19:21.054: pktsz: 84 extoff: 68
*Dec 4 08:19:21.054: (M) traffic code: redirect(0)
*Dec 4 08:19:21.054: src NBMA: 10.10.0.1
*Dec 4 08:19:21.054: src protocol: 123.0.0.1, dst protocol: 192.168.10.1
*Dec 4 08:19:21.054: Contents of nhrp traffic indication packet:
*Dec 4 08:19:21.054: 45 00 00 64 00 14 00 00 FE 01 76 D9 C0 A8 0A 01
*Dec 4 08:19:21.054: 7B 00 00 02 00 00 7F 5E 00 04 00
*Dec 4 08:19:21.065: NHRP: Receive Resolution Request via Tunnel0 vrf 0, packet size: 72
*Dec 4 08:19:21.065: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Dec 4 08:19:21.065: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 08:19:21.065: pktsz: 72 extoff: 52
*Dec 4 08:19:21.065: (M) flags: “router auth src-stable nat “, reqid: 5
*Dec 4 08:19:21.065: src NBMA: 10.0.2.1
*Dec 4 08:19:21.065: src protocol: 123.0.0.3, dst protocol: 123.0.0.2
*Dec 4 08:19:21.065: (C-1) code: no error(0)
*Dec 4 08:19:21.065: prefix: 32, mtu: 17916, hd_time: 7200
*Dec 4 08:19:21.065: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0
*Dec 4 08:19:21.065: NHRP: Forwarding Resolution Request via Tunnel0 vrf 0, packet size: 92
*Dec 4 08:19:21.065: src: 123.0.0.1, dst: 123.0.0.2
*Dec 4 08:19:21.066: (F) afn: AF_IP(1), type: IP(800), hop: 254, ver: 1
*Dec 4 08:19:21.066: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 08:19:21.066: pktsz: 92 extoff: 52
*Dec 4 08:19:21.066: (M) flags: “router auth src-stable nat “, reqid: 5
*Dec 4 08:19:21.066: src NBMA: 10.0.2.1
*Dec 4 08:19:21.066: src protocol: 123.0.0.3, dst protocol: 123.0.0.2
*Dec 4 08:19:21.066: (C-1) code: no error(0)
*Dec 4 08:19:21.066: prefix: 32, mtu: 17916, hd_time: 7200
*Dec 4 08:19:21.066: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0
SPOKE-1#ping 192.168.10.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/6/13 ms
SPOKE-1#
*Dec 4 08:19:21.063: NHRP: Receive Traffic Indication via Tunnel0 vrf 0, packet size: 84
*Dec 4 08:19:21.063: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Dec 4 08:19:21.063: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 08:19:21.063: pktsz: 84 extoff: 68
*Dec 4 08:19:21.063: (M) traffic code: redirect(0)
*Dec 4 08:19:21.063: src NBMA: 10.10.0.1
*Dec 4 08:19:21.063: src protocol: 123.0.0.1, dst protocol: 123.0.0.2
*Dec 4 08:19:21.063: Contents of nhrp traffic indication packet:
*Dec 4 08:19:21.063: 45 00 00 64 00 14 00 00 FE 01 76 D9 7B 00 00 02
*Dec 4 08:19:21.063: C0 A8 0A 01 08 00 77 5E 00 04 00
*Dec 4 08:19:21.070: NHRP: Receive Resolution Request via Tunnel0 vrf 0, packet size: 92
*Dec 4 08:19:21.070: (F) afn: AF_IP(1), type: IP(800), hop: 254, ver: 1
*Dec 4 08:19:21.070: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 08:19:21.070: pktsz: 92 extoff: 52
*Dec 4 08:19:21.070: (M) flags: “router auth src-stable nat “, reqid: 5
*Dec 4 08:19:21.070: src NBMA: 10.0.2.1
*Dec 4 08:19:21.070: src protocol: 123.0.0.3, dst protocol: 123.0.0.2
*Dec 4 08:19:21.070: (C-1) code: no error(0)
*Dec 4 08:19:21.070: prefix: 32, mtu: 17916, hd_time: 7200
*Dec 4 08:19:21.070: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0
*Dec 4 08:19:21.071: NHRP: Send Resolution Reply via Tunnel0 vrf 0, packet size: 120
*Dec 4 08:19:21.071: src: 123.0.0.2, dst: 123.0.0.3
*Dec 4 08:19:21.071: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Dec 4 08:19:21.071: shtl: 4(NSAP), sstl: 0(NSAP)
*Dec 4 08:19:21.071: pktsz: 120 extoff: 60
*Dec 4 08:19:21.071: (M) flags: “router auth dst-stable unique src-stable nat “, reqid: 5
*Dec 4 08:19:21.071: src NBMA: 10.0.2.1
*Dec 4 08:19:21.071: src protocol: 123.0.0.3, dst protocol: 123.0.0.2
*Dec 4 08:19:21.071: (C-1) code: no error(0)
*Dec 4 08:19:21.071: prefix: 32, mtu: 17916, hd_time: 7200
*Dec 4 08:19:21.071: addr_len: 4(NSAP), subaddr_len: 0(NSAP), proto_len: 4, pref: 0
*Dec 4 08:19:21.071: client NBMA: 10.0.1.1
*Dec 4 08:19:21.071: client protocol: 123.0.0.2

The below steps get executed when hub receives a packet from SPOKE-1 towards SPOKE-2. At the end of these steps, Spoke to Spoke dynamic tunnel is created.
Step 1:
HUB sent traffic indication (redirect) to SPOKE-1 (Time stamp Dec 4 08:19:21.050)
Step 2:
SPOKE-1 received the traffic indication (Time stamp Dec 4 08:19:21.063)
Step 3:
SPOKE-2 received the ICMP echo and sent the resolution request. (debug on SPOKE 2 was not captured)
Step 4:
HUB received a resolution request from SPOKE-2 for SPOKE-1 (Time stamp Dec 4 08:19:21.065)
Step 5:
HUB forwarded that resolution request to SPOKE-1 (Time stamp Dec 4 08:19:21.065)
Step 6:
SPOKE-1 sent the resolution reply to SPOKE-2 directly (Time stamp Dec 4 08:19:21.071)

SPOKE-1#show dmvpn
Legend: Attrb –> S – Static, D – Dynamic, I – Incomplete
N – NATed, L – Local, X – No Socket
# Ent –> Number of NHRP entries with same NBMA peer
NHS Status: E –> Expecting Replies, R –> Responding, W –> Waiting
UpDn Time –> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel0, IPv4 NHRP Details
Type:Spoke, NHRP Peers:2,

# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
—– ————— ————— —– ——– —–
1 10.10.0.1 123.0.0.1 UP 00:13:53 S
2 10.0.2.1 123.0.0.3 UP 00:11:40 DT1
123.0.0.3 UP 00:11:40 DT1

SPOKE-1#show ip route eigrp

D 192.168.0.0/16 [90/27008000] via 123.0.0.1, 00:13:40, Tunnel0
SPOKE-1#

Hold ON !!!!
We still don’t have any route pointing to Spoke-2 in the EIGRP table. Nothing is changed in the EIGRP route table as well. The next hop is still pointing to HUB tunnel IP and there is no indication of next hop override (%).
Is the traffic going via HUB again? Let’s check

SPOKE-1#traceroute 192.168.10.1 num
Type escape sequence to abort.
Tracing the route to 192.168.10.1
VRF info: (vrf in name/id, vrf out name/id)
1 123.0.0.3 8 msec 6 msec 6 msec
SPOKE-1#

This is the magic of DMVPN Phase III. We will get the specific route via NHRP.

SPOKE-1#show ip route 192.168.10.1 255.255.255.0
Routing entry for 192.168.10.0/24
Known via “nhrp“, distance 250, metric 1
Last update from 123.0.0.3 on Tunnel0, 00:16:57 ago
Routing Descriptor Blocks:
* 123.0.0.3, from 123.0.0.3, 00:16:57 ago, via Tunnel0
Route metric is 1, traffic share count is 1
MPLS label: none

SPOKE-1#show ip route nhrp

123.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
H              123.0.0.3/32 is directly connected, 00:18:08, Tunnel0
H        192.168.10.0/24 [250/1] via 123.0.0.3, 00:18:08, Tunnel0
SPOKE-1#

Let’s have a look at the cef table entry for both these routes.

SPOKE-1#show ip cef 123.0.0.3/32 internal
123.0.0.3/32, epoch 0, flags attached, connected, rib only nolabel, rib defined all labels, RIB[O], refcount 5, per-destination sharing
sources: RIB, Adj
feature space:
IPRM: 0x00058004
subblocks:
Adj source: IP midchain out of Tunnel0, addr 123.0.0.3 B53130B8
Dependent covered prefix type adjfib, cover 123.0.0.0/24
ifnums:
Tunnel0(24): 123.0.0.3
path B367A810, path list B5346404, share 1/1, type attached nexthop, for IPv4
nexthop 123.0.0.3 Tunnel0, adjacency IP midchain out of Tunnel0, addr 123.0.0.3 B53130B8
output chain: IP midchain out of Tunnel0, addr 123.0.0.3 B53130B8 IP adj out of Ethernet0/1, addr 10.0.1.2 B4FC8328
SPOKE-1#

SPOKE-1#show ip cef 192.168.10.0/24 internal
192.168.10.0/24, epoch 0, flags rib only nolabel, rib defined all labels, RIB[O], refcount 5, per-destination sharing
sources: RIB
feature space:
IPRM: 0x00058000
ifnums:
Tunnel0(24): 123.0.0.3
path B367A810, path list B5346404, share 1/1, type attached nexthop, for IPv4
nexthop 123.0.0.3 Tunnel0, adjacency IP midchain out of Tunnel0, addr 123.0.0.3 B53130B8
output chain: IP midchain out of Tunnel0, addr 123.0.0.3 B53130B8 IP adj out of Ethernet0/1, addr 10.0.1.2 B4FC8328
SPOKE-1#

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s