Dynamic Multipoint VPN (DMVPN) and IGP Routing Protocols

Dynamic Multipoint VPN (DMVPN) is a multipoint GRE-based tunnelling technology. DMVPN consists of one or more hub routers that are configured as Next-Hop Resolution Protocol (NHRP) Next-Hop Servers (NHS). NHS, or hubs, are used to create mappings between the public IP address used for the tunnel source (NBMA address), and the private IP address used inside of the tunnel (tunnel address).

The goal of the DMVPN network is to allow any-to-any communication over the private tunnel network, while at the same time not requiring that a full mesh of point-to-point tunnels be formed. Instead, tunnels can be formed on-demand based on the particular destination of traffic. This allows DMVPN designs to achieve very large scalability, because a full mesh of n*(n-1)/2 tunnels is not required.

From a configuration point of view, there are two roles in the DMVPN network, the hub(s) and the spoke(s). The spokes maintain a manual/static mapping of the hub’s tunnel address to NBMA address, while the hub dynamically learns about the spokes through Next Hop Resolution Protocol (NHRP) messages. The hub and spokes must agree on certain parameters, such as NHRP authentication, GRE tunnel key number, and whether multicast will be supported.

 

Next Hop Resolution Protocol (NHRP):

NHRP is similar in function to ARP, but does that in more efficient manner, suitable for partially meshed NBMA clouds supporting dynamic layer 2 connections.

One of the functions of Next Hop Client (NHC) is to register with NHS its private IP address mapped to NBMA address. To make registration possible, you configure each NHC with the IP address of at least one NHS. In turn, NHS acts as a database agent, storing all registered mappings, and replying to NHC queries (see Figure 2). If NHS does not have a requested entry in its database, it can forward packet to another NHS to see if it has the requested association. Note that a router may act as a Next-Hop server and client at the same time.

Although DMVPN is a multipoint tunnel, it is not a multicast tunnel. This means that the GRE source IP address and the GRE destination IP address are always unicast. Multicast is, however, supported inside of the tunnel, but essentially as a replicated. Multicast support is added to DMVPN by the spokes manually mapping multicast to the hub’s NBMA address with the “ip nhrp map multicast” command, whereas the hub maps multicast as “dynamic”.

 

DMVPN Phases and NHRP

NHRP  and DMVPN Phase 1

With NHRP Phase 1 mGRE uses NHRP to inform the hub about dynamically appearing spokes. Initially, every spoke is configured with the IP address of the hub as the NHS server. However, the spoke’s tunnel mode is GRE (regular point-to-point) tunnel with a fixed destination IP that equals to the IP address of the hub. The spokes can only reach the hub and only get to other spoke networks across the hub.

The benefit of Phase 1 is simplified hub router configuration, which does not require static NHRP mapping for every new spoke.

NHS – Hub Configuration:

 

NHCs –Spokes configuration:

 

Spokes only learn routes from the hub. This is because multicasts cannot be directly replicated between the spokes. Spokes would only become adjacent with the hub, simply because the spokes do not see each other’s hello packets. Hello packets have a TTL value of 1, so they are not routed. As a result the hub is always the next hop for the spokes in Phase 1.

An example of a Spoke to Spoke traffic flow is shown in Figure 3.

In DMVPN Phase 1, traffic cannot be directly exchanged between the spokes. There is no spoke-to-spoke dynamic tunnel negotiated.

Table 1 shows the routing configuration for different IGPs for DMVPN Phase 1.

PROTOCOL	HUB	SPOKES
EIGRP	router eigrp DUCONET ! address-family ipv4 unicast autonomous-system 10 ! network 192.168.1.0 0.0.0.255 network 172.16.1.0 0.0.0.255 ! af-interface Tunnel0 no passive-interface ! Split horizon must be disabled for spokes to learn routes from other spokes through the hub. no split-horizon !If a default route is advertised to the spokes, it is actually not required to disable split horizon. Spokes only learn default route from the hub and no more specific prefixes from spokes through the hub.      summary-address 0.0.0.0 0.0.0.0	router eigrp DUCONET ! address-family ipv4 unicast autonomous-system 10 network 192.168.1X.0 0.0.0.255 network 172.16.1X.0 0.0.0.255 ! Spokes receive the default route from the hub and spokes are configured as stub eigrp stub connected
OSPF	interface Tunnel0 ! Point-to-Multipoint will rewrite the nexthop to the hub’s tunnel IP address.This is required in Phase 1, all traffic goes through the hub ip ospf network point-to-multipoint ! Optional: If spokes have a static default route configured, OSPF LSAs can be safely filtered ip ospf database-filter all out router ospf 10 network 192.168.1.0 0.0.0.255 area 0 network 172.16.1.0 0.0.0.255 area 0	interface Tunnel0 ip ospf network point-to-multipoint ! Optional: ip route 0.0.0.0 0.0.0.0 172.16.1.1 router ospf 10 network 192.168.1X.0 0.0.0.255 area 0 network 172.16.1X.0 0.0.0.255 area 0
RIP	interface Tunnel0 ! Split horizon must be disabled for spokes to learn routes from other spokes through the hub. no ip split-horizon !If a default route is advertised to the spokes, it is actually not required to disable split horizon. Spokes only learn default route from the hub and no more specific prefixes from spokes through the hub. router rip version 2 network 192.168.1.0 network 172.16.1.0 default-information originate route-map set_interface ! route-map set_interface permit 10 set interface tunnel0!	router rip ! version 2 network 192.168.1X.0 network 172.16.1X.0

Figure 4 shows the advertisement of a default route from the hub to the spokes. Spoke 1 learns the default route with the next-hop value set to the hub’s tunnel IP address. Therefore, any spoke to spoke traffic is going across the hub in DMVPN Phase 1.

The following show commands can help to understand the NHRP Resolution in DMVPN Phase 1 and the routing decisions when EIGRP is configured in the network:

 

If, for example, there is a lot of spoke to spoke traffic in the network this design may not be desirable.

NHRP  and DMVPN Phase 2

DMVPN Phase 2 was the first optimization for DMVPN designs, which required frequent spoke to spoke traffic patterns.

Recall that in DMVPN Phase 1, all spoke to spoke traffic was always tunnelled through the hub, before being tunnelled back to the remote spokes. This design creates a bottleneck in the network, because the hub is not only responsible for maintaining the NHRP control plane for all spokes, but also the spoke to spoke data plane. To resolve this problem, DMVPN Phase 2 allows the spokes to form on-demand GRE tunnels, which can then be used for direct data plane exchange without packets having transit the hub.

Spokes:

 

To accomplish this, DMVPN Phase 2 requires that spokes learn about all specific routes in the topology, and that the next-hop value of spoke to spoke routes not be modified to the hub’s tunnel IP address. This requirement implies two key points about DMVPN Phase 2:

• First, that the hub cannot summarize routes between the spokes.
• Second, that the routing protocol used must support maintaining the next-hop of the spokes’ routes.

The preservation of the routing next-hop in DMVPN Phase 2 means that the spokes must now perform NHRP resolutions for each other’s addresses. Previously, the spokes only did static NHRP resolution for the hub, whereas the hub did dynamic resolution to the spokes.

NHCs send NHRP Resolution Requests to the hub in order to resolve other NHC’s addresses.  NHS looks into its database to check if already knows the NBMA to the Tunnel IP mapping for that specific NHC. If this NHC was previously registered to the NHC, this will already know this information and will reply to the query with a NHRP Resolution Reply. Figure 5 illustrates this process.

At this point, see Figure 6, NHC 1 knows the NBMA associated with NHC 2’s tunnel IP address (172.16.1.12 – 22.22.22.12). Because the hub does not change the next-hop address of the routes learned from the spokes when advertised to other spokes, now spoke 1 is able to dynamically create a tunnel to spoke 2 thanks to NHRP Resolution. Traffic does not need to go through the hub in Phase 2. Because the next-hop address needs to remain on the prefixes advertised, summarization is not allowed in the hub. Otherwise it would be modified and spoke to spoke traffic will break.

Within the scope of OSPF, this implies that the OSPF network type on the DMVPN tunnel needs to be either Broadcast or Non-Broadcast, as the DR does not modify the next-hop value in LSAs exchanged across the segment. Furthermore we need to ensure that none of the DMVPN spokes are elected as the OSPF DR, as this will break flooding on the segment, since the spokes do not have direct multicast reachability to each other.

Table 2 shows the routing configuration for different IGPs in DMVPN Phase 2.

PROTOCOL	HUB	SPOKES
EIGRP	router eigrp DUCONET ! address-family ipv4 unicast autonomous-system 10 ! network 192.168.1.0 0.0.0.255 network 172.16.1.0 0.0.0.255 ! af-interface Tunnel0 no passive-interface ! disable Split Horizon so spokes learn other spokes’ routes and do not modify the next hop when advertise no split-horizon no next-hop-self	router eigrp DUCONET ! address-family ipv4 unicast autonomous-system 10 network 192.168.1X.0 0.0.0.255 network 172.16.1X.0 0.0.0.255 ! Spokes receive all routes from the hub with the original next hop. Spokes are configured as stub eigrp stub connected
OSPF	interface Tunnel0 ip ospf priority 255 ip ospf network broadcast router ospf 10 network 192.168.1.0 0.0.0.255 area 0 network 172.16.1.0 0.0.0.255 area 0	interface Tunnel0 ip ospf network broadcast ip ospf priority 0 router ospf 10 network 192.168.1X.0 0.0.0.255 area 0 network 172.16.1X.0 0.0.0.255 area 0
RIP	interface Tunnel0 ! disable Split Horizon so spokes learn other spokes’ routes and do not modify the next hop when advertise no ip split-horizon no ip next-hop-self router rip  version 2 network 192.168.1.0 network 172.16.1.0	router rip ! version 2 network 192.168.1X.0 network 172.16.1X.0

The following show commands can help to understand the NHRP Resolution for DMVPN Phase 2 and the routing decisions when EIGRP is configured in the network:

 

NHRP and DMVPN Phase 3

DMVPN Phase 3 increases scalability of the network by minimizing the amount of routing information that the spokes need to maintain, while still allowing for on-demand spoke-to-spoke tunnels.

NRHP Phase 3 involves spokes in responding to NHRP resolution requests. Thus, the unique role of hub as the only source of NHRP information is reduced. Summarization is allowed to be send down to the spokes or even sending a default route is allowed in Phase 3.

Hub:

Spokes:

 

Figure 7 shows the initial topology, where the hub sends a default route to the spokes.

Spoke 1 sends traffic with destination 192.168.12.1 and source 192.168.11.1. Because its routing table does not contain a specific entry for 192.168.12.1/32, traffic is forwarded to the hub using the default route. Spoke 1 already has the static NHRP mapping for the hub in its tunnel configuration.

When the NHS receives an IP packet inbound on its mGRE tunnel interface from Spoke 1 and then switches it out of the same interface to Spoke 2, it sends back to the source a special NHRP Redirect message (see Figure 8). This message tells the NHC that it is using the suboptimal path to route packets, and that it should look for a better way using NHRP resolution. The original packet, first packet, is still routed using the routing table through the hub.

The important field in the NHRP Redirect received from the hub is the destination, 192.168.12.1, as its payload. Spoke 1 uses this information to create a NHRP Resolution Request containing that destination and its own tunnel and NBMA addresses (172.16.1.11 and 22.22.22.11). This request is sent to spoke 2 via the hub. The target of the NHRP Resolution Request is the final prefix, the destination 192.168.12.1, not the next-hop address as it was in Phase 1 and Phase 2.  The NHRP Resolution Request travels via regular IP routing path (means across the hub, as the hub originated the default route to spoke 1) until it reaches the target spoke 2. Figure 9 shows how the hub is able to relay the request to the correct spoke by looking at the next hop for the destination 192.168.12.1 contained in the NHRP Request.

Now the destination NHC, spoke 2, responds to the NHRP Resolution Request. Based on the IP found in the payload, it finds the corresponding prefix in its IP routing table. Using the NBMA IP of the source NHC, spoke 1, the local NHC sends the NHRP Reply back directly to the source NHC; NHRP Reply does not traverse the hub as it is shown in Figure 10.  Note that the reply contains the whole IP routing prefix found in RIB, not just the requested IP address.  This is used to indicate to spoke 1 for which destination a dynamic tunnel can be built. Spoke 2 also uses the information in the NHRP Request to build an NHRP dynamic entry for spoke 1’s tunnel IP address 172.16.1.11, mapping it to the NBMA address 22.22.22.11.

When spoke 1 receives this NHRP Reply, it knows the NBMA next hop of the prefix it was asking about (22.22.22.12 ßà 192.168.12.1). Now, it populates the NHRP table based on the NHRP Reply.

Return traffic will typically also need to be sent and will need to go through the exact same process.

Finally a dynamic spoke to spoke tunnel is established as illustrated in Figure 11.

In Phase 3 a dynamic tunnel is created, through the process explained above, for a destination prefix. This is why summarization and a default route is allowed in the hub. The NHRP Resolution is not triggered for the next-hop value anymore. Every time that a destination prefix is not found in the RIB, the NHRP Resolution process is triggered to generate a dynamic tunnel for that specific destination prefix.

The following show commands can help to understand the NHRP Resolution for DMVPN Phase 3 and the routing decisions when EIGRP is configured in the network and the hub is advertising a default route to the spokes:

 

First packet goes through the hub:

 

Next packets go directly to spoke 2:

 

 References

http://labs.ine.com

http://blog.ine.com/2008/08/02/dmvpn-explained/

https://fredrikjj.wordpress.com/2014/08/16/dmvpn-phase-3-spoke-to-spoke-tunnels/