DMVPN: Designing and Investigating Dynamic VPN Overlays
Dynamic Multipoint VPN (DMVPN) combines multipoint GRE, Next Hop Resolution Protocol (NHRP), IPsec and dynamic routing to build scalable VPN connectivity across an underlying IP network.
The engineering challenge is not simply creating a secure tunnel. A DMVPN deployment must dynamically discover remote endpoints, establish the appropriate tunnel path, maintain routing information and provide efficient communication between sites as the network grows.
This guide moves beyond configuration syntax and examines DMVPN as an operational system. You will follow the relationship between the underlay and overlay, observe how NHRP provides endpoint resolution, investigate how spoke-to-spoke connectivity develops, and correlate routing, tunnel and security evidence when troubleshooting the network.
DMVPN is built from several technologies that solve different parts of the connectivity problem. mGRE provides multipoint tunnelling, NHRP resolves dynamic tunnel endpoints, IPsec provides encryption, and routing protocols determine how networks are reached.
The interactive sections turn DMVPN behaviour into an engineering investigation: identify the path, inspect the evidence, determine what changed and verify the resulting network state.
By the end of the guide, you should be able to explain how DMVPN forms and optimises its overlay, identify the evidence created by NHRP and IPsec, distinguish hub-and-spoke from direct spoke-to-spoke forwarding, and use multiple observations to isolate a DMVPN failure.
Investigation Path
Follow the DMVPN dependency chain from architecture and discovery through security, routing, forwarding and fault investigation.
Understand the DMVPN Architecture
Start with the dependency chain. DMVPN is not a single protocol: the underlay, mGRE, NHRP, IPsec and routing functions each solve a different engineering problem. Before troubleshooting traffic, establish which layer is responsible for connectivity, discovery, security, routing and forwarding.
SEE — DMVPN Architecture
Before investigating a DMVPN failure, understand how the underlay, mGRE, NHRP, IPsec and routing functions fit together to create the overlay.
The underlying IP network provides basic reachability between DMVPN routers. If the underlay cannot reach the required peer, the overlay cannot form correctly.
Hub-and-Spoke
Traffic remains logically centred on the hub. The hub provides the central connectivity point for the spokes.
Spoke-to-Spoke
Spokes can dynamically establish direct connectivity when the routing and NHRP information support the destination.
Hub-Initiated
The hub can participate in redirecting traffic toward a more efficient spoke-to-spoke path.
Engineering View
DMVPN should not be investigated as a single protocol. A tunnel can appear operational while NHRP mappings, IPsec security associations or routing information are incorrect.
The investigation therefore follows the dependency chain: Underlay → mGRE → NHRP → IPsec → Routing → Forwarding.
SEE — DMVPN Architecture
Underlay Reachability
The underlay provides IP reachability between the physical or NBMA endpoints. DMVPN cannot establish useful overlay behaviour if the transport network cannot reach the required peers.
show ip route 198.51.100.1
How NHRP Builds Dynamic Peer Knowledge
Once the tunnel framework exists, DMVPN needs a way to associate logical tunnel addresses with real NBMA destinations. NHRP provides that discovery and mapping function, allowing spokes and the NHS to build the peer information required by the overlay without turning NHRP itself into the routing protocol.
DISCOVER — NHRP and Dynamic Peer Mapping
mGRE provides the multipoint tunnel framework, but it does not tell a DMVPN router where another peer exists on the underlay. NHRP provides the discovery and mapping mechanism that connects the logical tunnel address with the peer's NBMA address.
NHRP Mapping View
| DEVICE | ROLE | TUNNEL | NBMA | STATE |
|---|---|---|---|---|
| HUB | NHS | 172.16.100.1 | 198.51.100.1 | LOCAL |
| SPOKE 1 | NHC | 172.16.100.11 | 198.51.100.11 | REGISTERED |
| SPOKE 2 | NHC | 172.16.100.12 | 198.51.100.12 | PENDING |
Engineering Interpretation
NHRP is not the routing protocol that determines which destination network should be used. Its role is to provide the information required to reach another DMVPN peer by relating a logical tunnel address to an NBMA address.
This makes NHRP an important dependency when investigating dynamically established DMVPN connectivity.
DISCOVER — Dynamic Peer Mapping
198.51.100.11
198.51.100.1
198.51.100.12
| Node | Logical | NBMA | State |
|---|---|---|---|
| HUB | 172.16.100.1 | 198.51.100.1 | LOCAL |
| SP1 | 172.16.100.11 | 198.51.100.11 | REGISTERED |
| SP2 | 172.16.100.12 | 198.51.100.12 | PENDING |
NHRP is not a routing protocol. Its role here is to provide the information needed to map a logical DMVPN peer to its NBMA transport address.
show ip nhrp
show ip nhrp nhs
show dmvpn
show ip interface tunnel 0
The mapping establishes how a DMVPN logical peer relates to its underlying transport endpoint. It does not, by itself, prove that IPsec is established or that the routing table contains the correct destination path.
How IPsec Protects the DMVPN Overlay
NHRP can identify a peer, but discovery alone does not provide security. IPsec establishes the protection required for DMVPN traffic, securing the encapsulated overlay between peers. A tunnel interface being operational is therefore not sufficient evidence that encrypted traffic is successfully passing.
PROTECT — IPsec Security
NHRP can identify where a DMVPN peer exists, but discovery does not provide confidentiality or integrity. IPsec supplies the security layer that protects DMVPN traffic as it crosses the underlying network.
Security Association Evidence
Engineering Interpretation
A working tunnel interface does not by itself prove that protected traffic can pass. IPsec security associations must be established and traffic must be processed by the correct security policy.
When troubleshooting, separate the questions: Is the peer reachable? Is the security association established? Are packets actually being encrypted and decrypted?
PROTECT — IPsec Security
Follow the security establishment process from IKE negotiation through an operational IPsec SA and protected DMVPN traffic.
IKE establishes the negotiation context. At this point, an operational IPsec SA has not yet been demonstrated.
A DMVPN tunnel interface being operational does not by itself prove that encrypted traffic is successfully passing between peers.
How Routing Determines the Overlay Path
DMVPN provides the overlay connectivity, but routing protocols determine which destination prefixes are reachable and which path is preferred. EIGRP, OSPF and BGP can operate across the overlay, each applying its own path-selection logic.
ROUTE — Routing Across the DMVPN Overlay
DMVPN provides the overlay connectivity, but routing determines which destination prefixes are reachable and which path the router prefers. EIGRP, OSPF and BGP can operate across the overlay, with the routing design influencing how traffic moves between spokes and hubs.
EIGRP Routing Model
EIGRP can exchange routes across the DMVPN overlay and is commonly used in Cisco-centric DMVPN designs. Its metrics and next-hop behaviour influence the selected path.
Engineering Interpretation
A DMVPN tunnel being operational does not mean that the expected route will be installed. The overlay provides connectivity between routers; the routing protocol decides which prefixes are reachable and which path should be preferred.
Investigation Question
If the destination prefix exists but traffic takes the wrong path, investigate the routing information before assuming that the DMVPN tunnel itself is broken.
Following the Packet Through the Overlay
A DMVPN tunnel is not the packet itself. It is the transport framework that allows one router to carry an original IP packet across an underlying network. To understand forwarding behaviour, separate the inner packet from the DMVPN overlay and the underlay transport.
Consider a packet travelling from a host behind Spoke 1 to a host behind Spoke 2. The original packet might have the source 10.10.10.10 and destination 10.40.40.40. Those addresses describe the actual communication. They are not replaced simply because the packet enters the DMVPN tunnel.
The Inner Packet
The original IP packet contains the addresses used by the overlay routing decision and, ultimately, the destination host. For example:
SRC 10.10.10.10
DST 10.40.40.40
PROTOCOL IP
Spoke 1 uses its routing and forwarding information to determine that the destination prefix is reachable through the DMVPN overlay. The packet is therefore handed towards the tunnel interface rather than being forwarded directly using the physical underlay interface.
mGRE Creates the Overlay Transport
mGRE provides the multipoint tunnel framework. Instead of requiring a separate permanent tunnel interface for every possible peer, a DMVPN router can use a single multipoint GRE tunnel to support dynamically discovered peers.
The important engineering distinction is that mGRE provides the tunnel mechanism, while NHRP provides the peer discovery and logical-to-NBMA mapping information required to reach dynamic peers.
Original source and destination addresses remain associated with the actual IP communication.
mGRE carries the original packet across the DMVPN tunnel infrastructure.
The transport network carries the encapsulated traffic between the DMVPN peers.
IPsec Protects the Tunnel Traffic
After the tunnel traffic has been constructed, IPsec provides the security layer. Depending on the deployment, the GRE traffic is protected using IPsec security associations and ESP.
This creates an important troubleshooting boundary. A tunnel interface can appear operational while the required IPsec security association is missing or while protected traffic is not being successfully encrypted and decrypted.
Tunnel up does not automatically mean traffic is working. Validate the tunnel, NHRP state, IPsec security associations, routing information and forwarding behaviour independently.
The Underlay Sees the Outer Transport
Once the packet has been encapsulated and protected, the underlay network forwards the resulting transport traffic. The underlay is concerned with reaching the remote DMVPN peer rather than understanding the final application destination inside the overlay.
OUTER SRC 198.51.100.11
OUTER DST 198.51.100.12
PROTOCOL IPsec / ESP
This distinction is extremely useful during troubleshooting. If the underlay cannot reach 198.51.100.12, the overlay cannot carry the packet regardless of whether the routing configuration itself is correct.
Decapsulation at the Remote Peer
At the remote DMVPN router, the process is reversed. The protected transport is received and processed by IPsec. The tunnel encapsulation is then removed, exposing the original IP packet.
The remote router can then perform normal IP forwarding towards 10.40.40.40. The final destination therefore sees the original IP communication rather than the intermediate NBMA transport addresses used by the DMVPN infrastructure.
What Each Layer Actually Knows
Determines how the destination prefix is reached through the DMVPN overlay.
Protects the tunnel traffic and maintains the required security associations.
Provides transport reachability between the physical/NBMA endpoints.
Follow the packet, not just the tunnel. When a DMVPN path fails, determine which layer can no longer carry the packet. A working underlay does not prove NHRP is correct. An established NHRP mapping does not prove IPsec is protecting traffic. An established IPsec SA does not prove the routing table contains the expected destination path.
Evidence to Collect
The data plane can be correlated with the control-plane state using a small set of operational commands:
show ip cef 10.40.40.40
# Tunnel state and counters
show interfaces tunnel 0
# NHRP peer mappings
show ip nhrp
# IPsec protection and packet counters
show crypto ipsec sa
These outputs should be interpreted together. The goal is not simply to prove that individual commands return output, but to establish a continuous forwarding chain from the source host to the destination.
ROUTE — Routing Across the DMVPN Overlay
Compare candidate paths and see how the routing protocol determines which overlay path becomes the forwarding decision.
The routing table contains multiple candidate paths to the same destination. The selected path is the route that will be installed for forwarding.
R1# show ip cef 10.40.40.0
DMVPN provides the overlay connectivity. The routing protocol determines which destination prefix is reachable and which path should be preferred.
FORWARD — Follow the DMVPN Data Plane
Once the overlay is built, the real engineering question is simple: how does an actual packet travel? Follow the original IP packet from the source host through the DMVPN tunnel, across the underlay, and back into the remote network. This separates the inner packet, overlay tunnel, IPsec protection, and underlay transport so failures can be isolated to the correct layer.
FORWARD — DMVPN Data Plane & Packet Flow
Once the overlay is established and routing has selected a path, the next question is simple: what actually happens to the packet? The DMVPN data plane combines the original IP packet, the tunnel overlay, IPsec protection and the underlay transport. Understanding these layers makes it possible to identify exactly where forwarding succeeds or fails.
One packet, multiple forwarding views
The endpoint application creates an ordinary IP packet. The DMVPN tunnel then provides the transport framework around that packet. mGRE identifies the tunnel overlay while IPsec protects the resulting tunnel traffic before it crosses the physical network.
What each layer actually sees
Decapsulation restores the original forwarding context
At the remote DMVPN peer, the protected transport is processed in the reverse direction. IPsec protection is removed, the mGRE encapsulation is removed, and the original IP packet becomes available for normal forwarding toward its destination. This distinction is critical: the physical network forwards the outer transport, while the DMVPN overlay forwards the inner destination prefix.
CEF Decision
Confirm which interface and next-hop the router selected for the destination prefix.
show ip cef 10.40.40.0
Tunnel State
Confirm the tunnel interface is operational and carrying the expected traffic.
show interfaces tunnel 0
IPsec Counters
Confirm encryption and decryption counters are increasing when traffic is generated.
show crypto ipsec sa
If the underlay is reachable but the destination cannot be forwarded, do not treat the problem as a single “DMVPN tunnel failure.” Separate the investigation into the layers: underlay transport → tunnel encapsulation → IPsec protection → overlay routing → forwarding. The point at which the packet stops progressing identifies the most useful next piece of evidence.
FORWARD — Follow the DMVPN Data Plane
Original IP Packet
The source creates the original IP packet. At this point there is no DMVPN tunnel header and the destination is the final host.
show ip cef 10.40.40.40
show interfaces tunnel 0
show crypto ipsec sa
DST 10.40.40.40
→ 198.51.100.12
DMVPN — Phase 3 Explorer
INVESTIGATE — DMVPN Failure Workflow
A DMVPN failure should be investigated as a dependency chain, not as a single tunnel problem. Work from the underlay through mGRE, NHRP, IPsec, routing and finally forwarding to isolate the first broken layer.
INVESTIGATE — DMVPN Failure Workflow
Troubleshooting DMVPN becomes much easier when the investigation follows the dependency chain established throughout this guide. Instead of treating “the tunnel is down” as the diagnosis, identify the first layer that has failed and prove it with operational evidence.
A missing route may be caused by routing policy, an unresolved NHRP peer, an unavailable IPsec security association, or a broken underlay. The correct troubleshooting sequence prevents a downstream symptom from being mistaken for the root cause.
Start With the Dependency Chain
DMVPN is not a single protocol. It is a collection of functions operating together. Each layer depends on the layers beneath it, so investigation should normally move from basic transport reachability toward the final forwarding decision.
Underlay
Confirm that the NBMA transport addresses are reachable before investigating the overlay.
show ip route
mGRE
Verify that Tunnel0 is operational and that the tunnel interface has the expected state and addressing.
show interfaces tunnel 0
NHRP
Check whether logical tunnel peers are correctly registered and mapped to their NBMA addresses.
show ip nhrp
IPsec
Confirm that the required security associations exist and that protected traffic is being encrypted and decrypted.
show crypto ipsec sa
Routing
Determine whether the destination prefix exists and whether the expected path has been selected.
show ip route
Forwarding
Verify that the selected route produces the expected CEF forwarding decision and next hop.
show ip cef
Common Failure Signatures
The same user-visible symptom can originate from different layers. The objective is therefore to correlate the symptom with the control-plane and forwarding evidence underneath it.
The NBMA address of the remote peer cannot be reached. NHRP, IPsec and overlay routing may subsequently appear broken because their transport dependency is unavailable.
The tunnel framework may exist, but the required logical-to-NBMA peer information is absent or incomplete. Dynamic peer communication can therefore fail.
Peer discovery may succeed while protected traffic still fails because the required security association has not been established or is not passing traffic.
The DMVPN overlay can be operational while the routing process selects an unexpected path, rejects a prefix, or fails to install the destination in the forwarding table.
Evidence Before Diagnosis
A strong troubleshooting process does not begin by changing configuration. First establish what the network believes is happening. Then compare that evidence with the expected dependency state.
Peer and Tunnel Evidence
Establish whether the transport, tunnel and NHRP control plane agree about the remote peer.
show ip route 198.51.100.12
ping 198.51.100.12
! DMVPN / NHRP
show dmvpn
show ip nhrp
Security and Routing Evidence
Once peer reachability is established, determine whether protected transport and route installation are functioning.
show crypto ipsec sa
show crypto session
! Routing / forwarding
show ip route 10.40.40.0
show ip cef 10.40.40.0
The Correct Troubleshooting Order
The investigation should move from the dependency that everything else requires toward the final forwarding decision. Skipping directly to routing or application symptoms can produce misleading conclusions.
Engineering Conclusion
A DMVPN investigation should move from reachability → discovery → security → routing → forwarding. Each layer answers a different question. Underlay evidence proves transport reachability. NHRP proves peer discovery and mapping. IPsec proves protected transport. Routing proves the destination path. CEF and forwarding evidence prove what the router will actually do with the packet.
INVESTIGATE — DMVPN Failure Workflow
Inject a controlled DMVPN failure, collect evidence from each dependency layer, and identify the first failed component rather than treating the final symptom as the root cause.
No diagnosis yet
Inject a fault and run the investigation to correlate the evidence.
