I have a Cisco ASA with an IPSEC VPN to AWS. The VPN works and passes traffic but the problem is that it drops every hour for about 4 or 5 minutes. I have used the AWS generated config so all of my phase1/phase2 timers etc match. I wouldn't mind if it dropped for a few seconds but it drops for 4 or 5 minutes which makes it unusable.
Has anybody seen this behaviour before? I have done a debug but I can't see any obvious reasons as to why it's dropping from the debugs.
I have a single network to a single network (as AWS recommend)
Thanks
Below is a snippet from the logs from when it's down to when it comes back up again. There is a limit to how much I can post so can't post all the logs.:
May 05 09:15:06 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Sending keep-alive of type DPD R-U-THERE-ACK (seq number 0x2dcdeb4a)
May 05 09:15:06 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing blank hash payload
May 05 09:15:06 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing qm hash payload
May 05 09:15:06 [IKEv1]IP = 99.16.210.2, IKE_DECODE SENDING Message (msgid=557f54c8) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 84
May 05 09:15:16 [IKEv1]IKE Receiver: Packet received on 221.16.20.114:500 from 99.16.210.2:500
May 05 09:15:16 [IKEv1]IP = 99.16.210.2, IKE_DECODE RECEIVED Message (msgid=523756c0) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 84
May 05 09:15:16 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, processing hash payload
May 05 09:15:16 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, processing notify payload
May 05 09:15:16 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Received keep-alive of type DPD R-U-THERE (seq number 0x2dcdeb4b)
May 05 09:15:16 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Sending keep-alive of type DPD R-U-THERE-ACK (seq number 0x2dcdeb4b)
May 05 09:15:16 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing blank hash payload
May 05 09:15:16 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing qm hash payload
May 05 09:15:16 [IKEv1]IP = 99.16.210.2, IKE_DECODE SENDING Message (msgid=9143ab9f) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 84
May 05 09:15:22 [IKEv1]Group = 99.16.210.2, IP = 99.16.210.2, IKE Initiator: Rekeying Phase 2, Intf Outside, IKE Peer 99.16.210.2 local Proxy Address 0.0.0.0, remote Proxy Address 10.22.0.0, Crypto map (outside_map)
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Oakley begin quick mode
May 05 09:15:22 [IKEv1 DECODE]Group = 99.16.210.2, IP = 99.16.210.2, IKE Initiator starting QM: msg id = cebea1a5
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Active unit starts Phase 2 rekey with remote peer 99.16.210.2.
IPSEC: New embryonic SA created @ 0x7fea3ec8,
SCB: 0x79DB3F20,
Direction: inbound
SPI : 0x1516D12F
Session ID: 0x003A7000
VPIF num : 0x00020002
Tunnel type: l2l
Protocol : esp
Lifetime : 240 seconds
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, IKE got SPI from key engine: SPI = 0x1516d12f
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, oakley constucting quick mode
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing blank hash payload
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing IPSec SA payload
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing IPSec nonce payload
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing pfs ke payload
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing proxy ID
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Transmitting Proxy Id:
Local subnet: 0.0.0.0 mask 0.0.0.0 Protocol 0 Port 0
Remote subnet: 10.22.0.0 Mask 255.255.255.0 Protocol 0 Port 0
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing qm hash payload
May 05 09:15:22 [IKEv1 DECODE]Group = 99.16.210.2, IP = 99.16.210.2, IKE Initiator sending 1st QM pkt: msg id = cebea1a5
May 05 09:15:22 [IKEv1]IP = 99.16.210.2, IKE_DECODE SENDING Message (msgid=cebea1a5) with payloads : HDR + HASH (8) + SA (1) + NONCE (10) + KE (4) + ID (5) + ID (5) + NONE (0) total length : 308
May 05 09:15:22 [IKEv1]IKE Receiver: Packet received on 221.16.20.114:500 from 99.16.210.2:500
May 05 09:15:22 [IKEv1]IP = 99.16.210.2, IKE_DECODE RECEIVED Message (msgid=cebea1a5) with payloads : HDR + HASH (8) + SA (1) + NONCE (10) + KE (4) + ID (5) + ID (5) + NONE (0) total length : 320
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, processing hash payload
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, processing SA payload
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, processing nonce payload
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, processing ke payload
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, processing ISA_KE for PFS in phase 2
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, processing ID payload
May 05 09:15:22 [IKEv1 DECODE]Group = 99.16.210.2, IP = 99.16.210.2, ID_IPV4_ADDR_SUBNET ID received--10.22.0.0--255.255.255.0
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, processing ID payload
May 05 09:15:22 [IKEv1 DECODE]Group = 99.16.210.2, IP = 99.16.210.2, ID_IPV4_ADDR_SUBNET ID received--10.22.0.0--255.255.255.0
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, loading all IPSEC SAs
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Generating Quick Mode Key!
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, NP encrypt rule look up for crypto map outside_map 10 matching ACL VPN-TRAFFIC-INTELYS-AWS: returned cs_id=777bcd40; encrypt_rule=7b3ffc98; tunnelFlow_rule=78106dc8
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Generating Quick Mode Key!
IPSEC: New embryonic SA created @ 0x777ed790,
SCB: 0x84A0D7D0,
Direction: outbound
SPI : 0xC187DB6A
Session ID: 0x003A7000
VPIF num : 0x00020002
Tunnel type: l2l
Protocol : esp
Lifetime : 240 seconds
IPSEC: Completed host OBSA update, SPI 0xC187DB6A
IPSEC: Completed outbound VPN context, SPI 0xC187DB6A
VPN handle: 0x0711bb3c
IPSEC: New outbound encrypt rule, SPI 0xC187DB6A
Src addr: 0.0.0.0
Src mask: 0.0.0.0
Dst addr: 10.22.0.0
Dst mask: 255.255.255.0
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 0
Use protocol: false
SPI: 0x00000000
Use SPI: false
IPSEC: Completed outbound encrypt rule, SPI 0xC187DB6A
Rule ID: 0x7ae31e78
IPSEC: New outbound permit rule, SPI 0xC187DB6A
Src addr: 221.16.20.114
Src mask: 255.255.255.255
Dst addr: 99.16.210.2
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0xC187DB6A
Use SPI: true
IPSEC: Completed outbound permit rule, SPI 0xC187DB6A
Rule ID: 0x7d48e4d8
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, NP encrypt rule look up for crypto map outside_map 10 matching ACL VPN-TRAFFIC-INTELYS-AWS: returned cs_id=777bcd40; encrypt_rule=7b3ffc98; tunnelFlow_rule=78106dc8
May 05 09:15:22 [IKEv1]Group = 99.16.210.2, IP = 99.16.210.2, Security negotiation complete for LAN-to-LAN Group (99.16.210.2) Initiator, Inbound SPI = 0x1516d12f, Outbound SPI = 0xc187db6a
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, oakley constructing final quick mode
May 05 09:15:22 [IKEv1 DECODE]Group = 99.16.210.2, IP = 99.16.210.2, IKE Initiator sending 3rd QM pkt: msg id = cebea1a5
May 05 09:15:22 [IKEv1]IP = 99.16.210.2, IKE_DECODE SENDING Message (msgid=cebea1a5) with payloads : HDR + HASH (8) + NONE (0) total length : 76
IPSEC: New embryonic SA created @ 0x7fea3ec8,
SCB: 0x79DB3F20,
Direction: inbound
SPI : 0x1516D12F
Session ID: 0x003A7000
VPIF num : 0x00020002
Tunnel type: l2l
Protocol : esp
Lifetime : 240 seconds
IPSEC: Completed host IBSA update, SPI 0x1516D12F
IPSEC: Completed inbound VPN context, SPI 0x1516D12F
VPN handle: 0x0711c174
IPSEC: Completed outbound VPN context, SPI 0xC187DB6A
VPN handle: 0x0711bb3c
IPSEC: Completed outbound inner SPD rule, SPI 0xC187DB6A
Rule ID: 0x7ae31e78
IPSEC: Completed outbound outer SPD rule, SPI 0xC187DB6A
Rule ID: 0x7d48e4d8
IPSEC: New inbound tunnel flow rule, SPI 0x1516D12F
Src addr: 10.22.0.0
Src mask: 255.255.255.0
Dst addr: 0.0.0.0
Dst mask: 0.0.0.0
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 0
Use protocol: false
SPI: 0x00000000
Use SPI: false
IPSEC: Completed inbound tunnel flow rule, SPI 0x1516D12F
Rule ID: 0x7908d7f8
IPSEC: New inbound decrypt rule, SPI 0x1516D12F
Src addr: 99.16.210.2
Src mask: 255.255.255.255
Dst addr: 221.16.20.114
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0x1516D12F
Use SPI: true
IPSEC: Completed inbound decrypt rule, SPI 0x1516D12F
Rule ID: 0x7c161328
IPSEC: New inbound permit rule, SPI 0x1516D12F
Src addr: 99.16.210.2
Src mask: 255.255.255.255
Dst addr: 221.16.20.114
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0x1516D12F
Use SPI: true
IPSEC: Completed inbound permit rule, SPI 0x1516D12F
Rule ID: 0x7d3687d0
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, IKE got a KEY_ADD msg for SA: SPI = 0xc187db6a
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Pitcher: received KEY_UPDATE, spi 0x1516d12f
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Starting P2 rekey timer: 3420 seconds.
May 05 09:15:22 [IKEv1]Group = 99.16.210.2, IP = 99.16.210.2, PHASE 2 COMPLETED (msgid=cebea1a5)
IPSEC DEBUG: Inbound SA (SPI 0x1516D12F) sent an ACTIVE PFKey message to IKE (location 1)
May 05 09:15:22 [IKEv1 DEBUG]Pitcher: received KEY_SA_ACTIVE, spi 0x1516d12f
May 05 09:15:22 [IKEv1 DEBUG]KEY_SA_ACTIVE old rekey centry found with new spi 0x1516d12f
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, sending delete/delete with reason message
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing blank hash payload
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing IPSec delete payload
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, constructing qm hash payload
May 05 09:15:22 [IKEv1]IP = 99.16.210.2, IKE_DECODE SENDING Message (msgid=351bc571) with payloads : HDR + HASH (8) + DELETE (12) + NONE (0) total length : 68
May 05 09:15:22 [IKEv1 DEBUG]Group = 99.16.210.2, IP = 99.16.210.2, Active unit activates new SA for remote peer 99.16.210.2.