NX-OS Multicast Issue

While I wait to get TAC back on to assist on this...

Had an issue where our multicast stopped passing to remote sites (running SD-WAN). Basically AutoRP was intermittently stopping to work after any policy pushes to routers.

Anyways, we ended up just ripping out AutoRP and did it static, not a big deal and we could see group traffic reaching our destinations. I couldn't be arsed at 4am to go through debugging to validate it, so took a nap and got up this morning to find out while it's working...only for some groups.

​

At my source end:

N7K-1 connects to R1 for WAN

N7K-2 connects to R2 for WAN

Any PIM joins coming across the WAN through R1, no problem. Through R2? Doesn't work. R2 has *,G but no S,G. N7K-2 does.

(The RP is on a L3 device cross connected to both N7K and is fully reachable from entire network, no loops or RPF failure, PIM neighbors are up)

​

I've stumbled across some output I'm curious for comments on:

N7K-1# sh ip mroute 239.192.0.50
IP Multicast Routing Table for VRF "default"(*, 239.192.0.50/32), uptime: 12w6d, ip pim igmp
  Incoming interface: Ethernet1/5, RPF nbr: 172.20.10.2
  Outgoing interface list: (count: 4)
    Vlan991, uptime: 00:02:49, pim
    Vlan3103, uptime: 00:33:20, igmp
    Vlan1010, uptime: 07:30:38, pim
    Vlan1024, uptime: 9w1d, igmp

(172.20.22.225/32, 239.192.0.50/32), uptime: 2d17h, ip mrib pim
  Incoming interface: Ethernet1/5, RPF nbr: 172.20.10.2
  Outgoing interface list: (count: 4)
    Vlan991, uptime: 00:02:53, pim
    Vlan3103, uptime: 00:33:20, mrib
    Vlan1010, uptime: 07:30:38, pim
    Vlan1024, uptime: 2d17h, mrib

thats a working channel.  notice the outgoing list under the S,G

​

N7K-2# sh ip mroute 239.192.0.47
IP Multicast Routing Table for VRF "default"(*, 239.192.0.47/32), uptime: 3d22h, igmp ip pim
  Incoming interface: Vlan991, RPF nbr: 172.20.7.17
  Outgoing interface list: (count: 5)
    Vlan1011, uptime: 00:06:03, pim
    Vlan3103, uptime: 00:31:28, igmp
    Vlan3123, uptime: 00:33:03, igmp
    Vlan3143, uptime: 00:33:03, igmp
    Vlan1024, uptime: 21:08:19, igmp

(172.20.22.225/32, 239.192.0.47/32), uptime: 2d16h, ip mrib pim
  Incoming interface: Vlan991, RPF nbr: 172.20.7.17
  Outgoing interface list: (count: 5)

thats it. It never lists anything below it's S,G route in the outgoing list output although the count appears correct. I would expect Vlan1011 to be in the list as that's how it gets to R2..

​

R2 has the *,G and is trying..

R2# sh ip mroute 239.192.0.47

Outgoing interface flags: H - Hardware switched, A - Assert winner, p - PIM Join
 Timers: Uptime/Expires
 Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 239.192.0.47), 00:41:11/stopped, RP 172.20.22.241, flags: SG
  Incoming interface: Te0/0/4.1011, RPF nbr 192.168.181.1
  Outgoing interface list:
    Lspvif0, Forward/Sparse, 00:41:11/stopped

​

Any thoughts on to what could be going on here?