Why is IPv6 neighbor solicitation only responding to solicited-node multicast and not unicast?

I have guest A on host A and guest B on host B. Let's say guest A has a global addr of 2600::A and guest B has an address of 2600::B.

When guest A attempts to ping guest B it ends up taking about 10 seconds to complete. After doing a packet capture, this is because guest A tries three times to do an NS for guest B and gets no response. It's only after the fourth attempt that an NS is sent to the solicited-node multicast address for guest B where it gets a response. It looks like this:

attempt 1: src: fe80::A dst: 2600::B (timeout) attempt 2: src: fe80::A dst: 2600::B (timeout) attempt 3: src: fe80::A dst: 2600::B (timeout) attempt 4: src: 2600::A dst: FF02::1::ff00:B (success) 

I thought this may have been firewall related but I've flushed the rules and I'm still seeing this result. Why this may be the case?