What could cause a UDP packet to be blocked until a ping is sent?

I have a device that sends UDP packets every 5 secs directly to the IP address of a Windows Server machine (unicast/udp)

Running wireshark on the server show no incoming udp traffic...

Until I ping the device from the server.

first ping always times out, other 3 pings get a reply, then UDP packets start coming in!

Any ideas?

Thanks