IPSEC Tunnel only works if initiated from one side.

I have an issue that I can't really find a solution to.

I have an IPSEC tunnel between an ASA and Fortigate.

ASA side has two hosts, we'll call them 10.200.10.138 and 10.200.10.217

Fortigate side has one subnet in the tunnel, 10.9.10.0/24

The Fortigate side can initiate the tunnel and ping 10.200.10.217. However, the Fortigate side cannot hit 10.200.10.138 UNTIL 10.200.10.138 initiates a connection to the Fortigate side. After that point, everything works fine.

Until 10.200.10.138 initiates a connection to the Fortigate side, I get the following error in the log:

 The decapsulated inner packet doesn't match the negotiated policy in the SA. 

Usually I would suspect an ACL mismatch, but given the fact that the connection only works if initiated one way, what should I look for?