On which end of an IPSEC site2site tunnel would you create firewall rules?

This is by no means an enterprise setting, but we have two pfSense firewalls running a perpetual IPSEC tunnel between office and a remote location (datacenter).

Right now all traffic can pass between the IPSEC interfaces on both ends, going either way (not secure). I believe it's a best practice to limit that traffic within reason for security reasons.

Would it make a difference as to which end the firewall rules exist? Let's say I want to allow traffic only for backup replication, should I create matching rules on both ends (with reversed source/destination so it works)?

Thanks in advance!