What's your opinion on connecting to a private IP via VPN on a third-party network?

I've been asked to set up a site-to-site tunnel for this but I've generally regarded it as poor practice, and most third-parties I've dealt with won't allow it anyway. It would a public IP on my end and private on the other. I don't really want to do it but it's not my call.