VRRP master doens't retake master after restarting interface

I have VRRP set up on two Centos servers in a vm, when I turn off the eth0 interface on server 1, server 2 becomes master. But when I turn the server 1 interface back on it doesn't become master even though server 2 becomes the backup.

Edit: Only when I fully restart server 1 does it become master again