Are public ip addresses in internal networks really bad? Is it a good reason to nuke a network?

So i got a call from a friend yesterday. He needed advice and here is the issue:

1. He is the network admin (new company) and someone used public ip space internally. They dont know who the ip addresses belong to and it does not affect how the network works. They used say : 192.20.x.x
2. The dhcp server available addresses are already exhausted so they are constantly using angry ip to identify dormant machines and delete them. The whole network is not in good shape.

I suggested starting over (slowly migrate everything to a new network), but i wanted to hear what others say about this whole situation.