Migrating servers across datacenters keeping their IP address.

Good practice dictates servers should be addresses by their domain name, this would make them immune for IP address changes. Good practice also dictates one should document application’s architecture so one knows which servers communicates between each other. In practice, this can be very different, resulting in a constraint that one needs to keep the servers IP address even during the migration of the datacenter environment from one location to the other. Impossible, difficult? Here’s method on how to get this done. The starting point is that all our servers are in datacenter A, in a server vlan with network range 10.10.10.0/24. There’s the network WAN access router, the core switch and the access switch where the servers are connected. The goal is to move all servers towards datacenter B, where they will be part of the identical network range and will keep their same IP address. Of course, due to operational requirements, availability demand etc. we are talking about virtual server environment, were we have the initial servers active, operational in datacenter A, and an identical virtual server environment instantiated in datacenter B. Server storage is being replicated and kept synchronized across the datacenters via a separate network range. This is different from the 10.10.10.0/24 network range via which the servers are addressed by clients, or use to communicate between themselves. Let us first describe our initial situation. All servers are part of 10.10.10.0/24, connected to the access switches, which itself is connected to the core switch in datacenter A. The server vlan is terminated on the core switch. Egress to the WAN is via the access router, which also advertises the network 10.10.10.0/24 inside the WAN, so all participants know 10.10.10.0/24 servers resides in datacenter A. The mac address of vlan 10.10.10.0/24 on core switch is aaaa.aaaa.aaaa and unique core switch IP address in the vlan is 10.10.10.a/24. Core switch default GW is access router LAN IP in datacenter A (e.g. 10.10.20.254/24). This is unrelated to the server Vlan. The access router in datacenter B does not advertise any route related to 10.10.10.0/24 itself, but knows the range resides in datacenter A. The server vlan, network 10.10.100.0/24 is already created and terminated at the core switch. The core switch IP address in the server vlan is unique and not in use in datacenter A.
The mac address of VLAN 10.10.10.0/24 on core switch is aaaa.bbbb.bbbb and unique core switch IP address in the VLAN is 10.10.10.z/24. Core switch default GW is access router LAN IP in datacenter B (e.g. 10.10.30.254/24). This is unrelated to the server vlan.

To prepare for the migration, the following actions are needed. The first one is to prepare layer 2 at both ends. As server to server communication within the same vlan or network is initiated via an arp broadcast request that never will leave the local vlan, server to server communication would be impossible if one server resides in network range 10.10.10.0/24 in datacenter A and the other server is already moved to network range 10.10.10.0/24 in datacenter B. There would be no arp reply. This is true irrespective of any routing in place. As initially all servers resides in datacenter A, layer 2 on the core switch in datacenter B needs to be prepared as follows: This for each server IP address x active, residing in datacenter A. • ip route 10.10.100.x 255.255.255.255 10.10.30.254 This is a static route to the remote server with the local access router as next hop. As he knows the route to datacenter A. • arp 10.10.10.x aaaa.bbbb.bbbb arpa alias
This is called a proxy arp. The core switch interface in the server vlan will reply with its IP address. And because also a static route is present traffic will be forwarded to the access router and reach the server in datacenter A So if you 20 active servers in datacenter A, you add 20 static routes and 20 proxy arp entries.

Let’s start moving a server from datacenter A to B. At this point in time, our access router in datacenter A advertises the complete 10.10.10.0/24 network inside the WAN.
First, we verify server data/storage is in sync and up to data at both sides. Sever in datacenter A is then shut down. In datacenter B we now execute the following changes, for this server x, and only for this one we not remove the static route and the proxy arp on the core switch. • No ip route 10.10.10.x 255.255.255.255 10.10.30.254 This removes the static route to the remote Server, as the server is now local in datacenter B • No arp 10.10.10.x aaaa.bbbbb.bbbb arpa alias
This removes the proxy arp, as the active server will now reply locally.

In datacenter A, we now execute the following changes, for this server x, and only for this one we add the static route and the proxy arp on the core switch. • Clear ip arp 10.10.10.x
Housekeeping • ip route 10.10.10.x 255.255.255.255 10.10.20.254 This adds the static route to the remote Server, as the server is now local in datacenter B • No arp 10.10.10.x aaaa.aaaa.aaaa arpa alias
This adds the proxy arp and the core switch vlan interface will reply to the arp request.

As last part of the migration, we now start advertising a more specific route 10.10.10.x/24, x the IP address of the moved server from our access router in datacenter B. That’s it, one server moved. With all servers moved, one can do the cleanup, and remove the 10.10.10.0/24 route advertisement from datacenter A access router, and replace all more specific route advertisements from datacenter B access router with the full range advertisement, as well as the specific routes and proxy arps that have been added to the core switch in datacenter A.