Saturday, October 24, 2020

Reassembling raw packets then re broadcast

I have a bit of a unique ask. I'm using a home grown packet sniffer in my company to capture all public traffic and feed it to an analytics (AI/ML) system. So I have the Layer 2/3 raw TCP specific packets. This is all HTTP payloads.

The problem is I'm sniffing them off the loopback (used to prevent unencrypted traffic from hitting the wire). The problem is that the lo has an MTU of 65535, and we're using AWS ECS/Fargate, so I can't control the MTU in fargate to make it smaller.

The original packets are currently re-broadcasts on a VMware cluster where the analytics runs, but VMware MTU is max of 9000. Since I'm just broadcasting a raw packet at layer 2 no fragmentation is occurring and since the packet exceeds my MTU it fails.

Below I describe what I think I want to do, but if someone has a better idea (maybe to force this raw packet to be fragmented again, I'm all ears.

So I'm thinking can I reassemble the payloads of each packet into a single large payload for each HTTP session request? I think this is fairly easy. If I use the high port # to identify each unique session then once I have all of the packets, I should be able to drop any empty (payload len=0) then verify the sequence # ordering and concatenate the results into one large block of payload, right? I don't think there's any gotcha's here.

Next I now have a 30k block of memory (size chosen arbitrarily) representing the payload of the original HTTP request or response

Now I need to broadcast over a private VMware network so the analytics sniffers can properly ingest the traffic. I need to verify with the vendor, but I think all they care about is the HTTP headers, cookies and data. If the original TCP/IP/Ethernet headers are gone I don't think it matters. Although my current solution maintains all of that and works on EC2 where I can control the MTU. It just doesn't work where MTU is larger than 9k

Therefore, I'm assuming if I properly broadcast this 30k packet the network will chunk it up into a munch of 9k packets and send it through

So my questions is how to setup the network so I can broadcast this to anything on the network running in promiscuous mode to ingest it? Mind you I'm a generalist, not a deep dive network programmer/engineer. I know enough to be dangerous which usually gets me in over my head, but I'm very resourceful and technical and will eventually come up with some solution.

Thanks all, I really appreciate it



No comments:

Post a Comment