Tuesday, July 9, 2019

TCP RTOs, Spurious Retransmissions, and unable to access cloud hosted application.

Ok, so I'm currently having an issue that we can't seem to solve. We have a application hosted on a 3rd party's server, which we access over the internet. This application is accessed by time clocks that automatically transmit data to the server, and by users over https via a web browser. Currently, no one LAN side can access the the web page, but time clocks have no issue sending data to the server. Both the web service and the time clocks talk to the same IP, via https. Despite numerous troubleshooting calls with the time clock vendor, our ISPs, and Palo Alto, (our edge firewalls), we have been unable to pinpoint where the issue lies.

Pcaps sourcing from the client and post-nat WAN show what you see in the title. We have had the vendor remove IP restrictions on their edge firewall. Outside our network, we are able to access the URL that redirects us to the application's web page. We have even had our ISP NAT to one of our public IP addresses, and they accessed the web page with no issue, bypassing our edge firewall. Really stuck on how to find out where the issue lies.

Everything was working until yesterday around noon. We had made no changes, but the vendor did make changes to IP restrictions. Apparently they have removed them since we have started troubleshooting.

Things we've tried so far: - Fail over to backup circuit. - Fail over firewalls - Verified, with Palo techs that firewalls are not blocking traffic. - Tested on machines that surpass their minimum requirements. - Verified routing. - Sent multiple Pcaps to vendor's network team.

Thanks for any advice, and I'm sure I'm leaving out important info, so please feel free to ask for clarification!



No comments:

Post a Comment