Investigating MTU/MSS Issues
I have seen an increase in issues that go back to IP MTU or TCP MSS issues.
Its very important to figure out which one you are dealing with so you can troubleshoot it efficiently.
Both TCP MSS and IP MTU refer to the amount of payload that layer can carry.
As a general rule of thumb is that IP MTU is a layer 3 concept, therefore any layer 3 device would affect the MTU like a router. TCP MSS is a layer 4 concept and affected by NAT, proxies, etc..
When using Wireshark, you will see ICMP error messages stating “Destination Unreachable (Fragmentation needed)”. Within the ICMP packet you will see the “MTU of next hop” that is allowed. MTU issues are tricky to troubleshoot since the symptoms range from timeouts, drops or poor performance. Also pay attention to your network firewalls and ensure that ICMP error packets are allowed back to the client.
When it comes to TCP MSS, you will see the MSS values in the TCP SYN packets. Super small TCP MSS values such as anything less than 1,000 bytes will cause performance-related issues.