The rfc1323 tunable

The rfc1323 tunable enables the TCP window scaling option.

The TCP window scaling option is a TCP negotiated option, so it must be enabled on both endpoints of the TCP connection to take effect. By default, the TCP window size is limited to 65536 bytes (64 K) but can be set higher if the rfc1323 value is set to 1. If you are setting the tcp_recvspace value to greater than 65536, set the rfc1323 value to 1 on each side of the connection. If you do not set the rfc1323 value on both sides of the connection, the effective value for thetcp_recvspace tunable will be 65536. This option adds 12 more bytes to the TCP protocol header, which deducts from the user payload data, so on small MTU adapters this option might slightly hurt performance.

If you are sending data through adapters that have large MTU sizes (32 K or 64 K for example), TCP streaming performance might not be optimal unless this option is enabled because a single packet will consume the entire TCP window size. Therefore, TCP is unable to stream multiple packets as it will have to wait for a TCP acknowledgment and window update from the receiver for each packet. By enabling the rfc1323 option using the command no -o rfc1323=1, TCP's window size can be set as high as 4 GB. After setting the rfc1323 option to 1, you can increase the tcp_recvspace parameter to something much larger, such as 10 times the size of the MTU.

If the sending and receiving system do not support the rfc1323 option, then reducing the MTU size is one way to enhance streaming performance for large MTU adapters. For example, instead of using a MTU size of 65536, which limits TCP to only one outstanding packet, selecting a smaller MTU size of 16384 allows TCP to have 4 packets outstanding with a tcp_recvspace value of 65536 bytes, which improves performance. However, all nodes on the network need to use the same MTU size.