Troubleshooting
Problem
TCP Off-loading is not supported in the IBM FileNet P8 4.x and 5.x environments.
Symptom
End users may experience long delays during query operations as well as content access. From taking a network capture, you can identify the behavior of TCP-Offloading as seen in figure 1.3.
Fig. 1.3
In frames 129 thru 134 we have the client at IP address 10.0.0.219 sending the SYN request followed by the Servers SYN ACK response. Frame 131 shows the Checksum being incorrect as well as the tool indicating that this “maybe caused by TCP checksum offloading”. In this case the client was not able to continue with any further communication on this connection. The next frame we see the Client station starting another connection only to repeat with a TCP Checksum error. On the client station, there were different error messages (failure to connect, page not found, the clients browser wind would just hang, etc...) If there are signs of ambiguity, please check the TCP setting for that interface to be sure it has been disabled.
Cause
The major issues that have caused problems for many years are the inefficient use of the host CPU—in some cases 90% of the available processing power can be lost when handling TCP due to context switching and interrupt overhead, cache thrashing and inefficient data handling. These issues also lead to limited throughput, extremely long latencies for applications and have become increasingly problematic as the rest of the equipment in the data center has scaled by orders of magnitude in performance.
Figure 1 – Conventional Ethernet architecture
Fig.1.1
TCP Offload (see Figure 2) was the first alternative to Ethernet's shortcomings that was explored by the industry. Unfortunately it has fallen short as a solution.
Figure 2 – TCP Offload architecture
Fig. 1.2
The premise of TCP Offload is that a coprocessor (a TOE, or TCP Offload Engine) can execute TCP processing faster than the host CPU. However, while this approach solves the CPU loading issue for large data transfers, it does not solve the other critical issue of latency. Additionally, TCP Offload suffers the classic coprocessor problem of not being able to scale up in performance and down in cost through time as quickly as the system processor.
Microsoft’s version
TCP Chimney Offload provides automated, stateful offload of Transmission Control Protocol (TCP) traffic processing to a specialized network adapter implementing a TCP Offload Engine (TOE). For long lived connections with large-sized packet payloads, like those associated with file server, backup and storage workloads and other content-heavy applications, TCP Chimney Offload greatly reduces CPU overhead by delegating network packet processing tasks, including packet segmentation and reassembly to the network adapter. By using TCP Chimney Offload, you can free up CPU cycles for other application tasks, such as supporting more users sessions or processing application requests with lower latency.
Where TCP Offloading Engine is seen to cause its most impact in the IBM FileNet P8 environment is by buffering content that should be readily return from a request. As the layer 4 PDU's ( Protocol Data Unit) are processed at the network interface card itself, buffer space becomes an element of delayed response
Resolving The Problem
Microsoft Windows 2003 SP2 (How to disable TCP Chimney offloading engine):
Here are the instructions from the Microsoft Web site.
|
This link (included as a reference) refers to connection related issues you may encounter when TCP Chimney Offload is enabled. Be advised that we also want to make sure that TOE is disabled on all hardware NICs, as well. The article refers to a way of patching the Broadcom card to help alleviate the issues, and while that addresses a specific scenario with SQL server and replication, it will not address issues encountered with FileNet software. The bottom line is that the nature of our software is one of a high number of transactions. TOE and/or TCP Chimney Offload is not appropriate in this context.
For further information see Microsoft website for KB942861.
Windows 2008 requires additional configuration here:
http://technet.microsoft.com/en-us/library/gg162709%28WS.10%29.aspx
Hardware NIC's and TOE:
This has been seen on Dell computers with Broadcom NIC Cards -- to disable TOE, launch the Broadcom Advanced Control Suite 2, Select the Advanced tab, Select the "checksum offload" property and then select the Value of None.
It has also been seen on HP Blade servers running HP NC7782 Gigabit Adapter (original manufacture Broadcom)
--To disable TOE on the HP NIC card --
Go to Network Properties / select the interface that is enabled /and select HP Network Configuration Utility - -then select the adapter and go to the Advanced tab, Select the Checksum Offloading and select the value of "None".
Command Line for TOE.
To disable TOE offload:
netsh int ip set chimney disable
To re-enable TOE offload:
netsh int ip set chimney enable
To check TOE:
Operating systems that support TCP Offloading Engine (TOE) have added an additional command line switch to the existing netstat command, –t. This switch returns information regarding the connections TOE is handling.
To make the information meaningful and easy to read, you should also include –o and –n switches. The complete command is as follows:
netstat –ont
The following image shows the command running on a system that does not have TOE enabled but does have TOE support.
The following image shows TOE enabled with one connections currently being offloaded.
Related Information
Was this topic helpful?
Document Information
Modified date:
17 June 2018
UID
swg21578088