Flash (Alert)
Abstract
When a Real-time Compression Appliance running V3.7.1 or V3.8.0.01 receives NFS/UDP traffic with destination IP address of a storage system which is not configured in the appliance Servers page (in V3.7.1) or Storage page (in V3.8.0.01), the appliance kernel may crash causing the appliance to reboot.
Content
Starting from V3.7.1 the appliance is using internal firewall rules to block traffic that is not directed to storage system it is serving. A problem was found in the kernel used in V3.7.1 and V3.8.0.01 which is causing it to crash when it receives NFS/UDP traffic with destination IP address which is not part of the configured storage systems.
Symptoms
Appliance reboots unexpectedly. The following messages are displayed in the serial console and VGA display:
[ 935.386357] general protection fault: 0000 [1] SMP
[ 935.391006] last sysfs file: /sys/class/net/br1.196/bridge/topology_change_detected
[ 935.391006] CPU 2
[ 935.391006] Modules linked in: i2c_core(N) e1000e(N) nf_defrag_ipv4(N) coretemp(N) ipmi_watchdog(N) hwmon_vid(N) f75375s(N) nfnetlink(N) nf_conntrack_netlink(N) ipt_REDIRECT(N) i2c_i801(N) nf_conntrack(N) nf_conntrack_ipv4(N) nf_nat(N) iptable_nat(N) bonding(N) bnx2(N) cxgb3_1_4_0_8(N) [last unloaded: e1000e]
[ 935.391006] Supported: No
[ 935.391006] Pid: 0, comm: swapper Tainted: G 2.6.27.19-147-64bit-stw #1
[ 935.391006] RIP: 0010:
[ 935.391006] RSP: 0018:ffff88127f8f78f8 EFLAGS: 00010282
[ 935.391006] RAX: ffff88127f8f1fd8 RBX: 3031313320312020 RCX: ffff88127f8f7680
[ 935.391006] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff805b8071
[ 935.391006] RBP: 2031202020203139 R08: 0000000067617473 R09: 0000000000000008
[ 935.391006] R10: 0000000000000000 R11: 0000000000000000 R12: 2020202030203020
[ 935.391006] R13: 2520313930202020 R14: 0033312084312020 R15: d000000084000000
[ 935.391006] FS: 0000000000000000(0000) GS:ffff88127f870640(0000) knlGS:0000000000000000
[ 935.391006] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
[ 935.391006] CR2: 00007f82facd4180 CR3: 0000000000201000 CR4: 00000000000006e0
[ 935.391006] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 935.391006] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 935.391006] Process swapper (pid: 0, threadinfo ffff88127f8f0000, task ffff88127f8e8c30)
[ 935.391006] Stack: 002a020000000000 0610ac0600000000 ac0b43dd9b000064 00ff88127f1b4aca
[ 935.391006] ffffffff8022c7ea 0000000300000000 00000000000000a0 ffff88126d71b718
[ 935.391006] ffff88126d4d2000 ffff88126d71b788 ffff88127bdc7ac0 ffff88126d4d2000
[ 935.391006] Call Trace:
[ 935.391006] <IRQ>
[ 935.391006]
[ 935.391006]
[ 935.391006]
[ 935.391006]
[ 935.391006]
[ 935.391006]
[ 935.391006]
[ 935.391006]
[ 935.391006]
[ 935.391006]
[ 935.391006]
Solution
A solution to this kernel problem is available in V3.8.0.02. Upgrade to V3.8.0.02 or higher.
Workaround
1. Verify that the appliance does not receive traffic not intended to the storage systems behind it.
2. Disable the blocking rules by running:
option
option Z_OPTIONS:UseLegacyIPTableRules 1
engine restart
IBM does not recommend to apply the workaround unless the problem exists in the environment.
Rate this page:
Copyright and trademark information
IBM, the IBM logo and ibm.com are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml.