IBM Support

IBM Security Guardium - Sniffer crashing with segfault errors

Troubleshooting


Problem

You have already applied the latest sniffer patch available on fix central and can see sniffer crashing with segfault errors under log files captured via "support must_gather sniffer_issues"

Symptom

Collector is having issues capturing the traffic and you notice there are sniffer crash errors in the appliance syslog file ( /var/log/messages).

You noticed continuous messages with segfaults in the syslog file:

Oct  4 11:57:04 guardium-col08 kernel: gdmAnalyzerMgrT[4108]: segfault at 98995088 ip 00000000038eee7a sp 00007f2aa6a89760 error 4 in snif[400000+5452000]
Oct  4 11:57:04 guardium-col0 init: guard-snif main process (4023) killed by SEGV signal
Oct  4 11:57:04 guardium-col0 init: guard-snif main process ended, respawning
Oct  4 11:57:05 guardium-col0 snif: Guardium Sniffer Started
Oct  4 11:57:07 guardium-col0 GuardiumSniffer[11842]: Guardium Sniffer license verified.
Oct  4 11:57:07 guardium-col0 GuardiumSniffer[11842]: Starting WTAP_SERVER

Diagnosing The Problem

1. Make sure you have installed the latest sniffer patch for your appliance version from IBM Fix Central.
2. Run following command as user CLI on affected Collector:
support must_gather sniffer_issues

3. Unzip the resultant must_gather file in your desktop and open "messages" file in a notepad.
4. Verify if you can see sniffer restarts on a regular basis with "segfault" errors.

Resolving The Problem

1. If the latest sniffer patch is not installed, please install the latest sniffer patch for your appliance version from IBM Fix Central

2. If the latest sniffer patch is already installed, contact IBM Guardium Support team, providing following information while opening a new PMR :

i) sniffer core dumps :

support store snif-debug on <period>
where period is NUMBER[SUFFIX] and Optional SUFFIX may be 's' for seconds, 'm' for minutes, 'h' for hours (default) or 'd' for days.

You need to pass time duration to run sniffer in debug mode during which it will collect core dumps for any crashes that occur.

For example, to run for 15 minutes:

support store snif-debug on 15m

If the crashes are happening quite often, you should try to start with a low amount of time, and if nothing is captured, increase the time duration until you capture core dumps.

If crash dumps are captured (if crashes occur), two files are created which can be accessed through fileserver:

  • snif-debug_<DATE_TIME>_output.log
  • coredumps_<DATE>_<TIME>.tar.gz


To check the status of the snif-debug, run the CLI command:

support show snif-debug


Note: the core files will only be zipped once the snif-debug has been stopped or finishes running. If you see that the core has been dumped before the snif-debug has finished running you can stop the snif-debug with the CLI command:

support store snif-debug off

ii) A new copy of support must_gather sniffer_issues output after the core dumps have been generated

[{"Product":{"code":"SSMPHH","label":"IBM Security Guardium"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Guardium Database Activity Monitor","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"},{"code":"PF035","label":"z\/OS"}],"Version":"10.0;10.0.1;10.1;10.1.2;9.0;9.1;9.5","Edition":"All Editions","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Document Information

Modified date:
28 September 2018

UID

swg21993153