Use these CLI command to access troubleshooting and maintenance utilities through diag.
Use the diag command as directed by Technical Support.
There are no functions that you would perform with this command on a regular basis. Each main menu entry is described in a separate topic (see Main Menu Commands).
To use the diag command, follow the procedure outlined:
The Guardium user attempting to use the diag command must have an assigned CLI or admin role. The only user who has a CLI role by default is admin. The user with a CLI or admin role is permitted to enter the diag command, use the unlock admin and unlock accessmgr CLI commands, and use the export audit-data CLI command without restrictions. The user with a CLI role does not have to enter user name and password required of a GUI login and does not go through any further role check.
If the Guardium user attempting to use CLI does not have a CLI or admin role, CLI will not start. The accessmgr assigns CLI and admin roles.
This output is accessed through the fileserver CLI command. See fileserver for further information.
Each directory is described in the following subsections.
Most output from the diag commands is written in text format to the current directory. For most commands, this directory contains a separate output file. Each time you run the same command, output is appended to the single file for that command. For a smaller number of commands, a separate file is created for each execution, usually incorporating a date and time stamp in the filename.
We recommend that you “clean up” after each session, so in subsequent sessions you are not looking at old information. When you pack files to a single compressed file for exporting (see the following topic), all files in the current directory are deleted. Alternatively, you can use the Delete recordings command of the Output Management menu to delete individual files.
The files in the current directory are easy to identify since the names are created from menu and command names. For example, after you use the File Summary command from the System Interactive Queries menu, a file named interactive_filesummary.txt is created in the current directory.
If you look at the current directory while in the process of using a command, you may see a hidden temporary file with the same name as the one that will contain the output for that command. The temporary file will be removed when the output is appended to the command output file.
When you pack the diag output files in the current directory to a compressed file (to send to Guardium Technical Support, for example), it is stored in the depot directory. The filename is in the format diag_session_<dd_mm_hhmm>.tgz, where the variable portion of the name indicates when the file was created. For example, a file created at 12:15 PM on May 20th would be named as follows: diag_session_20_5_1215.tgz.
After exporting files (see the Export recorded files topic), you can remove them from the depot directory using the Delete recordings command of the Output Management menu.
The Output Management commands control what is done with the output produced by the diag command. Each Output Management command is described separately.
Use this command to pack all diagnostic files in the current directory into a single compressed file, and remove those files from the current directory. When you enter this command, there is no feedback to indicate that the command has completed. You can verify that the command has finished by displaying the directory of the depot directory. When the command completes, there is a file named in the following format: diag_session_<mm_dd_hhmm>.tgz, where the variable portion of the name is a date and time stamp, as described previously. Use the Export recorded files command of the Output Management menu to send the file to another system.
Use this command to delete files in the depot or current directory. (To delete only the current session files, use the Delete current session files command.) When you enter this command, the depot directory structure displays:
You can navigate the directories using the Up and Down arrow keys and pressing Enter. For example, selecting ../ and pressing Enter moves the selection up one level in the directory structure.
You could then select the current directory and press enter, to navigate down to that folder and delete individual command output files. Note that you can navigate to other directories, but you cannot delete files except from the current and depot directories.
When you have selected the file you want to delete, press Enter.
Caution: You will not be prompted to confirm the delete action
Use this command to send a file from the depot directory to another site. To export a file:
Use this command to delete files created during the current session.
Use the Exit command to return to the main menu.
Use the System Static Reports command of the Main Menu to produce an extensive set of reports.
Use the Up and Down arrow keys to scroll up or down in the report. When you are done viewing the report, press Enter to return to the Main Menu.
For an outline of the information contained in this report.
The following subtopics provide an outline of the major components of the System Static Reports output. The fragments of output shown are intended to illustrate the type and level of information contained in the report, rather than provide a detailed description of the actual contents (that is beyond the scope of this document).
The System Static Reports output describes the build version, the patches applied, the current system up time, and name server information:
Build version: 34e1eb12eb68ba76cb49028251c9a0d6 /opt/IBM/guardium/etc/cvstag
Patches:
2009/02/22 16:16:50: START Installation of 'Update 5.0'
2009/02/22 16:18:04: Installation Done - Successfully Installed
< lines deleted… >
Current uptime:
09:03:43 up 6 days, 17:34, 1 user, load average: 0.44, 0.50, 0.41
System nameservers:
192.168.3.20
DB nameservers:
192.168.3.20
Gateway: 192.168.3.1 (system) 192.168.3.1 (def)Next, the file system information displays (shown partially):
Filesystem Size Used Avail Use% Mounted on
/dev/hdc3 2.0G 1.1G 813M 58% /
/dev/hdc1 97M 9.2M 83M 10% /boot
none 504M 0 504M 0% /dev/shm
/dev/hdc2 71G 1.2G 66G 2% /var
total: used: free: shared: buffers: cached:
Mem: 1055199232 1041711104 13488128 0 63275008 186220544
Swap: 536698880 295432192 241266688
MemTotal: 1030468 kB
MemFree: 13172 kB
< lines deleted… >This is followed by information about the mail and SNMP servers configured:
SMTP server: 192.168.1.7 on port 25 : REACHABLE
SMTP user: undef
SMTP password: undef
SMTP auth: NONE
SNMP trapsink: undef UNREACHABLE
SNMP trap community: undef
SNMP read community: undefThe final section of the system configuration section describes the network configuration for the unit: IP address, host and domain names, etc:
eth0: 192.168.3.101 (system) 192.168.3.101 (def)
hostname: (system) g1 (def)
domain: (system) guardium.com (def)
mac address: 00:04:23:A7:77:F2 (MAC1) 00:04:23:A7:77:F2 (MAC2)
unit type: 548 Standalone STAPThe next major section of the System Static Reports output contains information about the internal database status and threads (only the first few threads are shown):
uptime 77097 seconds.
27 threads.
78545028 queries.
+------+------------+-----------------------------+---------+---------+------+-----------
| Id | User | Host | db | Command | Time | State | +---------------------------------------------------------------------------------------
| 1137 | enchantedg | localhost | TURBINE | Sleep | 26 | | 1257 | enchantedg | localhost.localdomain:33587 | TURBINE | Sleep | 0 | | 1258 | enchantedg | localhost.localdomain:60409 | TURBINE | Sleep | 7716 | | 1259 | enchantedg | localhost.localdomain:48233 | TURBINE | Sleep | 322 |
< lines deleted… >The list of threads is followed by an analysis of table status.
The next several sections of the System Static Reports output contain information about the Web servlet container environment (Tomcat):
============================================================================
Currently defined Tomcat port is 8443.
The TOMCAT daemon is running and listening on port(s): 8005 8443.
Currently OPEN ports
java run by tomcat on port *:8443
< lines deleted… >
============================================================================
These are the nanny latest actions:
May 19 14:13:09 guard nanny:[5528]: Also checking tomcat.
May 19 14:13:09 guard nanny:[5528]: Going for my initial nap.
< lines deleted… >
This is the TOMCAT command line:
463 sh -c ps -o pid,cmd -e | grep Dcatalina.base
21917 grep Dcatalina.base.The next major section contains information about the IP tables:
===========================================================================
IPTABLES:
-------------
tcp -- 192.168.2.0/24 192.168.1.0/24 tcp spts:1521:60000 set 0x23
tcp -- 192.168.1.0/24 192.168.2.0/24 tcp dpts:1521:60000 set 0x22
< lines deleted… >The next major section contains IP traffic information:
IP traffic statistics.
OUTPUT OF ETH0
Fri May 20 11:57:04 2012; ******** Detailed interface statistics started ********
*** Detailed statistics for interface eth0, generated Fri May 20 11:58:04 2009
< lines deleted… >
OUTPUT OF ETH1
Fri May 20 11:57:04 2012; ******** Detailed interface statistics started ********
*** Detailed statistics for interface eth1, generated Fri May 20 11:58:04 2009
Total: 82440 packets, 53892382 bytes
(incoming: 82440 packets, 53892382 bytes; outgoing: 0 packets, 0 bytes)
IP: 82440 packets, 52632747 bytes
(incoming: 82440 packets, 52632747 bytes; outgoing: 0 packets, 0 bytes)
< lines deleted… >The next section contains the last messages output by the sniffer:
Snif STDERR:
< lines deleted… >
Snif STDOUT:
Fri_20-May-2009_04:04:35 : Guardium Engine Monitor starting
Fri_20-May-2009_04:14:37 : Guardium Engine Monitor starting
Fri_20-May-2009_04:24:38 : Guardium Engine Monitor starting
< lines deleted… >The next section lists the import directory contents:
These are the contents of the importdir directory:
total 0This section lists the following summary information (see example):
============================================================================
Range of time in logs: 01/14/10 13:12:26.348 - 01/18/10 12:48:01.073
Selected time for report: 01/14/10 13:12:26 - 01/18/10 12:48:01.073
Number of changes in configuration: 4 - changes to the audit configuration
Number of changes to accounts, groups, or roles: 0
Number of logins: 22 - logins into the machine - ssh and console
Number of failed logins: 114
Number of authentications: 22 - "su", etc.
Number of failed authentications: 5
Number of users: 2
Number of terminals: 18
Number of host names: 9
Number of executables: 7
Number of files: 0
Number of AVC's: 0
Number of MAC events: 0
Number of failed syscalls: 0
Number of anomaly events: 3
Number of responses to anomaly events: 0
Number of crypto events: 0
Number of keys: 0
Number of process IDs: 9173
Number of events: 98669
============================================================================This section lists the following (see example):
============================================================================
# Date Time Type Exe Term Host AUID Event
============================================================================
1. 01/14/10 13:16:02 ANOM_PROMISCUOUS /usr/sbin/brctl (none) ? -1 8 - this is expected
to appear - it means the bridge is listening to all trafficThis section lists the following (see example):
============================================================================
# Date Time Type Exe Term Host AUID Event
============================================================================
1. 01/14/10 13:13:22 tomcat ? console /bin/su yes 4
2. 01/14/10 13:16:44 tomcat ? console /bin/su yes 11
3. 01/14/10 13:16:44 tomcat ? console /bin/su yes 17
4. 01/14/10 13:16:45 tomcat ? console /bin/su yes 23
5. 01/14/10 13:16:48 tomcat ? console /bin/su yes 29
6. 01/14/10 13:22:29 tomcat ? ? /bin/su yes 155
7. 01/14/10 13:28:10 ? ? tty1 /bin/login no 252
8. 01/14/10 13:28:20 ? ? tty1 /bin/login no 254This section lists the following (see example):
============================================================================
# Date Time Type Exe Term Host AUID Event
============================================================================
1. 01/14/10 13:22:15 root 192.168.2.9 sshd /usr/sbin/sshd no 142
2. 01/14/10 13:22:15 root 192.168.2.9 sshd /usr/sbin/sshd no 143
3. 01/14/10 13:22:17 root 192.168.2.9 sshd /usr/sbin/sshd no 144
4. 01/14/10 13:22:17 root 192.168.2.9 sshd /usr/sbin/sshd no 145
5. 01/14/10 13:22:20 root 192.168.2.9 sshd /usr/sbin/sshd no 146Select System Interactive Queries from the main menu to open the Interactive Queries menu. (Use the Down arrow key to scroll past the tenth item to see all items on this menu.)
In addition to displaying the requested information, each interactive query command creates output in a separate text file in the current directory. See the Overview topic for more information about the files created.
Each command is described in the following sections.
Use the Files Changed command to display a list of files changed either before or after a specified number of days.
Use this command to list the contents of various directories.
Use the Summarize Folder command to display the output of the du (Disk Usage) command:
Use this command to list all or some portion of a log file.
Be aware that when the Summary Style is used, variables are replaced by the pound sign character (#). For some log data containing variables such as IP addresses or dates, the replacements can be extensive.
Use this command to send a test email using the configured SMTP server.
Use this command to send a test SNMP trap to the configured SNMP server.
Use this command to display the actual select statement used for a report query. This might be useful if a user-written report is producing unexpected output.
User written reports are listed following the pre-defined reports, beginning with number 20001 (for version 3.6.1).
The selected report select statement will be displayed.
Use this command to display a count of observed SQL calls during a 100 second interval.
Use this command to create a TCP dump. For this command, output is written to a command file only and not to the screen. Unlike most other commands, a separate file is created in the current directory for each execution of this command. The file name is in the format: tcpdump_<mmyyyy-hhmmss>, where the variable portion is a date and time stamp: mmyyyy is the month and year, and hhmmss is the hours, minutes, and seconds.
Use this command to display a TCP dump file created previously.
Use this command to watch activity in the Guardium buffers:
Use this command to run the slon utility, which tracks packets. Typically, you would only run this command as directed by Technical Support. For this command, output is not written to the screen. Output is written to one of two command files in the current directory, for each execution of the command: apks.txt.<day_dd-mmm-yyyy_hh.mm.ss.ttt> OR requests.txt.<day_dd-mmm-yyyy_hh.mm.ss.ttt>
The variable portions or the file names are date and time stamps. For example, apks.txt.Fri_20-May-2011_08.52.00.789.
(a) to dump Analyzer rules info
(f) to filter Analyzer packets based on IP and/or mask
(p) to dump packets to apks.txt
(l) to dump logger requests to requests.txt
(m) to dump STAP packets (Select how long to run. Wait for completion and then check the msg-dump file under /var/log/guard/diag/current/tap/ )
(r) to record IPQ traffic
(s) to dump State machine info
(t) to configure throttle parameters
Use this command to show indexes for various internal tables:
Use this command to display interface link status.
Use this command to display throttle data.
Use this command to create a TCP dump and run the slon utility, which tracks packets. Typically, you would only run this command as directed by Technical Support. See the individual topics, Generate TCP dump, and Slon Utility.
Use this command to create a SSL dump..
Use this command to display bash history.
Use this command to create GDM_ERROR dumps.
When Tomcat has a first outOfMemory error, it will do a memory dump to /var/tmp/tomcat/tomcat.dmp. Use this command to compress, encrypt and move this file to /var/log/guard/diag/tomcat/ for fileserv to retrieve.
Click on Extended Network Information option under System interactive query to display the network diagnostics information.
Example
SQLGuard Diagnostics
Network Parameters from ADMINCONSOLE_PARAMETER:
SYSTEM_NETMASK1: 255.255.255.0
SYSTEM_DOMAIN:
SYSTEM_DEFAULT_ROUTE:
SYSTEM_DNS1:
SYSTEM_DNS2:
SYSTEM_DNS3:
TOMCAT_IP:
MANAGER_IP:
HOST_MAC_ADDRESS:
SECOND_DEVICE:
This selection is different from other diag selections in the section called Generate TCP and Generate TCP and slon.
For Generate TCP dump in rotation, enter Filter IP address (enter blank for all IPs). Enter Filter Port number. For the question, How long to run? if the TCP dump in rotation is already running, choose the option “Rotation OFF” or “Rotation” (ON). If Rotation is selected, add file size.
The TCP dump will be output to /var/log/guard/tcp.bin1 and /var/log/guard.bin2 in rotation.
Select TCP dump in rotation again to stop the process loop_tcpdump.sh.
Select the Perform Maintenance Actions option from the Main Menu to open the Maintenance menu. Use these commands only under the direction of Technical Support. These do not need to be run on a regular basis.
Use this command to optimize index cardinality on Guardium’s internal database. A progress bar displays while the operation is running. When the operation completes, you are returned to the Maintenance menu.
Use this command to analyze and re-index Guardium’s internal database.
Use this command to clean unused disk space. You are returned to the Maintenance menu when the procedure completes.
Use this command only under the direction of Technical Support. This command provides access to the Management Menu of the RAID controller utility program, which can be used to display the status of the RAID drives. If your system does not have a RAID controller, an error message displays if you select this command. You must be extremely careful when using the RAID controller utility program, since several of the functions provided will erase all information on the disk.
Use this command to turn debugging on or off. You are prompted to enable or disable logging, or to reset the system defaults.
Use this option to change the timeout limit for long queries.
Use this option only when directed to do so by Technical Support.
Use this command to restore a backed up version of the internal database. You will be prompted to confirm the operation.
Use this command to select the component debug level. Choose one of the following options:
Classifier, Data Level Security, Workflow, or Other.
Choose Classifier to select debug level options: ERROR, WARN, INFO, DEBUG, ALL.
Choose DLS (data level security), Workflow, or Other (text input) to select debug level options: ERROR, WARN, INFO, DEBUG, ALL.
If Other is chosen (text input separated by ',') , enter valid components (dls, workflow, audit, customtable, gui, other, job).
This option should be used only by Technical Support and only in those cases where static tables grow too much and needed to be cleaned. This utility cleans all the old construct records that don’t have any Instances associated with them. A progress message will display during the Clean Static Orphans.
Select Exit to CLI on the Main Menu. Press Enter to close the diag command and return to the command line interface.