Troubleshooting: Agents

This section provides a table that shows problems that might occur with your agents:

Table 48. Problems and solutions for agents
Problem Solution

Program fails

No data for a script provider or a command return code is unusable. You will see the following in the trace log:

(46C44462.0000-184C:commandwithtimeout.cpp,278,

"CommandWithTimeout::threadMain")
*Error: Failure in call to CreateProcess()
for script script1 not.bat
Error The system cannot find the file specified.

This shows that it tried to execute the command 'script1' with the argument 'not.bat'.

If you call a program with spaces in the name, use quotation marks around the name so that it is not parsed by the command interpreter. For example, this is a test.bat argument becomes:

"this is a test.bat" argument

Negative or wrong attribute value

I created an agent that includes a number that should be a large positive number, but I see a negative number or a number that is completely wrong.

IBM® Tivoli Monitoring 6.2 uses 32-bit signed integers to represent numeric values. A 32-bit signed integer can display values from -2,147,483,648 to 2,147,483,647. The value you are trying to display has overflowed the 32-bit signed number. In many cases, the values are traced to the log file and replaced with enumerations indicating that the value exceeds maximum or minimum. Overflows can usually be handled by creating another attribute that scales the large one to a more reasonable value.

For example, if the number represents the size of the disk in bytes, it is more useful to use megabytes or even gigabytes. To do this:

  1. Select the Data Sources tab in the Agent Builder.
  2. Right-click the data source.
  3. Select New Derived Attribute...
  4. Choose a new name. For example, for Size, you can add units like Size_MB.
  5. Add a description.
  6. Select the appropriate data type. For example, Counter is correct for a total size.
  7. Create the formula.
    • Select the attribute. For example, Size.
    • Convert the value appropriately. For instance, "Size/1048576" converts from bytes to megabytes

You can now either leave the original attribute or hide it by selecting the attribute and deselecting Display attribute in the Tivoli Enterprise Portal. Hide the original attribute if it is likely that it will overflow a 32-bit signed integer.

IBM Tivoli Monitoring V6.2.1 introduces 64-bit numeric attributes. Changing a 32-bit numeric attribute that is overflowing to a 64-bit value is a natural way to represent large numeric values.

JMX Notifications

I do not have the ability to create a data source to receive notifications.

When using the browser, take note of the MBean information displayed at the bottom of the browser panel for the MBean you are working with. If the MBean contains items in the Attributes tab, then you will get an attribute group containing those metrics. If the MBean contains items in the Notifications tab, you will get an event attribute group containing a standard set of metrics for a Notification object. If both tabs contain values, then you get attribute groups for each.

Without there being notifications defined for the MBean, the browser does not create the event attribute group. You can create the event attribute group manually by not clicking Browse to display the JMX browser. Instead, just manually type in the Object Name pattern and click Finish. This creates the same event attribute group that you would have gotten from the browser had notifications been detected. It also creates an attribute group to receive data with no attributes defined. This other attribute group can be deleted.

Locally configuring an agent fails on Windows

An Agent Builder created agent containing configuration properties requires IBM Tivoli Monitoring to install the Java Runtime Environment. If the Monitoring Agent for Windows OS was installed using the tacmd createnode command to create an IBM Tivoli Monitoring v6.2 Fix Pack 1: Windows OS Agent, then the Java Runtime Environment will not be installed. On this system, you cannot locally configure any Agent Builder created agents that contain configuration properties.

The agent can still be configured remotely using the Tivoli Enterprise Portal. The following solutions can be used to install the Java Runtime needed by IBM Tivoli Monitoring:
  • Install an IBM provided agent that requires the Java runtime.
  • Install the Monitoring Agent for Windows OS locally instead of using the tacmd createnode command.
  • Install the IBM Tivoli Monitoring V6.2: Windows OS Agent using the tacmd createnode command instead of the IBM Tivoli Monitoring V6.2 Fix Pack1: Windows OS Agent.

Command line does not allow me to configure an agent

I cannot use the itmcmd config command (or CandleConfig) to configure an agent containing JMX, SNMP, or JDBC attribute groups.

You must have IBM Tivoli Monitoring v6.2 Fix Pack 1 to configure an agent containing JMX, SNMP, or JDBC attribute groups using the itmcmd config (or CandleConfig) commands.

You can use the GUI to configure the agent if upgrading is not an option.

JMX monitors

JMX Monitors are not working with the JBoss application server.

The following is needed to make JMX Monitors work. Copy the connJboss-1.0.jar file from CANDLE_HOME/TMAITM6/kxx/jars/common/connectors/jboss on Windows or CANDLE_HOME/dynarch/kxx/jars/common/connectors/jboss on UNIX to JBoss_install/server/default/lib.

If you have a server configured other than the default server, the "default" part of the path will be different for your server. If the JBoss server is running, it must be restarted after copying this file.

JMX browser connecting to WebSphere 6.1

The JMX MBean browser cannot connect to WebSphere 6.1 with security enabled, using the SOAP connector protocol.

The browser can collect the MBean names using the rmi connector protocol.

Error code for attribute group, but data being returned

The Performance Object status error code for my JMX attribute group is ATTRIBUTE_ERROR. Data is being returned for the MBeans, so what does this mean?

A JMX attribute group that has an error code set to ATTRIBUTE_ERROR in the performance object status attribute group has one or more attributes that could not be collected. This not only indicates a problem with one or more attributes, but it is a performance issue as well.

To determine which attributes are having a problem, look for the exceptions in the JMX trace log file. The exceptions typically indicate the class path could not locate a certain class or the attribute object could not be serialized.

When you see this error, the attributes have to be collected individually from the MBean Server instead of collecting all attributes in one remote call. This can significantly impact the performance of the agent.

JMX data provider connecting to WebSphere 7.0

JMX data provider fails to connect to WebSphere 7.0 with security enabled, using the RMI connector.

This is an issue with the WebSphere Application Server.

Upgrade to WebSphere 7.0.0.1 or later to resolve this connection issue.

JMX data provider connecting to Oracle WebLogic Server on AIX

The JMX data provider fails to connect to the Oracle WebLogic Server when running on AIX.

This connection problem is a limitation documented by Oracle WebLogic.

Connect to Oracle WebLogic remotely from an operating system with a compatible Sun Java runtime environment. In this case, the agent must be configured to use the compatible Sun Java runtime environment instead of the IBM Java runtime environment.

Title bar not displaying the workspace name

When looking at a generated agent through the Tivoli Enterprise Portal, the title bar displays Nav_Node - TEPS_Hostname - UserID rather than Workspace_Name - TEPS_Hostname - UserID.

When an agent is generated, workspaces are not created by default. If no workspace is defined for an attribute group, the Tivoli Enterprise Portal displays a generated workspace for the attribute group that shows all attributes in a table view. The title bar of the Tivoli Enterprise Portal will display the Navigation Node ID because the workspace name is not defined. You can click File > Save to save the generated workspace as the default workspace. A workspace name will now be displayed in the title bar. Refer to Creating workspaces for more information.

The log file is not monitored properly

If you build an agent with a Log File data source, and all or part of the log file name comes from a Configuration Property, and that Configuration Property value contains a space, the log file will not be monitored properly.

Enter the Windows short name for the file or path in the agent configuration. You can get the short name of any file or directory with the DIR /X command issued at a Windows command prompt.

Agents not translated

Agents created by Agent Builder are not translated when shown on the Tivoli Enterprise Portal.

The Agent Builder does not build language packs, so the text that is displayed in the Tivoli Enterprise Portal is in the language you use when building the agent.

RAS1 log errors

I see the following in the RAS1 log for my agent. What does it mean?

(46C30EA0.0000-2180:getprocesscmdline.cpp,387,

"GetProcessCmdLine")
Unable to read the process environment block.
ReadProcessMemory returned 0.
(46C30EA0.0001-2180:getprocesscmdline.cpp,589,
"getPIDCommandLine")
Failed to get process command line, pid(4)
error: Only part of a ReadProcessMemory or
WriteProcessMemory request was completed.

This can occur on some systems for the process that represents the 'system' in Windows. The process environment block is not available for this process. System is a special process and normally will not be one you are actually monitoring. If all of the data in your Availability table is filled in, then this does not represent a problem. You can verify this by checking the PID printed in the trace against the PIDs for the processes you are monitoring.

Log file data mismatch

A record from the log file is not displayed on the Tivoli Enterprise Portal or the last entry or last few entries on the Tivoli Enterprise Portal do not match the contents of the log record.

Check for error messages in the trace file, for example, HOSTNAME_81_k81agent_
465c087e-01.log. Look for a trace entries like these:
(465C08D7.0000-2A4:logmonitorqueryclass.cpp,
506,"LogMonitorQueryClass::setInstanceData")
Agent metric count and UA metric count do no match!
Agent count=<7>, UA count=<6>. (465C08D7.0001-2A4:logmonitorqueryclass.cpp,
561,"LogMonitorQueryClass::setInstanceData")
UA ran out of values for agent metric! Agent name=<rest>
The "Agent count" indicates the number of attributes that are expected to be filled in from a log file record. The "UA count" is the number of records that the data provider parsed from the log record. A mismatch means that some of the attributes could be parsed from the log record, but others could not because there was less data in the log record than expected.

Installed agent does not show up

An installed agent does not show up in the Tivoli Enterprise Monitoring Services utility.

Select View > Refresh from the Manage Tivoli Enterprise Monitoring Services window.

Situations not showing up

The situations should be true (or my node should display the little red circle indicating that the situations are true), but they are not displayed as true. I checked and the data has exceeded the threshold. Why can't I see the situation on my node in the navigation tree.

Some of the associations between nodes and situations are loaded when the Tivoli Enterprise Portal starts. Restart the Tivoli Enterprise Portal.

Queries not showing up

The new queries are not showing up.

Install the agent and then recycle the Tivoli Enterprise Monitoring Server and Tivoli Enterprise Portal.

Error states the startagent command failed

A popup error on the Tivoli Enterprise Portal says that the "C:\IBM\ITM\InstallITM\Batch\kincli
-startagent -akxx" command failed.

After installing IBM Tivoli Monitoring on a Windows endpoint, the machine must be rebooted before it can successfully be a target of remote deployment, or the start, stop, and remove functions that are available on the Tivoli Enterprise Portal. The reason for this is that the IBM Tivoli Monitoring installation changes the system's PATH to include some DLLs that need to be found for those functions to work, but the services do not pick up those changes until the machine reboots.

When running command return code, the return code is not accurately shown

When running the Windows command return code to run a command and analyze the return code, and the command is a .bat or .cmd script, the return code is not accurately shown when the script exits with something like "exit /b 2".

To get this value, surround the target script with a script that calls the target script and then executes "exit %ERRORLEVEL%".

Attribute group's data missing from Tivoli Enterprise Portal

I built an attribute group using a 'script' source and I do not see any data in my Tivoli Enterprise Portal.

Read the log file and look for text that looks like:

(46543D0F.0019-1A0C:shellqueryclass.cpp,331,

"internalCollectData") Missing metrics. Skipping
row. expected 15 tokens, parsed 2. Input:<FINDSTR>,
separator:<:>

This text indicates that your script returned data that did not match the defined format (in this case, items separated by a colon). It was attempting to parse the string contained within the first <> pair.

Fix the script to return data in the correct format. You can test this by editing the script in the agent directory because it will call the script each time it tries to collect data.

Changing version number causes errors

If you have an agent that you created with the Agent Builder, and you modify it and change the version number, if you attempt to deploy the agent remotely with the new version, you get an error that says "KFWITM291E An agent configuration schema was not found."

The Tivoli Enterprise Monitoring Server and Tivoli Enterprise Portal Server support have to be installed again, even if you changed nothing but the version number. This is due to certain configuration files having the version number in them. You receive failures if the files with the new version number are not present.

Changing product code, company identifier, or agent identifier causes problems

You have an agent that you created with the Agent Builder, and you modify it and change the product code, company identifier, or agent identifier after you have created workspaces or situations.

All the situations and workspaces need to be re-created.

A script provider behaves oddly when edited

For instance, you edit the script and add a sleep 30 to simulate a timeout. The timeout occurred as expected during the next refresh of the group. Then you take out the sleep 30 and refresh the group again. The calculated values are now all set to 0.

This is due to the previous data point being lost.

Core dump after editing the ref file

Core dump with a seemingly innocent modification to the .ref file. For example, splitting attributes of an element in the ref file onto different lines still produces valid XML, but the agent coredumps.

Do not edit this file.

The Tivoli Enterprise Portal does not show any columns or column headers

Navigator groups in an agent do not show any columns or column headers. Instead, an error at the bottom of the view is displayed: KFWITM220E Request failed during execution.

In addition to navigator groups that the user created, this error is also seen with internally generated navigator groups such as the Availability and JMX Monitors navigator groups.

When defining a navigator item, there can be more than one attribute group with a node in the navigation tree. When there is more than one attribute group, you need to assign a query to a workspace. After you do this, you can see the data.

Bad string values collected from SNMP are displayed in attribute groups This happens for OCTETSTRING types where the value is binary data and not strings. Binary data is not translated for display. It is forwarded as the binary data.

No data in the Tivoli Enterprise Portal

I have a script data source and I am not getting any data in the Tivoli Enterprise Portal.

Change the trace setting to: ERROR (UNIT:shell ALL)

If you see a trace like the one below your script is not returning the data in the expected format.

(45FEFE30.001F-D14:shellqueryclass.cpp,329,

"internalCollectData") Missing metrics.
Skipping row
(45FEFE30.0044-D14:shellqueryclass.cpp,329,

"internalCollectData") Missing metrics.
Skipping row

Cannot remove agent from the Tivoli Enterprise Portal

When trying to remove an agent, you see that the agent is still listed in the desktop navigation view, but it is greyed-out.

This problem occurs if the agent remains in the managed system list in IBM Tivoli Monitoring. Perform the following steps:

  1. Select the Enterprise node, which is the top node in the physical tree in the Tivoli Enterprise Portal.
  2. Right-click Workspace > Managed System Status.
  3. Select the entry for the agent.
    Note:
    The name is displayed in the second column.
  4. Right-click and select Clear offline entry.
  5. The element is removed from the tree in the Tivoli Enterprise Portal.

Service or process monitoring returns zeros for metrics

When creating an agent to monitor Service availability, the column values are not correct. All metrics, including the Process ID, are zero.

For example:

  • For Services, you might receive the following information:
    Status=UNKNOWN
    and zeros for the availability metrics.
  • For Processes, you might receive the following information:
    Status= PROCESS_DATA_NOT_AVAILABLE
    and zeros for the availability metrics except for Process ID.

Ensure that the agent is being run under an Administrator ID.

Trying to monitor a non-existent Performance Monitor object

The agent runs and tries to monitor a Performance Monitor object, but a status of INACTIVE is displayed along with the error code OBJECT_NOT_FOUND.

The Performance Monitor object does not exist in the system. Specify a Performance Monitor object that exists in the system, or install the monitored application that will create the Performance Monitor object.

Trying to monitor a non-existent WMI class

The agent runs and tries to monitor a WMI class, but a status of ACTIVE is displayed along with the error code NO_INSTANCES_RETURNED.

The WMI class was not found and does not exist in the system. WMI collection displays the error message in the log each time the agent tries to collect the data. Specify a WMI class that exists in the system, or install the monitored application.

Agent runs a command return code but does not return data

When your agent runs a command return code script, the end of the log might display the information such as the line in the following example:

FACWIN5B_test_datasource_442816d1-02.log:
(4428171C.30A2-1A50:applicationpinglistelement.
cpp,100,"isApplicationAvailable")
Running Application Ping Command hello.bat 

If this type of information is in the log and is the last to be displayed, the script for the command return code did not end normally. The agent is locked up and cannot return data.

Investigate the trace log. An entry similar to this example indicates that the command return code did not run to completion.

Amend your command return code so that it is completed in a reasonable amount of time (for example, no more than 10 seconds).

UNKNOWN status displayed when trying to monitor a service

You try to monitor a service but the message Status=UNKNOWN is displayed, even though you have already verified that the agent has Administrator privileges.

The service is not installed on the system. When the view is being built for the Availability table, rows where the status is Unknown should be filtered out to prevent confusion, especially when an application is composed of a set of optional services. The lack of a service is not an error in this case; it is normal.

No data shown for an attribute group

You try to monitor a data source but no data is shown from an attribute group that collects data.

If the systems used for developing and testing the agent are different, the WMI classes and the Performance Monitor objects could be different too. Develop and test the agent on the same version of Windows and the same version of the monitored application that you want to manage.

Do not see situations in the console

I installed my agent, situations and workspaces. I configured the agent and started it. I see it in the Tivoli Enterprise Portal, but I do not see the situations in the navigator tree or the Situation Event Console. I have checked, and the situations are associated with the right nodes.

Restart the Tivoli Enterprise Portal.

Data being sent by the agent does not look like it should in the Tivoli Enterprise Portal

The data looks like the attributes are not being parsed in the right places.

Re-installing IBM Tivoli Monitoring is the safest way to ensure that you have not introduced incompatibilities between the phases of your development.

Note:
The agent builder includes features to prevent you from introducing these types of changes into later versions of your agent, so you won't have this type of problem as you build and deploy updates to an agent.

The status for a re-installed agent indicates that the agent is configured in Managed Tivoli Monitoring Services

While developing an agent, you will likely generate, install, test, update the agent and then generate, install and test again. When you do this, the agent status indicates that the agent is configured in Managed Tivoli Monitoring Services after the agent is re-installed.

If you modified the agent configuration, then reconfigure the agent, and restart.

Do not see situations in the navigator tree

I installed my agent, situations and workspaces. I configured the agent and started it. I see it in the Tivoli Enterprise Portal, but I do not see the situations in the navigator tree. They are not associated with the correct node. (I right-click on the node and select the situations, but they do not appear in the default list.)

When you created the situation on a live system, it was automatically distributed to the agent using the host name. This host name will not generally be available in every environment, so you should distribute the situation to a generic managed system list that exists when the agent is installed.

Distribute the situations to the CUSTOM_app_name00 Managed System List, and re-import the situations. Then rebuild the Solution Installer image, and re-install the Tivoli Enterprise Monitoring Server support for the agent.

Agent crashes

One reason this could occur is that the ICCRTE_DIR might not be set in the ENV file on Windows systems.

You might have cygwin installed, so ICCRTE_DIR is set as:
ICCRTE_DIR=
The Kxxinstall.log has the following text that shows the problem:
find: ICCRTE_DIR=: No such file or directory
You can fix this problem by taking cygwin out of the path and setting ICCRTE_DIR manually.
Subnode names

Subnodes from different agent instances have the same managed system name.

The Managed System Name for a subnode consists of 2 letter agent Product Code:first 24 characters of the Subnode ID:3 letter Subnode Type

The first 24 characters of subnode IDs must be unique for all instances of the subnode type in the IBM Tivoli Monitoring installation.

The agent automatically prepends "PC" to prevent the subnodes from colliding with subnodes created by other agents. It automatically appends the subnode type to prevent collisions with other subnode types in the same agent. It uses the first 24 characters of the Subnode ID (which you control) as the final token.

No part of the Agent Instance or the Agent Host System is used in constructing the Subnode Managed System Name, so by using the same Subnode ID in 2 instances of the Agent, even Agent Instances hosted on separate systems, the Managed System Names will collide and the subnodes will not function.

Installing 2 agents with the same script name

On Windows operating systems, when two different agents are installed on the same Tivoli Enterprise Monitoring Agent, and they both have a script with the same name, the script from the last agent that was installed or deployed overwrites any existing scripts of the same name. Scripts are copied into the instdir\tmaitm6 directory without a warning.

There is no solution. The files are copied into the TEMA directory (Windows and UNIX) so that they are all in a consistent place and easily accessed by the agent and each other.

Agent Configuration stops

When deploying or configuring an agent that contains subnodes and requires a minimum IBM Tivoli Monitoring version of 6.2.1 on a Tivoli Enterprise Monitoring Agent that has an IBM Tivoli Monitoring version earlier than 6.2.1, configuration might stop.

  1. Upgrade the OS agent on the target system to the required prerequisite for the agent, version 6.2.1 or later.
  2. Change the Minimum ITM version field to 6.2 and rebuild the agent, or upgrade to IBM Tivoli Monitoring 6.2.1 or later.

    The subnode configuration parameters will not have User Configurable Initial Values, although default values might still be assigned when building the agent. You also cannot override configuration parameters that are not explicitly listed in the subnode configuration overrides section. This can be addressed in two ways:

    • Configure multiple instances of the agent. Each instance can provide a different set of values for parameters that are not included in the subnode overrides.
    • Rebuild the agent so that the Subnode Configuration Overrides contain all parameters that might need to be overridden.

SNMP attribute group not collecting data reliably

Data is collected intermittently or not at all. The SNMP version and credentials are configured correctly The Performance Object Status Error Code for the attribute group shows "NO RESPONSE RECEIVED".

Note:
This applies to SNMP attribute groups, so the Object Type in the Performance Object Status table is SNMP.

The agent trace file shows the following message: Timeout occurred. No response from agent.

Here is a sample entry: (48A18C71.000A-12:snmpqueryclass.cpp,1714,"internalCollectData") Timeout occurred. No response from agent.

The IBM Tivoli Monitoring SNMP data provider is multithreaded to enhance performance. The SNMP data source that is being monitored might not be able to respond to multiple incoming requests in a timely manner. There are tuning options that can improve reliability of data collections:
Reduce the thread pool size
The default thread pool size is 15. Try reducing the size to 5. This setting can be adjusted in the agent ENV file by setting the CDP_DP_THREAD_POOL_SIZE environment variable.
Increase the SNMP Response Timeout
The default SNMP Timeout is 2 seconds. Try increasing the timeout to 6 seconds. This setting can be adjusted in the agent ENV file by setting the CDP_SNMP_RESPONSE_TIMEOUT environment variable.
Reduce the number of SNMP retries
The default number of SNMP retries is 2. Try reducing the size to 1. This setting can be adjusted in the agent ENV file by setting the CDP_SNMP_MAX_RETRIES environment variable.
For more information about setting agent environment variables see Environment variables.

Windows commands not running as expected

Take Action command does not run Windows command as expected.

Availability Functionality Test does not run Windows command as expected.

Script data source does not run Windows command as expected.

The mechanism used to start a process in Take Action commands, and agent runtime data providers is the native Windows process management API, CreateProcess(). With this command you can start processes that are implemented in .bat, .cmd, or .exe files. Windows implements several common command functions as internal commands in the shell and not as programs. These include commands like echo and dir (common commands used to test script execution). Since these are not programs, they cannot be started with createProcess().

To invoke these commands, create a .bat or .cmd file that contains the commands.

It is possible to invoke the command processor and provide the built-in command as an argument, as shown in the following example:

cmd /c "echo val1;1;val2;2"

Remember that you are collecting the return code from "cmd", not "echo" in this example.

CIM data provider stops collecting data from AIX OpenPegasus 2.6.1 CIM Server.

Data collection resumes if the AIX CIM Server is stopped and restarted.

This issue has been resolved in IBM Pegasus CIM Server V2.6.1.35 available at

https://www.ibm.com/services/forms/preLogin.do?lang=en_US&source=aixpegcim

You can display the current version of the IBM Pegasus CIM Server file sets by using lslpp -l sysmgt.pegasus.cimserver.rte

Upgrade the IBM Pegasus CIM Server to 2.6.1.35 or later

For current details, refer to AIX Information - Common Information Model Guide at http://publib.boulder.ibm.com/infocenter/systems/scope/aix/topic/com.ibm.aix.cim/doc/cim/About.htm?tocNode=int_187407

Collecting metrics through the Windows APIs

To collect metrics through the Windows APIs, the agent must be hosted on a Windows operating system, and remote registry administration must be enabled on the remote systems.

CIM data provider stops collecting data from the Solaris WBEM Server.

Several Solaris patches are required to achieve a stable Solaris WBEM CIM Server. Minimum versions required are:

Solaris 9:
Patch Synopsis

  • 112945-46 WBEM Patch (Sparc)
  • 114193-36 WBEM Patch (X86)
  • 116807-02 SMC Security Patch (Sparc)
  • 116808-02 SMC Security Patch (X86)
  • 114501-01 DRM Provider Patch (Sparc)
  • 114502-01 DRM Provider Patch (X86)
  • 114503-14 User Manager (VUserMgr.jar) Patch (Sparc)
  • 114504-14 User Manager (VUserMgr.jar) Patch (X86)
  • 114711-03 Disk Manager (VDiskMgr.jar) Patch (Sparc)
  • 114712-03 Disk Manager (VDiskMgr.jar) Patch (X86)
  • 112943-09 Volume Management (VVolMgr.jar) Patch (Sparc) [DiskSuite/SVM]
  • 114192-06 Volume Management (VVolMgr.jar) Patch (X86) [DiskSuite/SVM]

Solaris 10:
Patch Synopsis

  • 119313-22 WBEM Patch (Sparc)
  • 119314-23 WBEM Patch (X86)
  • 121308-14 Console Patch (Sparc)
  • 121309-14 Console Patch (X86)
  • 119315-14 Solaris Management Applications Patch (Sparc)
  • 119316-14 Solaris Management Applications Patch (X86)
  • 124188-02 Trusted Solaris Attributes Patch (Sparc)
  • 124189-02 Trusted Solaris Attributes Patch (X86)

CIM data provider intermittently fails to collect data from Solaris 9 WBEM Server after all patches have been applied.

This issue has been reported to Sun Microsystems and is a vendor limitation.

If you are running a typical multi-threaded agent against a small number of remote systems, the agent might send all of the requests concurrently to a single WBEM CIM Server. The CIM Server might not handle all requests. In a typical environment, with multiple remote CIM Servers, the requests are spread out across systems and this problem does not occur.

Set the following environment variable in the agent env or ini file:
CDP_DP_LOCK_CIM_ACCESS=YES

When CDP_DP_LOCK_CIM_ACCESS=YES is set, the agent serializes the CIM requests that it sends. The lock covers the lifespan of the request to the other system. When the request is received, the agent unlocks and then processes the result.

Note:
This flag can have a negative impact on the scale of the agent. Use this flag only when it is absolutely necessary.

CIM data provider stops collecting data from Solaris 10 WBEM CIM_FileSystem class.

The Performance Object Status for this attribute group reports GENERAL ERROR.

This issue has been reported to SUN and is a Vendor Limitation. There is an error in the Solaris WBEM CIMOM. Requests for the CIM_FileSystem Class log this error in the WBEM log:
nfs_mntinfo: Can't access mnttab. 
Too many open files
CIM_ERR_FAILED:
nfs_get_mount_list Failed.

The workaround is to collect data from the Solaris_LocalFileSystem class, or to recycle the WBEM CIM Server.

CIM data provider cannot collect data remotely from Solaris 10 WBEM server after applying Solaris Patches:

  • 121308 Console Patch (Sparc)
  • 121309 Console Patch (X86)
If you are running Solaris 10 6/06 or earlier, you must modify the your WBEM configuration file to allow remote connections after installing patch 121308-XX. Refer to the following Sunsolve document for detailed information:
  • Document ID: 211275
  • Title: Solaris[TM] 10 WBEM only listens to port 898 on localhost

Local configuration JRE warning

Locally configuring an agent displays this warning: Kincfgexit Java Runtime Environment was not detected! Extended agent configuration is disabled - if remote configuration for the agent is supported, complete the process using the Tivoli Enterprise Portal.

The Monitoring Agent for Windows OS was deployed using tacmd createnode, so no Java was installed on the local system. The agent has been configured enough to allow it to connect to the Tivoli Enterprise Monitoring Server using the default agent configuration parameters. Start the agent to allow it to connect to the Tivoli Enterprise Monitoring Server. You can then complete the configuration using the Tivoli Enterprise Portal. Optionally, you can install a supported JRE locally to configure the agent using the Manage Tivoli Enterprise Monitoring Services interface.

Also, perform the steps in the following problem: Password is not stored when locally configuring the agent on a Windows system.

Password not stored when locally configuring the agent on a Windows system

When IBM Tivoli Monitoring installs the Java Runtime Environment it applies a special patch to encrypt the password. When the JRE was manually installed, this patch was not applied. If you manually installed the JRE, perform the following steps so that passwords are correctly encrypted:
  1. Copy the following three files from the %CANDLE_HOME%\InstallITM directory to the C:\Program Files\IBM\Java50\jre\lib\security directory:
    • java.security
    • local_policy.jar
    • US_export_policy.jar
  2. Reconfigure the agent to correctly encrypt the password and store it in the configuration file.

Connecting to Microsoft SQL Server using integrated authentication is failing

You can connect to a Microsoft SQL Server without a user ID or password by using Microsoft's integrated authentication. This only works on Microsoft Windows operating systems and requires that you have a JDBC driver that supports integrated authentication. Integrated authentication can be used for JDBC connections in the Agent Builder browser or in the agent runtime.
  • To connect using the JDBC browser, make sure the JDBC driver authentication dll is present somewhere in the system path of the computer. Start Agent Builder after the dll is located in the path. After Agent Builder is running, you can use the JDBC browser without specifying a user ID or password in the connection properties dialog box to connect to a Microsoft SQL Server using integrated authentication. This uses the current user's account to perform the authentication.
  • The agent runtime can connect to a Microsoft SQL Server using integrated authentication by performing the following steps:
    • Make sure the JDBC user name and JDBC password configuration properties are changed so the Required check box is not selected in the Runtime Configuration editor tab.
    • The authentication dll that comes with the JDBC driver must be located in the system's path or when you configure the agent, include a JVM argument in Java properties that specifies the location of the user authentication dll. for example: -Djava.lib.path=E:\sqljdbc_1.2\enu\auth\x86
    • When you configure the agent, do not specify a user name or password in the configuration properties for JDBC.

Agent support files remain after uninstalling

After uninstalling an agent, the agent support files remain on the system.

The support files for the Tivoli Enterprise Monitoring Server and the Tivoli Enterprise Portal are not removed by uninstalling the agent.
  1. Remove Tivoli Enterprise Monitoring Server Application support by completing the following steps:
    1. Use Manage Tivoli Enterprise Monitoring Services.
    2. Select Tivoli Enterprise Monitoring Server.
    3. Right-click and select Advanced.
    4. Select Remove TEMS application support.
    5. Select the agent to remove its application support.
  2. Remove the agent from the Tivoli Enterprise Portal using the following procedure:
    1. Ensure that your Tivoli Enterprise Monitoring Server and Tivoli Enterprise Portal Server are up and running.
    2. Log in to your Tivoli Enterprise Portal client.
    3. From the Tivoli Enterprise Portal client Physical Navigator views, right-click Enterprise and select Workspace > Managed System Status. The Managed System Status workspace is displayed.
    4. Select all of the IBM Tivoli Managed Systems for your agent.
    5. Right-click and select Clear off-line entry, which clears all of the entries from that table.

JDBC connections are failing to find my JDBC driver

JDBC connections are failing to find my JDBC driver that was added to the classpath.

Make sure the JDBC driver is compatible with the JRE that you are using. JDBC 4.0 drivers are compiled with Java 1.6. To use a JDBC 4.0 driver, the JRE that you configure the agent to use must be at least Java 1.6. Note that Agent Builder uses Java 1.5, so you must use a JDBC driver that is compatible with Java 1.5 to use the JDBC browser in Agent Builder.

Navigator display in the Tivoli Enterprise Portal shows a combination of old and new nodes or shows the wrong data when you click on a node

When you remove attribute groups, rename them, or reorganize existing groups into or out of navigator groups, the display in the Tivoli Enterprise Portal might show a combination of old and new nodes.

If you are using Agent Builder 6.2.1.2 or later, resolve this issue by restarting the Tivoli Enterprise Portal Server and then, restart the Tivoli Enterprise Portal.

Agent installation fails for agents built with Agent Builder V6.2.2 or earlier

For agents built with Agent Builder V6.2.2 or earlier, installing the agent might prevent other agents from being successfully installed. If the TECLIB directory does not exist in TMAITM6 when the Agent Builder agent is installed, then a file is created with the name TECLIB. When subsequent agents attempt to create files in the TECLIB directory, the agent installation fails.

Perform the following steps:
  1. Rename the file from TECLIB to kXX.baroc, where XX is the two-character product code for the agent to which the file belongs.
  2. Create a directory called TECLIB and copy the renamed kXX.baroc file into the newly created TECLIB directory.
  3. Re-try the installation or deployment of the agent that failed.

Installing the agent or application support fails with error message: KQZ0208E

When installing the agent or application support and specifying a password with special characters, the login to the monitoring server fails with error message: KQZ0208E. The specified username or password is incorrect.

The password cannot include special characters.

Ping attribute group on Windows slow to respond

It is a long time before I start getting data back from my ping attribute group on Windows.

Name resolution on Windows can take about 5 seconds to timeout if an IP address does not resolve to a host name. If you have defined several devices that have this name resolution issue the ping attribute group takes longer than expected to initialize. To prevent this problem, make sure the devices you define can be resolved in DNS or using entries in your local host file.

Authorization failures when using SSH public key authentication

Authorization failures when using SSH public key authentication.

Check the agent log file. An error message similar to the one below indicates that the agent process can not open the private key file. This might happen on Windows systems since the agent runs as a service that might not be able to access another users private key file. Resolve this problem by making sure the agent process runs as a user that can read the private key file.

The following is a an example of an error message when he agent process can not open the private key file: (4C6D417B.0048-1230:userauth.c,631,"file_read_privatekey") -16 - Unable to initialize private key from file

Running the installIraAgentTEMS.sh results in an error on UNIX

Running the installIraAgentTEMS.sh results in the following error on UNIX: Installation failed. Please see the log in /opt/IBM/ITM/logs/product_code_TEMSInstall.log

The end of the product_code_TEMSInstall.log contains the following lines:

. . .
+ rm -rf /tmp/k4s
rm: 0653-610 Cannot remove current directory /tmp \ 
/k4s.
+ [ -d /opt/IBM/ITM//tables/cicatrsq/TECLIB ]
+ cp ./ira/agent/common/k4s.baroc /opt/IBM/ITM// \ 
tables/cicatrsq/TECLIB/k4s.baroc
cp: ./ira/agent/common/k4s.baroc: A file or \ 
directory in the path name does not exist.
+ fail
~
Extract your agent package into a directory other than /tmp/product_code, where product_code is in lower case.

Agent with SSH to Windows does not run a command

An agent with SSH to Windows, does not run a command, or, it appears the agent runs the command with SSH to Windows with no effect.

To run a remote command to a Windows host, you must have a Linux-like shell environment installed. Cygwin is an example of a Linux-like shell environment.

To verify if a shell environment exists, SSH or log in to the remote host and enter the command:

PATH=$PATH:. <command>

If the command runs, then a shell environment exists.

On Windows, a mapped drive to a Ping, Log file, JMX, or JDBC configuration file cannot be read

On Windows, a mapped drive to a Ping, Log file, JMX, or JDBC configuration file cannot be read. The agent runs as a service and cannot see the mapped drives.

On Windows, do not use a mapped drive to store files required by the agent.

Missing or unexpected data for a socket attribute group in the Tivoli Enterprise Portal

There is missing or unexpected data for a socket attribute group in the Tivoli Enterprise Portal.

Check the agent log if there are missing rows of data for a socket attribute group or if the data is not as expected. In the case of missing or unexpected data, check the log even if the Performance Object Status for the attribute group displays NO_ERROR, as NO_ERROR is displayed if any valid rows were returned.

The HTTP data provider does not properly handle URLs that use the https protocol

This problem occurs when you use an IBM Java runtime on Solaris or HP-UX systems. You will know you have this problem if the Tivoli Enterprise Portal shows "http://" before each of your https URLs. For example, http://https://website.ibm.com

This problem can be solved in one of two ways. One solution is to add the following parameter to the "JVM arguments" runtime configuration property:
-Djava.protocol.handler.pkgs=com.ibm.net.
ssl.www2.protocol 

Another solution is to use a Java runtime that was not provided by IBM if you have one available on the system.