IBM Support

QRadar: Verifying SSH connectivity to the target Managed Host

Troubleshooting


Problem

When a Managed Host is suspected as the source of a problem, verifying SSH connectivity to that Managed Host is an important step.

Cause

Verifying that you are able to connect to a Managed Host from your Console by using SSH. This SSH test provides a wealth of information about the state of network connectivity to that host. Before you can start troubleshooting any issues on a Managed Host, you must be able to access it. While SSH is not intended to be a diagnostic tool, you cannot establish an SSH connection, unless the networking layers 1 and 2 are operating correctly for the host in question. How SSH is behaving can give you an indication of what actual diagnostic tools need to be used next.

Diagnosing The Problem

Successful Connection: If you are able to connect without any issues, this connection shows most (but not all) Layer 1 and Layer 2 networking issues can be eliminated as the possible cause of your symptoms.

If you are receiving error messages, or you are experiencing stability issues, responsiveness issues, these situations usually indicate problems in the network.

Layer 4 Problems or Authentication Errors: Various SSH authentication errors are considered Layer 4 SSH problems. Layer 4 problems with SSH are very much relevant as they interfere with the correct operation of QRadar. However, these layer 4 problems can be considered problems with the QRadar configuration or integration. They do not necessarily indicate anything that is wrong with the rest of your network environment. Some examples of these Layer 4 problems include receiving a Password Prompt or a Remote Host Identification Changed message.

Instability and Performance Problems: Instability and performance problems usually (but not always) indicate the presence of more complex layer 1 issues such as insufficient bandwidth, link layer aggregation (bonding or NIC teaming) misconfiguration, MTU problems, and so on.

Other Error Messages:
A simplified way of thinking about an SSH connection can be picturing that your console is sending messages called packets and the Managed Host responding with packets of its own. This example works especially regarding Layers 1, 2, and 3. Understanding where this simplified model is failing can help you pinpoint the problem. The three most common error messages encountered are the following:

  • Connection Timed Out: The connection that is timed out message means that the Console successfully sent packets onto the network but did not receive a response in a reasonable period. This error message can mean that the packets console sent were either not received or perhaps the response was lost. The presence of this error message proves that you are facing a communication layer issue and most likely in layers 2 or 3. The typical Layer 3 problem that causes this type of an issue is a firewall blocking your connection and dropping your packets without any notification. A typical Layer 2 issue is an incorrect gateway either on your console or your Managed Host. Other types of gateway, or router, problems on your network can also cause these error messages.
  • Connection Refused: The connection refused message indicates that the Console sent packets onto the network and received a response but this response was a rejection of the connection attempt. This message can happen if the packet was received by an application that is configured to refuse your connection attempt such as a firewall. In other words, you are likely facing layer 3.
  • No Route to Host: This message indicates that the Console does not know how to reach the targeted Managed Host. This message is usually an indication of a layer 2 or a layer 1 problem. Most likely causes include an incorrect gateway or subnet mask configuration, cabling, or other hardware problems on your Console.
  • Connection reset by peer: Can be caused by the clients firewall.This is due to it seeing the numerous SSH sessions opened thus flagging them as threat and dropping the connections.Typically, you can see this when attempting to add a remote managed host and if fails.

Resolving The Problem

If you are able to access the host by using SSH, SSH can be used to work around potential Layer 4 and some Layer 3 issues: Nonencrypted communications in QRadar use various transport layer ports. To learn more about it, read QRadar port usage. However, when encryption option is enabled, all transport layer communications are done over SSH tunnels and thus over TCP port 22, which is verified to work correctly with SSH.

However, if you are facing issues with your SSH connection, you need to take further actions to focus your troubleshooting. Since communication occurs between two endpoints, the QRadar Console, and your targeted Managed Host, it is best to have access to both end points during troubleshooting.

For this purpose, it is best to use the Integrated Managed Modules (IMM) that QRadar appliances come equipped with. It allows for out of band management of devices even when the management network access is not possible. QRadar: Managing QRadar Appliances with IMM can provide further.

First, ensure that you have out of band access to the targeted Managed Host. Next, try an SSH connection from the Console to the Managed Host. This SSH test provides further hints about which networking layer has a problem. The results of this attempt can be interpreted in much the same way as discussed in the Diagnosing the Problem section. However, note, SSH connections can legitimately be blocked from the Managed Hosts to the Console and even if they are not, you are normally prompted for a password.

Example: Attempting to SSH into the Managed Host A from the Console C, receives a Connection Timed Out error. This time out error indicates that you are most likely facing a layer 2 or a layer 3 problem. Then, you connect to the Managed Host A using its IMM and attempt to connect to the Console C from managed host A and you receive an "No Route to Host" error.

If you are unsure about what problem you are facing, even after the SSH connection attempt at the reverse direction:

Layer 4 Problems or Authentication Errors
QRadar: About Secure Shell (SSH) provides more documentation about troubleshooting these issues.


[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwsyAAA","label":"Admin Tasks"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions"}]

Document Information

Modified date:
13 November 2023

UID

swg22002165