IBM Support

Troubleshoot data center network connections to the "Top of Rack Switch" within the IBM Cloud Pak System

Troubleshooting


Problem

IBM Cloud Pak System client administrators and network data center teams debug problems between a client network and the IBM Cloud Pak System Top of Rack (TOR) switch.
From the IBM Cloud Pak System User Interface, queries are sent to the IBM Cloud Pak System Top of Rack (TOR) and returned. These commands are similar to the commands that the network data center engineers enter on other switches.
The IBM Cloud Pak System administrators download log files for their network data center teams to analyze. 

Environment

There is not a IBM Cloud Pak System Administrator option to stop or restart ports on the Top of Rack (TOR) switches.   Stop and restart ports from the datacenter network switches attached to the TORs.   
There is not a IBM Cloud Pak System Administrator option to power off or power on a Top of Rack switch.  Bypassing the IBM Cloud Pak System User Interface by removing power cables is not supported as this causes a disruption to the operation of the cloud environment and outages of unknown scope. 

For help with network changes and to prevent disruption to your business work with your IBM Sales team to engage IBM Expert Labs teams. 

  

Diagnosing The Problem

The IBM Cloud Pak System has two Top of Rack (TOR) switches.  The switch included in the product depends on the IBM Cloud Pak System Machine Type(MT). 

Reference the product documentation for more information
IBM Cloud Pak System W3500 MT 8558
IBM Cloud Pak System W3550 MT 8564
IBM Cloud Pak System W3700 MT 8536
IBM Cloud Pak System W4600 MT 4600

Check the status and type of switch from the IBM Cloud Pak System User Interface

  1. Select System Console > Hardware > Network Devices.
  2. Click "Top of Rack Switch" in the list displayed.  Look at both switches.
  3. On the "Events" line, click the number after "Error" and "Warning".  Investigate the events.
  4. Confirm the "Power status" and "Status" show available.
  5. Expand the "Customer Ports" section and confirm all the ports connected from your data center network to the IBM Cloud Pak System show a status of "Up".

Match the configuration of the ports to your network environment

  1. From System Console > System > Hardware > Infrastructure Map.
  2. Switch to Tree View.
  3. Select the TOR that is used for this connection.
  4. Expand the section for the TOR.
  5. Look under Customer Ports for information.
  6. Expand Switch Status.
  7. Under "Select a command", select "show interface status".
  8. Click "Run this Command".
  9. Analyze the results.

Use the other commands under the "Select a Command" section

Scroll to a "Select a command" pull-down with several commands:


You can use these commands to start troubleshooting problems with any of the ports that are not working.  The show transceiver command is a good command to use to check the state of the connection.

Look at the Top of Rack Switch logs

On the "Request System Logs" line, there is a "Generate" button. This feature runs a series of queries to the switch when pressed. A message appears for fast access to the Problem determination > System Troubleshooting page. On this page, watch for the 'collection set' created by the "Generate" button to complete processing. Download the file. Within this compressed file, Start with the "Network" folder. 

Depending on the IBM Cloud Pak System model type, the log files can be different.  
For example, if there is a BNTTOR.tech-support.txt" file,
  • Look for "Last 2000 syslog message information:" this section notes if the port is up or down. Look for the "snmpTargetParams Table:" section where you can verify the light levels on the connection. 
  • Check whether the logs show any port that is down. This grep command shows the "link down on port <port number>" occurrences in the logs.
    grep "port ${port}" BNTTOR.tech-support.txt|grep "link down"|wc -l 
  • Check the status of transceivers. This command matches on "<port number> SFP+"
    grep "${port} SFP+" BNTTOR.tech-support.txt
Compare the information that you gathered by using this technote with the information from the network data center equipment.  Look for what is different and what needs to be changed.

Resolving The Problem

Top of Rack (TOR) switch ports show in "down" in show interface status output?

If ports show in "down" state in the "show interface status" output, disable then re-enable the port on the network datacenter switches attached to the IBM Cloud Pak System TORs.   
The IBM Cloud Pak System User Interface for the TOR is updated periodically.  The commands in the "Select a command" list described earlier in this document provide an immediate response on the state of the TOR (for example, "show interface status").
Are the connections not working as expected?
Compare the configuration of the network datacenter switch with the IBM Cloud Pak System TOR configuration. Refer to Matching the configuration of the ports to your network environment in this document. 
How to manage changes to the configuration and connections to the IBM Cloud Pak System User Interface
The IBM Cloud Pak System has a tray and display on the system called the service console.  This feature allows an admin user to log in to the system user interface on an internal system network.  If you need to recover network access to the device, you can change the IBM Cloud Pak System network configuration with this service console.  It is directly connected to the device and bypasses any connectivity issues that are the result of changes in the IBM Cloud Pak System network configuration options.
How to mitigate problems and outages?
For help with network changes and to mitigate disruption to your business, work with your IBM Sales team to engage IBM Expert Labs teams.   

 

[{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSTQRPX","label":"IBM Cloud Pak System W4600 Commercial for VMware"},"ARM Category":[{"code":"a8m0z000000cwm2AAA","label":"Product Components"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSJPC5","label":"IBM Cloud Pak System W3700"},"ARM Category":[{"code":"a8m0z000000cwm2AAA","label":"Product Components"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS97LW","label":"IBM Cloud Pak System W3550"},"ARM Category":[{"code":"a8m0z000000cwm2AAA","label":"Product Components"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSZQFR","label":"IBM Cloud Pak System W3500"},"ARM Category":[{"code":"a8m0z000000cwm2AAA","label":"Product Components"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
12 September 2023

UID

swg21677265

Page Feedback