Troubleshooting security

Use this information to troubleshoot issues with your security configuration.

Procedure

Problem: When your .NET application connects to the WebSphere® eXtreme Scale data grid, the following ConnectException exception is thrown by the .NET client:
```
IBM.WebSphere.Caching.ConnectException: Exception of type 'IBM.WebSphere.Caching.ConnectException' was thrown. ---> IBM.WebSphere.Caching.GridException: Exception of type 'IBM.WebSphere.Caching.GridException' was thrown. ---> IBM.Ws.Caching.Xio.TransportException+CommFailure: Authentication failed because the remote party has closed the transport stream. ---> System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
```
Additionally, the following error message is displayed in the .NET client SystemErr.log file:
```
[timestamp] 00000001 Client.Security.SSLCallback    Er CWOBJ6208E: A client certificate with a friendly name of “Bad Cert Name" does not exist in the key store location:name "CurrentUser":"My".
```
The exception and error message occur when clientAuthentication=true, and the .NET client configuration specifies a client certificate name that does not exist in the client keystore.

Solution: Verify that the .NET client clientCertificateKeyStoreLocation, clientCertificateKeyStore, and clientCertificateFriendlyName properties are correct and that the corresponding certificate exists in the associated Windows keystore that is on the system where the .NET client application runs.

Alternatively, if you do not intend to enable SSL client authentication, then you can set clientAuthentication=false to avoid the exception and error message.
Problem: When your .NET application connects to the WebSphere eXtreme Scale data grid, the following ConnectException exception is thrown by the .NET client:
```
IBM.WebSphere.Caching.ConnectException: Exception of type 'IBM.WebSphere.Caching.ConnectException' was thrown. ---> IBM.WebSphere.Caching.GridException: Exception of type 'IBM.WebSphere.Caching.GridException' was thrown. ---> IBM.Ws.Caching.Xio.TransportException+CommFailure: Authentication failed because the remote party has closed the transport stream. ---> System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
```
Additionally, the following error message is displayed in the .NET client SystemErr.log file:
```
[timestamp] 00000001 Client.Security.SSLCallback    Er CWOBJ6206E: The client certificate with a friendly name of "No Private Key" in key store location:name "CurrentUser":"My" does not have a private key.
```
The exception and error message occur when clientAuthentication=true, and no private key exists in the client certificate. When the .NET client is configured for SSL client authentication, the client certificate that is used to authenticate the .NET client must include the private key from the client so that the SSL handshake can authenticate the client. The current client certificate does not have a private key.

Solution: To avoid this error, configure the .NET client to use a client certificate that contains a private key.

Alternatively, if you do not intend to enable SSL client authentication, then you can set clientAuthentication=false to avoid the exception and error message.
Problem: When you connect to a dynamic cache data grid that is configured to use SSL, you might experience the following error in your log file:
```
FFDC Exception:javax.net.ssl.SSLHandshakeException SourceId:com.ibm.ws.xs.ssl.channel.impl.SSLConnectionLink ProbeId:540 Reporter:com.ibm.ws.xs.ssl.channel.impl.SSLConnectionLink@60b2d165
javax.net.ssl.SSLHandshakeException: General SSLEngine problem
```
The extended error message from the SSL handshake exception is:
```
PKIX path building failed: java.security.cert.CertPathBuilderException: unable to find valid certification path to requested target
```
Solution: A signer certificate was sent from a target host. However, that signer certificate entry is missing from the local truststore. You can use the Retrieve from port option in the administrative console to retrieve the signer certificate (also known as public certificate), and resolve the problem. If you determine that the request is trusted, complete the following steps:
1. Log in to the WebSphere Application Server administrative console.
2. Expand Security and click SSL certificate and key management. Under Configuration settings, click Manage endpoint security configurations.
3. Select the appropriate outbound configuration to get to the (cell):cell_name management scope.
4. Under Related Items, click Keystores and certificates, and click the CellDefaultTrustStore keystore.
5. Under Additional Properties, click Signer certificates > Retrieve From Port.
6. Enter a host name, port number, and alias.
7. Click Retrieve Signer Information.
8. Verify that the certificate information is for a certificate that you can trust.
9. Click Apply and Save.

Problem: The client end of the connection requires Secure Sockets Layer (SSL), with the transportType setting set to SSL-Required. However, the server end of the connection does not support SSL, and has the transportType setting set to TCP/IP. As a result, the following exception gets chained to another exception in the log files:

java.net.ConnectException: connect: Address is invalid on local machine, or
port is not valid on remote machine
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:389)
    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:250)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:237)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:385)
    at java.net.Socket.connect(Socket.java:540)
    at
com.ibm.rmi.transport.TCPTransportConnection.createSocket(TCPTransportConnection.java:155)
    at
com.ibm.rmi.transport.TCPTransportConnection.createSocket(TCPTransportConnection.java:167)

The address in this exception could be a catalog server, container server, or client.

Solution: See Configuring secure transport types for a table with the valid security configurations between clients and servers.

When agent is used, the client sends the agent call to the server, and server sends the response back to the client to acknowledge the agent call. When the agent finishes processing, the server initiates a connection to send the agent results. This makes the container server a client from connect point of view. Therefore, if TLS or SSL is configured, make sure the client public certificate is imported in the server truststore.
Problem: When users are authorized to access a WebSphere eXtreme Scale data grid, those users might also be authorized to perform management operations using the xscmd command or the stopOgServer command. Most data grid deployers restrict administrative access to only a subset of the users who can access grid data.
If you use the following command to access the data grid, you might also be authorized to perform administrative actions, such as listAllJMXAddresses:
```
./xscmd.sh -user <user> -password <password> <other_parameters>
```
If this operation works for this user, then any xscmd operation might also be performed by the same user.
Resolution: When eXtreme Scale components run with WebSphere Application Server, use the WebSphere Application Server administrative console to activate the security manager. Click Security > Global Security, and select the check boxes, Enable administrative security and Use Java 2 Security, to restrict application access to local resources.

Access to the management operations is controlled by the WebSphere Application Server security manager and is granted only to the users who belong to the WebSphere Administrator role. The xscmd command must be run from the WebSphere Application Server directory.

When eXtreme Scale components run in a stand-alone environment, additional steps are required to implement administrative security. You must run the catalog servers and container servers using the Java™ security manager, which requires a policy file.
The policy file resembles the following example:
Remember: There are typically MapPermission entries as well, as documented in Java SE security tutorial - Step 5.
```
grant codeBase "file:${objectgrid.home}/-" {
permission java.security.AllPermission;
};

grant principal javax.security.auth.x500.X500Principal "CN=manager,O=acme,OU=OGSample"
     {
        permission javax.management.MBeanPermission "*", "getAttribute,setAttribute,invoke,queryNames,addNotificationListener,removeNotificationListener";
     };
```
In this case, only the manager principal is authorized to do administrative operations using the xscmd command. Other lines can be added as necessary to give additional principals MBean permissions. A different type of principal is needed if you use LDAP authentication.
Enter the following command:
```
startOgServer.sh <arguments> -jvmargs -Djava.security.auth.login.config=jaas.config -Djava.security.manager 
-Djava.security.policy="auth.policy" -Dobjectgrid.home=$OBJECTGRID_HOME
```
```
startXsServer.sh <arguments> -jvmargs -Djava.security.auth.login.config=jaas.config -Djava.security.manager 
-Djava.security.policy="auth.policy" -Dobjectgrid.home=$OBJECTGRID_HOME
```
```
startOGServer.bat <arguments> -jvmargs -Djava.security.auth.login.config=jaas.config -Djava.security.manager 
-Djava.security.policy="auth.policy" -Dobjectgrid.home=%OBJCTGRID_HOME%
```
```
startXsServer.bat <arguments> -jvmargs -Djava.security.auth.login.config=jaas.config -Djava.security.manager 
-Djava.security.policy="auth.policy" -Dobjectgrid.home=%OBJCTGRID_HOME%
```
You specify -Djava.security.policy in this case, instead of -Djava.security.auth.policy.