Configuring LDAP security fails with SSL handshake exception: End user tried to act as a CA

IBM WebSphere Portal is successfully installed. The Portal configuration task wp-modify-ldap-security or wp-create-ldap is run to configure the Portal against an LDAP server. The configuration task fails with a SSL handshake exception noting "End user tried to act as a CA". This technote will document how to diagnose and resolve this error message so the configuration task completes successfully.

One of two conditions can cause this to occur:

1. The trustmanager being utilized is Ibmx509.

2. Custom properties set on the IbmPKIX trustmanager.

Any WebSphere Portal 6.1, 7.0, or 8.0 environment.

Collect the files noted in the document Collecting Data: Configuring Security for WebSphere Portal 7.0. Review the security.xml file to determine which trustmanager is in use and if the trustmanager has any custom properties set.

One of two solutions may resolve this issue:

1. Configure the trustmanager to be IbmPKIX.

- Login to the Deployment Manager or WebSphere Application Server console.

- Navigate to Security > SSL certificate and key management > SSL configurations > NodeDefaultSSLSettings > Trust and key managers

- Modify the Default trust manager drop-down box

from: IbmX509

to: IbmPXIX



- Save changes. If clustered, sync nodes.
- Rerun the failing configuration task.


2. Change custom properties

- Note: This solution assumes the condition is occurring with the trustmanager already set to IbmPKIX

- Login to the Deployment Manager or WebSphere Application Server console.

- Navigate to Security > SSL certificate and key management > SSL configurations > NodeDefaultSSLSettings > Key stores and certificates > NodeDefaultTrustStore > Custom Properties

- Review this screen and the values for the following true custom properties:
com.ibm.jsse2.checkRevocation
com.ibm.security.enableCRLDP



- With the current configuration, WebSphere Application Server will attempt to utilize a certificate revocation list (CRL) for SSL communications. A valid distribution point to obtain the CRLs must also be defined in the configuration. Two additional custom properties must be added to the configuration to allow CRL checking to function properly:

com.ibm.security.ldap.certstore.host = myldapserver.companyname.com
com.ibm.security.ldap.certstore.port = 389

NOTE: If a distribution point is not available, then delete both of the custom properties to disable CRL checking. The properties may be re-enabled at a later time when a distribution point is made available.

- Save changes. If clustered, sync nodes.
- Rerun the failing configuration task.