HTTP 500 Internal Server Error is Received with an Apache HTTP Server Configured For SSL and IBM WebSphere Application Server v8.5 on the IBM i

Beginning at IBM WebSphere Application Server v8.5.5, the WebSphere Web Server Plugin product is no longer redirecting HTTPS SSL communications to the HTTP IP transport if the WebSphere Web Server Plugin is not properly configured to accept SSL communications. The change causes the following errors are recorded in the plugins_root/logs/web_server_name/http_plugin.log file if the Web Server plugin is not properly configured to accept SSL communications.

The following messages indicate the Web Server plugin's key database file was not copied to the web server keystore directory. Thus, the secure HTTPS transport cannot be initialized.

ERROR: lib_security: logSSLError: str_security (gsk error 202):  Key database file was not found.
ERROR: lib_security: initializeSecurity: Failed to initialize GSK environment. Secure transports are not possible.
ERROR: ws_transport: transportInitializeSecurity: Failed to initialize security. Secure transports are not possible.
ERROR: ws_server: serverAddTransport: Failed to initialize security. Secure transports are not possible.
ERROR: ws_server: serverAddTransport: HTTPS Transport is skipped. IMPORTANT: If a HTTP transport is defined, it will be used for communication to the application server.
ERROR: ws_server: serverAddTransport: Plugin will continue to startup, however, SSL transport PMICI7.PNAT.COM:6003 did not initilize.  Secure communication between app server and plugin will NOT occur. To run with SSL, additional products may need to be installed: 1) OS/400 Digital Certificate Manager (5722-SS1 or 5769-SS1, option 34) 2) Cryptographic Access Provider 5769-AC1 (40-bit), 5722-AC2 or 5769-AC2 (56-bit), 5722-AC3 or 5769-AC3 (128-bit)
...

The following messages indicate no active secure HTTPS transport can be found. These errors are a direct result of the previous messages.

ERROR: ws_common: websphereFindTransport: Nosecure transports available
ERROR: ws_common: websphereWriteRequestReadResponse: Failed to find a transport
ERROR: ESI: getResponse: failed to get response: rc = 4
ERROR: ws_common: websphereHandleRequest: Failed to handle request

To resolve your issue, IBM recommends the following steps be taken to enable the Web Server plugin to accept SSL/TLS communications.

The steps to configure the web server plugin to accept SSL communications are listed here:
NOTE:  You can ignore steps 1, 5, 6, and 8 since they are not needed on the IBM i.
https://www.ibm.com/support/knowledgecenter/SSEQTP_8.5.5/com.ibm.websphere.base.doc/ae/tsec_httpserv2.html

Once the web server plugin is properly configured for SSL, restart your Apache HTTP Server and review the http_plugin.log file to confirm the following messages no longer appear in the log.

ERROR: lib_security: logSSLError: str_security (gsk error 202):  Key database file was not found.
ERROR: lib_security: initializeSecurity: Failed to initialize GSK environment. Secure transports are not possible.
ERROR: ws_transport: transportInitializeSecurity: Failed to initialize security. Secure transports are not possible.
ERROR: ws_server: serverAddTransport: Failed to initialize security. Secure transports are not possible.
ERROR: ws_server: serverAddTransport: HTTPS Transport is skipped. IMPORTANT: If a HTTP transport is defined, it will be used for communication to the application server.
ERROR: ws_server: serverAddTransport: Plugin will continue to startup, however, SSL transport PMICI7.PNAT.COM:6003 did not initilize.  Secure communication between app server and plugin will NOT occur. To run with SSL, additional products may need to be installed: 1) OS/400 Digital Certificate Manager (5722-SS1 or 5769-SS1, option 34) 2) Cryptographic Access Provider 5769-AC1 (40-bit), 5722-AC2 or 5769-AC2 (56-bit), 5722-AC3 or 5769-AC3 (128-bit)

================================================================

If the Web Server plug-in key database (plugin-key.kdb) does not exist in the location specified in the plugin-cfg.xml file used by the IBM HTTP Server, then you might be able to implement the "UseInsecure=true" custom plug-in property to your web server definition to resolve your issue. In some cases when the web server is partially configured for SSL/TLS communications, the "UseInsecure=true" custom property is ignored. In this case, you would need to disable the HTTPS transport for the Application Server to use non-SSL between the plugin and the application server. IBM strongly does not recommend disabling the HTTPS transport for your application server since this change would prevent all secure HTTPS connections to your application server.

NOTE: This process allows non-secure communications between the Web Server Plug-in and the WebSphere Application Server. If you would like these communications to be secure, refer to the recommendation on how to "Configure the Web Server plugin to accept SSL/TLS communications".

=================================================================

Optional:  Implement the "UseInsecure=true" custom plug-in property for your web server definition.   Non-secure HTTP traffic would be used between web server plugin and application server.

(Not recommended by IBM.  Will only work if "Copy web server keystore" has never been ran using the instructions here: https://www.ibm.com/support/knowledgecenter/SSEQTP_8.5.5/com.ibm.websphere.base.doc/ae/tsec_httpserv2.html)

You can implement the "UseInsecure-true" custom plug-in property to allow HTTPS traffic to be redirected to the HTTP transport.  This property enables WAS (WebSphere Application Server) to function like it did at WAS v8.0 and earlier.  

Take this step to allow the Web Server plugin to create non-secure connections when secure connections are defined (the old behavior).

Create the custom property UseInsecure=true

This property is on the Servers > Web Servers > Web_server_name > Plug-in properties > Custom properties page in the IBM WebSphere Integrated Solution Console application for the failing WebSphere Profile.