Troubleshooting WebSphere Portal, Domino Extended Products, and Domino SSO issues

This document is meant to serve as a troubleshooting guide for Single Sign-on (SSO) issues between IBM® WebSphere® Portal and Lotus® Sametime®, Lotus QuickPlace® and/or Lotus Domino® databases.

Index:
I. Correct way to test each product
II. General setup and configuration for all three products
III. Additional steps to take if the server is a QuickPlace server
IV. Additional steps for a Sametime server
V. Additional configuration needed if Portal uses a different directory than QuickPlace, Sametime or, Domino
VI. Other possible known problems and configurations

---

I. Correct Way to Test Each Product:

QuickPlace (also called Team Workplace) --

1. Sign on to WebSphere Portal.
2. Change the url in the browser to http://qpserver.domain.com/quickplace

Your name should appear at the top left corner. If it does not, SSO is not working to QuickPlace.

Sametime (also called Instant Messaging & Web Conferencing) --

1. Sign on to WebSphere Portal.
2. Change the url in the browser to http://stserver.domain.com/stcenter.nsf
3. Click on 'Attend a Meeting'.

You should see "signed on as <your name>" at the top right corner. If it does not, SSO is not working to Sametime.

Domino

1. Sign on to WebSphere Portal.
2. Make sure you have a Domino database where the Access Control List (ACL) has the -Default- and Anonymous access set to "No Access". The example in the next step assumes your database name is test.nsf, and it is located in the Domino \Data directory.
3. Change the url in the browser to http://dominoserver.domain.com/test.nsf

If a sign on screen appears, then SSO is not working for the Domino server.

---

II. General Setup and Configuration for all three products:

If you have not already done so, set up Multi-server Single Sign-on (MSSO) on the Domino server that will be used with WebSphere Portal. Follow the instructions in the following document to set this up:

"Enabling Single Sign-on for Domino and WebSphere Application Servers" (# 1098010)

If you have already enabled Multi-server SSO, then check the following settings:

A. Open the Web SSO Configuration for LTPA Token document in the Web Configurations view of your Domino Directory (Names.nsf).

1. Make sure the DNS domain is the same as what is configured in WebSphere and what you are entering in the browser to access the server.

To check the DNS Domain in WebSphere Portal v4.x:
1. Open the WebSphere Administration Console.
2. Go to Console -- Security Center.

On the Authentication Tab, Domain should be the same as DNS Domain above, with the exception of a leading period. Domino's file will contain the leading period, whereas WebSphere's will not.

To Check the DNS Domain in WebSphere Portal v5.x:
1. Open the WebSphere Administration Console.
2. Click on Security - Authentication Mechanisms.
3. In additional Properties, click on Single Sign-on (SSO).

To Check the DNS Domain in WebSphere Portal v6.x:
1. Open the WebSphere Administration Console.
2. Click on Security -> Global Security -> Under Authentication Mechanisms -> Authentication Mechanisms
3. Select LTPA -> SSO

Note: The Domain Name field should be the same as the DNS Domain above, with the exception of a leading period. Domino's file will contain the leading period, but WebSphere's will not.

Note: If the users are on a Domino 6.x server, then it is not necessary to place the '\' before the port number.

2. Make sure the Domino server name contains the Domino canonical name of the server that you are testing SSO with.

3. Make sure the Realm set by WebSphere Portal is the same as the LTPA Realm listed in the WebSSO document.

To check the Realm setting in WebSphere Portal v4 and v5.0.x:

The realm is the same as the LDAP server Portal uses to authenticate with, and is in the correct format. For example, if you configure WebSphere to use server ldap.domain.com on port 389 as the user repository, then the LDAP Realm field entry should read "ldap.domain.com\:389".

** Note: you must add the \ between before :389 for Domino 5.x Server. This is not needed on Domino 6.x Servers, but will not cause problems if it is present.

To check the Realm setting in WebSphere Portal v5.1.x:

If you enabled security with no realm support, the process did not change from earlier versions of Portal, so follow the steps for Portal v4 and 5.x.

If you enabled security with realm support (via running the configuration task enable-security-wmmur-ldap or enable-security-wmmur-db) you will need to manually synchronize the Realm values in Portal and Domino. There are 2 different methods in which to accomplish this:

Method #1:
By default the realm value will be set to WMMRealm by Portal after running the enable-security-wmmur-ldap or enable-security-wmmur-db task. If you simply update the LDAP Realm value in the Web SSO document in Domino to WMMRealm, you will need to restart Domino for the change to take effect.

Note: The LDAP Realm value is case sensitive. For example, if you enter wmmrealm instead of WMMRealm, SSO will not work.

Method #2:
If you want to control the value of the realm set by Portal use the following steps:

1. Set the userRegistryRealm value.
a. Open the Admin Console of the AdminServer.
b. Go to Security > User Registry > Custom.
c. Select Custom Properties.
d. Check if the property userRegistryRealm already exists. If yes select the property and select Update. Otherwise select New.
e. Set the userRegistryRealm name to the value of your choice.
For example <full qualified name of the LDAP Server>:<Port> as Value.
Sample value: myldapserver.myorg.com:389

f. Save your changes and restart the server.

2. Update the security.xml document.
Edit the file AppServer/config/cells/<cellname>/security.xml and make sure the property realm="<full qualified name of the LDAP Server>:<Port> " is set for the tag <userRegistries xmi:type="security:CustomUserRegistry" ...>
The sample below shows this:
<userRegistries xmi:type="security:CustomUserRegistry" xmi:id="CustomUserRegistry_1" serverId="uid=wpsbind,dc=users,ou=bvt,c=de,o=ibm.com" serverPassword="{xor}..." ignoreCase="true" customRegistryClassName="com.ibm.websphere.wmm.registry.WMMUserRegistry" realm="myldapserver.myorg.com:389">

To Check the Realm Setting in WebSphere Portal v6.x
1. Open the WebSphere Administration Console
2. Click on Security -> Global Security -> Under User Registries -> Custom
3. Select Custom Properties
4. Check if the property userRegistryRealm already exists. If yes, select the property and select Update.
5. If the property does not exist, select Update.
6. Set the userRegistryRealm to a value of your choice.
7. Save your changes and restart the server.

3. Export the LTPA token from WebSphere, and import it into Domino. For more information please see the following technote:

"Single-Sign-On issues between WebSphere Portal and other applications (e.g. Lotus Domino or Sametime) within the same Single-Sign-On domain" (# 1198736)


B. Make sure Multi-Server SSO is enabled and loads properly.

1. Open the Server document you are testing and select the Internet Protocols tab > Domino Web Engine tab. Make sure "Multi-server" is selected in the Session Authentication field.
2. When the HTTP task loads on Domino you should see one of the following messages, depending on the server you are running:

Domino v5: HTTP: Successfully loaded Web SSO Configuration
Domino v6: HTTP Server: No Web SSO Configuration specified, using default ('LtpaToken').

C. If you continue to have SSO problems, add the following to the Notes.ini on the Domino Server (located in the Domino program directory).

debug_sso_trace_level=2
websess_verbose_trace=1
debug_outfile=c:\outfile.txt

Then restart the Domino Server and sign into Portal. Change the browser to http://qpserver.domain.com/database.nsf and send the resulting Outfile.txt to Lotus Technical Support for review.

NOTE: If you make any changes to WebSphere security settings, you need to export and reimport the WebSphere LTPA key, as discussed in the document cited above, # 1098010.

NOTE: If you make any changes to the Domino Web SSO configuration for LTPA Token document, then you need to restart the Domino server for the changes to take effect.

---

III. Additional steps to take if the Server is a QuickPlace Server:

A. Follow the steps to configure SSO on a QuickPlace server, found in Chapter 6 of the QuickPlace Administrator's Guide.

1. Add the following settings to the server Notes.ini file:

NoWebFileSystemACLs=1
h_ScopeUrlInQP=1

2. Enable Multi-server Session Authentication.

a. From a Notes Client, open the Domino Directory (Names.nsf) on the QuickPlace server.
b. In the Server > Servers view, open the QuickPlace server document.
c. Click on the Internet Protocols tab.
d. Click on the Domino Web Engine tab.
e. Next to Session Authentication, select "Multi-server".

3. Create or customize an existing Domino Web Server Configuration database.

For testing purposes, if you have an existing Domino Web Server Configuration database (domcfg.nsf) please remove it from your Lotus directory (it will be found in the Domino \Data directory) and create a new Domino Web Server Configuration database by following the steps below:

a. Create a database from the Domino Web Server Configuration (5.0) template and give it the file name "domcfg.nsf".
b. Open the new database.
c. Select Create > Mapping a Login Form.
d. In the 'Target Database File Name' field, enter QuickPlace/resources.nsf.
e. In the 'Target Form Name' field, enter QuickPlaceLoginForm.
f. Save the new form.
Once the problem is resolved, and you would like to return to using your custom Domino Web Server Configuration database, you can do so by working with the original designers of that database to do the following:

NOTE: This step is needed only if you want to return to a custom Domino Web Server Configuration database. And it should be done only after you know SSO works with the above database to assist in troubleshooting any problems that your customizations may cause.

a. From Domino Designer, open quickplace/resources.nsf.
b. Open the QuickPlaceLoginForm.
c. Copy the <Computed Value> field from this form to the login form in domcfg.nsf.

B. Modify the qpconfig.xml for nonstandard distinguished names (DNs).

For example, if the dn for users is: uid=tuser,cn=users,dc=acme,dc=com. , the key part to this is the cn=users. Domino names do not use "cn" after the person's common name. Modify the user_directory of the qpconfig.xml, and add the following to the schema section:

<secondary_cn_component enabled="true"/>

If the dn contains a space, modify the user_directory of the qpconfig.xml, and add the following to the schema section:

<dn_delimiter>,@</dn_delimiter>
<dn_delimiter robust_compare="true"/>

Example of the dn containing a space, between "ou=people" and "dc=com":

uid=tuser,ou=people, dc=acme,dc=com

For more information on these, please refer to the qpconfig_sample.xml and the QuickPlace Administration guide.

C. If you continue to have SSO problems, add the following to the Notes.ini file on the QuickPlace server (located in the Domino program directory)
debug_sso_trace_level=2
websess_verbose_trace=1
quickplaceauthenticationlogging=5
debug_outfile=c:\outfile.txt

Then restart the Domino Server, and sign into Portal. Change the browser to http://qpserver.domain.com/quickplace and send the resulting Outfile.txt to Lotus Technical Support for review.

NOTE: If you make any changes to WebSphere security settings, then you need to export and reimport the WebSphere LTPA key, as discussed in the document cited above, (# 1098010).

NOTE: If you make any changes to the Domino Web SSO configuration for the LTPA Token document, then you need to restart the Domino server for the changes to take effect.

NOTE: For QuickPlace 6.5.x, the WebSSO configuration document must be called "LTPAToken". In QuickPlace 7.0 or later, you can name it anything you want.

---

IV. Additional Steps for a Sametime server:

A. Does the LDAP server used for Sametime require a bind user to authenticate?

If so, add the bind user to the directory assistance document for the LDAP directory.
1. Open the Directory Assistance database (usually called "da.nsf") on the Sametime server.
2. Open the document for your Sametime LDAP server.
3. On the LDAP tab set the following fields:

Username field: Enter in the user's distinguished name as it appears in the LDAP directory.
Password field: Add this user's password.
Base DN field: Add the base dn you entered in Portal to perform searches.

Note: Also make sure the Port field is correct for your LDAP server.

B. If you continue to have SSO problems, add the following to the Notes.ini on the Domino server (located in the Domino program directory)

debug_sso_trace_level=2
websess_verbose_trace=1
debug_outfile=c:\outfile.txt

Then restart the Domino Server and sign into Portal. Change the browser to http://qpserver.domain.com/stcenter.nsf and send the resulting Outfile.txt to Lotus Technical Support for review.

NOTE: If you make any changes to WebSphere security settings, you need to export and reimport the WebSphere LTPA key, as discussed in the document cited above (# 1098010).

NOTE: If you make any changes to the Domino Web SSO configuration for LTPA Token document, you need to restart the Domino server for the changes to take effect.

NOTE: If you name the WebSSO configuration document something other than LTPAToken (for example, "MyLtpaToken") you need to add the following line to the notes.ini:

ST_TOKEN_TYPE=MyLtpaToken

Starting in 8.5 you put ST_TOKEN_TYPE in the sametime.ini, [AuthToken] section.

---

V. Additional Configuration Needed if Portal Uses a Different Directory than QuickPlace, Sametime or Domino:

A. Domino databases must authenticate with the Domino Directory they reside on. If Portal uses another LDAP server other than Domino to authenticate its users, there are two options to get Single-Sign on working with Domino and Portal.

1. Sync the username and passwords in the Domino Directory with the names Portal uses to authenticate a user.

For example, if WebSphere Portal's user directory is IBM Directory Server, and a user's dn from IDS is:

uid=wpsadmin,cn=users,dc=acme,dc=com

...then you will need to add the following to the username field of the Person document for wpsadmin in Domino:

uid=wpsadmin/cn=users/dc=acme/dc=com
wpsadmin

These should be added below the Domino canonical name, which should be the top line of the User Name filed.

2. Use Directory Assistance so that Domino can authenticate with the external LDAP user directory.

For more information on creating and configuring Directory Assistance see the IBM Lotus Domino Administrator help database. The Domino Administrator help database can be found on developerWorks: Lotus.

• Extend LDAP Schema by adding the following attributes or use an attribute that is already available:
  
  NotesDN=CN=Test User1,O=ACME
  
  *****Must match attribute name defined in Directory Assistance*******
• Use Directory Assistance on all the Domino servers and point it to your LDAP Directory (same as Portal is using). On the LDAP tab you add the LDAP attribute that contains your Notes Canonical name. You use this to solve the multiple identity issue, as your Notes Name is used for everything while you are connected to the Domino Server, and you get access to your mail database without modifying the ACL.


B. If QuickPlace authenticates with a Domino LDAP and Portal uses another LDAP server to authenticate, perform the steps in the following technote to correctly configure the environment:

"How to configure SSO between WebSphere Portal and QuickPlace when each use a different LDAP directory" (# 1205905)

C. If Sametime authenticates with Domino LDAP and Portal uses another LDAP server to authenticate, perform the steps in the following technote to correctly configure the environment:

"How to configure SSO between WebSphere Portal and Lotus Sametime when each use a different LDAP directory" (# 1205909)

D. If Sametime authenticates with Native Domino and Portal uses another LDAP server to authenticate, perform the steps in the following technote to correctly configure the environment:

"Configuring SSO between WebSphere Portal and Lotus Sametime when each use a different user directory" (# 1231292)

---

VI. Other Possible Known Problems and Configurations

1. For multi-server Single Sign-on to work on a Domino Directory, the Domino server name cannot contain any underscores or other special characters. This is a limitation caused by a Microsoft Internet Explorer (IE) security patch.

There should not be any user names with the same hierarchy as the Domino name. For example, if the Domino server name is domsrv/acme there should not be any Person documents with the first line of the Username field set to domsrv/acme.

---