IBM Support

Troubleshooting Windows single sign-on for Web clients (SPNEGO)

Technote (troubleshooting)


Problem

Users are experiencing problems using Windows single sign-on for Web clients.

Symptom

A user is prompted for a name and password or a user receives access denied messages when connecting to a Domino server configured for Windows single sign-on for web clients.


Resolving the problem

There are a variety of techniques and tools that can be used to troubleshoot issues with a Windows single sign-on for Web clients setup on Domino. This technote describes what to do in the following two problem scenarios:
A user is challenged to provide a user name and password.
A user receives an Access Denied message.

A user is challenged to provide a user name and password

Step 1. A web user enters a URL in a browser to connect to a Domino server participating in Windows single sign-on. For example, the user enters the following URL:
http://www.renovations.com/names.nsf

If the user is prompted to log into the server, start by investigating the following:

  • First consider whether the client is in a non-supported configuration (if so, then Windows single sign-on cannot be expected to work). For more information, refer to the following topic in the Notes/Domino 8 Information Center: Setting up Windows single sign-on for Web clients. For supported browsers, also see the Notes 8.5.1 System Requirements.
  • Check the user's browser configuration to ensure that it is set up for Windows single sign-on to the Domino server. For more information, refer to the following topic in the Notes/Domino 8 Information Center: Configuring Web client browsers for Windows single sign-on.
  • Turn on debugging at the Domino server, by adding the following notes.ini setting: DEBUG_HTTP_SERVER_SPNEGO = 1
  • If desired, collect your debug output from the Domino server into a file by adding the similar notes.ini setting: debug_outfile=c:\tmp\notes.log
  • Repeat the step to bring up a browser and enter the Domino URL.
  • When the user is prompted to log in, check the server console log for errors. A very common error is "Attempt by HTTP client to authenticate using Windows NTLM security is not supported." An NTLM security error indicates that Windows single sign-on is not functioning. Assuming that the client has a supported configuration including a proper browser configuration, there may be a Windows Kerberos configuration error. If Domino error messages do not provide sufficient information to understand the problem, continue troubleshooting.

Step 2 . If your Domino server is properly configured for Windows single sign-on, you should see evidence that the server is participating in the SPNEGO protocol.

You can investigate by doing the following:
  • Turn on advanced debugging at the Domino server,by using the following notes.ini setting: DEBUG_HTTP_SERVER_SPNEGO = 4
  • Repeat the step to bring up a browser and enter the Domino URL.
  • When the user is prompted to log in, check the server console log. When contacting a Domino server that is properly configured for Windows single sign-on, a debug statement similar to this is in the console log:
      Negotiate - properly configured HTTP client should send an Authorization: Negotiate header containing SPNEGO token when repeating the request /names.nsf
  • If there is no SPNEGO negotiate message, there may be a Domino server configuration problem. Check that your server is properly configured for SSO in the applicable Internet site document (or in the Server document if Internet sites configurations are disabled). If you suspect a Domino server configuration issue, it is a good idea to first verify that basic SSO is functional when you do not have Windows single sign-on enabled. Once you verify that SSO is functional without Windows single sign-on enabled, you should check that Windows single sign-on is properly enabled in your SSO configuration document. For information on preparing the SSO configuration for Windows single sign-on, refer to the following topic in the Notes/Domino 8 Information Center: Preparing a Domino server for Windows single sign-on for Web clients. For general information on configuring SSO, refer to the following topic: Multi-server session-based authentication (single sign-on).
  • If there is a SPNEGO negotiate message, check the server console log for further error. If the error message does not provide sufficient information to understand the problem, continue troubleshooting.

Step 3. If your Domino server is properly configured for Windows single sign-on and is participating in the SPNEGO protocol, Domino should be attempting to authenticate the user by SPNEGO Kerberos.

You can investigate by doing the following:
  • Turn on advanced debugging at the Domino server, by adding the following notes.ini setting, if not already set: DEBUG_HTTP_SERVER_SPNEGO = 4
  • Repeat the step to bring up a browser and enter the Domino URL.
  • When the user is prompted to log in, check the server console log. You will see log statements similar to these:
      SPNEGO> Starting SPNEGO Negotiate - properly configured HTTP client should send an Authorization: Negotiate header containing SPNEGO token when repeating the request /names.nsf
      SPNEGO> Security token format received is SPNEGO NegTokenInit
  • If there is no indication that Domino is receiving a SPNEGO NegTokenInit, this likely indicates a Windows Kerberos configuration problem.
  • If the console log shows that Domino is receiving a SPNEGO NegTokenInit, you can collect more detailed information about Windows Kerberos operations by changing the debug spnego notes.ini setting to 5 and then repeating the browser access of the Domino URL. The additional logging to the server console shows native Windows calls being made by Domino, and can shed light on whether a particular Windows call is causing a problem. DEBUG_HTTP_SERVER_SPNEGO = 5
  • If Domino error messages do not provide sufficient information to understand the problem, continue troubleshooting.

Step 4. If there is a likely Windows Kerberos configuration problem, first check the Windows Active Directory configuration
  • For your Active Directory domain, your Windows administrator must have set the domain functional level to Windows Server 2003 or higher. Backwards compatible modes for Windows Server 2003 cannot be used (e.g. Windows Server 2003 set to use Windows 2000 mixed mode).
  • To check the domain functional level, do the following:
      1. Open the Active Directory Users and computers utility.
      2. Right-click the domain, and then click Properties.
      3. The information is located on the General tab.
  • Computers participating in Windows single sign-on must have clocks synchronized since Kerberos operations are sensitive to clock skew. You should correct clock skew whenever possible. If necessary, the Windows administrator can perform the following steps to increase the tolerance for clock skew:
    1. Launch the Group Policy Management console. (For example, on Windows Server 2008, click Start > Run, type gpmc.msc and click OK.)
    2. In the console tree, find your domain and expand the tree.
    3. Expand the Group policy objects.
    4. Under the Group policy objects, click Default Domain policy (which then appears at the right). On the Default Domain policy Settings tab, click Security settings, Account policies/Kerberos policy.
    5. Double-click Maximum tolerance for computer clock synchronization to edit the setting.

Step 5. While optional, it is often very helpful to use additional troubleshooting tools to investigate problems with Kerberos and HTTP activity. Additional information will help to determine next steps for troubleshooting.

For understanding where additional tools can be helpful, note the initial steps to accomplish Windows single sign-on:
  • the client browser extracts the DNS name contained in the Domino URL
  • the client browser constructs a Service Principal Name (SPN) from the DNS name
  • the client browser requests a Kerberos service ticket for the SPN from Active Directory
  • the client browser sends the Kerberos service ticket to the Domino server.
You can investigate whether the browser acquires the Kerberos service ticket for Domino, and passes the ticket to Domino.

For example, the browser constructs the following SPN:
HTTP/www.renovations.com@AD.EAST.RENOVATIONS.COM
The DNS name is www.renovations.com.
The Active Directory domain that the Domino server's machine belongs to is AD.EAST.RENOVATIONS.COM.
For more information on SPNs, refer to the following topic in the Notes/Domino 8 Information Center: Setting up the Windows service for Domino.
To use tools to investigate:
  • Download and install the Windows Server 2003 resource kit tools. You can use one of these tools:
  • Klist.exe. Kerberos List is a command-line tool that is used to view and delete Kerberos tickets granted to the current user's logon session.
  • Kerbtray.exe. Kerberos Tray is a graphical user interface tool that is used to view and delete (purge) Kerberos tickets granted to the current user's logon session. You can view and delete (purge) the Kerberos ticket cache by right-clicking on the Kerberos Tray tool icon located in the notification area of the desktop.
  • After downloading and installing, use the resource kit tool (Kerbtray) to purge the user's current list of Kerberos tickets.
  • Repeat the step to bring up a browser and enter the Domino URL.
  • Use the resource kit tool to view the user's Kerberos tickets.
  • If there is no Kerberos ticket for the expected Domino SPN, there is likely a Kerberos configuration issue to resolve, therefore continue troubleshooting.
  • If there is a Kerberos ticket for the expected Domino SPN, you can optionally use a network protocol analyzer such as the open source tool Wireshark to collect data from the live network and to verify that the HTTP negotiation includes the Kerberos ticket being sent by the client to the Domino server. A trace of HTTP activity for a functional Windows single sign-on would include HTTP commands similar to these:
      client: get /names.nsf
      Domino server: 401 unauthorized (WWW-Authenticate : Negotiate)
      client: get /names.nsf Authorization : Negotiate (Kerberos ticket)

Step 6. If the client browser did not acquire a Kerberos ticket to the Domino server, consider whether the user and the Domino server are located in the same Active Directory domain, or are in different domains. If the user and Domino server are located in separate Active Directory domains (are registered in different Active Directories), the problem may be in your cross-domain set up.
  • Review the list of requirements for Windows single sign-on to work across Active Directory domains. Verify that your setup adheres to all the requirements, particularly that the Windows trust between Active Directory domains is set up as a Forest trust. For more information, refer to the following topic in the Notes/Domino 8 Information Center: Windows single sign-on for Web clients across multiple Active Directory domains.

Step 7. There may be a problem with the SPN in Active Directory which you should suspect if:
  • the client browser did not acquire a Kerberos ticket to the Domino server, or
  • the client browser did acquire a Kerberos ticket to the Domino server but a tool such as Wireshark reports a Kerberos error (for example, KRB5_KRB_AP_ERR_MODIFIED)

Your Active Directory administrator can investigate the Windows SPN in Active Directory using the Domino ldapsearch command line tool. For more information on configuring SPNs, refer to the following topic in the Notes/ Domino 8 Information Center: Setting up the Windows service for Domino.

Use this syntax to research your SPN:

ldapsearch -h <domain controller name> -D <administrator distinguished name> -w <administrator password> -b <starting point for the search> "serviceprincipalname=<YourSPN>" dn  
For example, to research your SPN HTTP/ www.renovations.com :

ldapsearch -h server1.ad.east.renovations.com -D "cn=administrator,cn=users,dc=ad,dc=east,dc=renovations,dc=com" -w myAdminstratorPw -b "dc=ad,dc=east,dc=renovations,dc=com" "serviceprincipalname= HTTP/www.renovations.com " dn

Problem A: SPN is mistakenly registered to two or more different accounts.
The result of the ldapsearch should return the distinguished name of the account to which the SPN is registered. A Windows SPN can only be owned by one Windows account. Your Windows single sign-on will not work if the SPN is mistakenly registered to two or more different accounts. If your Active Directory administrator finds that the ldapsearch returns more than one instance of your SPN, then the Windows administrator must remove the SPN from all accounts except for one. This can be done using the setspn.exe command and the -d parameter. For example, suppose your Windows administrator finds that your SPN has been registered to the Windows account server2 and also to the Windows account dominoacct. Your Windows administrator can remove the SPN from the dominoacct account with this command:
setspn -d HTTP/www.renovations.com dominoacct

Problem B: The SPN is registered to an account which is different from the account under which the Domino server Windows service is running.

Note that if you uninstall Domino server and then you reinstall it, the Domino Windows service is set by default to run as "Local system". Any time that you reinstall Domino, you must check that the Domino Windows service is configured to run under the proper account.

Usually the proper course of action is to change the Domino server Windows service to run under the account that owns the SPN. However, if the SPN was created for the wrong account, your Windows administrator should remove the SPN and create a new SPN that is owned by the expected account.

If you determine that the SPN is correctly registered to the appropriate Windows account, verify that the Domino server is running under the correct account that can accept and process the Kerberos ticket. To determine which account your Domino Windows service is running, do the following:
  • From the Windows Control Panel, click Administrative Tools - Services.
  • Locate the Lotus Domino Server entry and verify that the "Logon as" column displays the correct account name. If you have assigned the SPN to a named account such as "dominoacct", the named account should be shown. If you define the SPN for the default account associated with the Domino server computer ("domino1"), the "Local system" should be shown.

Problem C: The client browser did not acquire a Kerberos ticket to the Domino server, and your server name is an alias.

In DNS, a CNAME (canonical name) record may be used to define an alias. In some scenarios, the client browser may use DNS to resolve a CNAME alias to a hostname when determining the SPN for which to request a Kerberos service ticket; in this case you need an SPN defined for the resolved host name that the alias represents.

To investigate your DNS settings for aliases used by your Domino server, from a command prompt you can use nslookup in interactive mode with debug settings (set d2). For example, to see DNS information including which hostname your alias, www.renovations.com resolves to:
    C:\>nslookup
    > set d2
    > www.renovations.com

For example, if the output of nslookup identifies www.renovations.com as a CNAME alias for the hostname server3.ad.east.renovations.com, research whether an SPN exists for "HTTP/server3.ad.east.renovations.com". If this SPN does not exist, your Windows administrator must to define it. Your Domino server at server3.ad.east.renovations.com must be providing the HTTP service for www.renovations.com (e.g. accomplished by an Internet site document configuration).

Step 8. If required HOST SPNs have been mistakenly altered or deleted, you will not have a functioning Windows single sign-on.

Each computer in an Active Directory domain by default has HOST SPNs assigned. A HOST SPN is separate from the HTTP SPN needed for Windows single sign-on to your Domino server. However, when the Windows administrator is editing SPNs, problems will arise if the computer's HOST SPN is accidentally altered or deleted.

Normally you will see the HOST SPNs when you execute "setspn -l" for your server, for example: setspn -l domino1

In this case you would see these HOST SPNs listed:
    HOST/DOMINO1
    HOST/domino1.ad.east.renovations.com

If your Windows administrator must reset the HOST SPNs for your server computer, the Windows administrator can use the "setspn -r", for example, setspn -r domino1.

Step 9. When the Kerberos authentication portion of Windows single sign-on has succeeded, you will see an indication in your Domino server console log, if SPNEGO debug logging is set to level 2 or higher.
  • Turn on informational debugging at the Domino server by adding the following notes.ini setting, if not already set to a higher level: DEBUG_HTTP_SERVER_SPNEGO = 2
A Kerberos authentication success message is similar to this:
SPNEGO> User jdoe@ AD.EAST.RENOVATIONS.COM  authenticated by Kerberos service  HTTP/www.renovations.com@AD.EAST.RENOVATIONS.COM

When Kerberos authentication succeeds, usually Windows single sign-on will also succeed. An error occurring after Kerberos authentication succeeds often causes an Access denied message to be displayed, rather than a prompt for the user to provide a user name and password. See next scenario for access denied scenarios.

A user receives an Access Denied message.

Step 1. A web user enters a URL in a browser to connect to a Domino server participating in Windows single sign-on. For example, the user enters the following URL:
http://www.renovations.com/names.nsf

If the user is denied access, start by investigating the following:
  • If the user has a Mozilla / Firefox browser, the problem may be that the client is attempting to access a Domino server configured for Windows single sign-on while the client is located on the Internet, or not logged into the Active Directory domain.
    To address this problem, provide your Mozilla/Firefox user with an alternate URL for accessing Domino in any scenario in which Windows single sign-on is not supported. The alternate URL should be set to prompt the user to login to Domino using a user name and password. Having an alternate URL may require setting up a Domino Internet site with a security configuration that disables Windows single sign-on. For more information, refer to the following topic in the Notes/Domino 8 Information Center: Setting up separate Web sites for participating and non-participating Web clients.
  • If the user has a Mozilla / Firefox browser and can't access Domino in a setting where Windows single sign-on should be supported, check the user's browser configuration to ensure that it is set up for Windows single sign-on to the Domino server. For more information, refer to the following topic: Configuring Web client browsers for Windows single sign-on.
  • Regardless of browser type, check that the user's name (or group name) is listed on the Domino Access Control List and is granted access rights. If so, continue troubleshooting.

Step 2. You can follow steps described in the previous scenario (user is challenged to provide user name and password) in order to determine whether the Domino server has received a Kerberos service ticket for the user.
  • If the client browser does not acquire a Kerberos ticket and furthermore if there is no evidence of HTTP activity at the Domino server when the user browses to the Domino URL, there is likely a DNS issue for the client to resolve the Domino server DNS name. Your network administrator should help you to diagnose the DNS problem.

Step 3. Check for an indication of Kerberos authentication in your Domino server console log, if logging is set to level 2 or higher.
  • Turn on informational debugging at the Domino server, by adding the following notes.ini setting, if not already set to a higher level: DEBUG_HTTP_SERVER_SPNEGO = 2
  • If desired, collect your debug output from the Domino server into a file, by adding the similar notes.ini setting: debug_outfile=c:\tmp\notes.log
  • Repeat the step to bring up a browser and enter the Domino URL.
    A Kerberos authentication success message in the server console log is similar to this:
    SPNEGO> User jdoe@ AD.EAST.RENOVATIONS.COM  authenticated by Kerberos service  HT TP/www.renovations.com@AD.EAST.RENOVATIONS.COM

Step 4. If the Kerberos authentication portion of Windows single sign-on has succeeded, the Domino server may be having difficulty determining the user's corresponding Domino name, and therefore cannot match the user to a person or group listed on the Domino Access Control List.
  • If name mapping has failed, you will see a Kerberos name in the session login debug message if you enable this server notes.ini variable:
    WEBSESS_VERBOSE_TRACE = 1
    Note: The server needs to be restarted to have this notes.ini variable work
  • Repeat the step to bring up a browser and enter the Domino URL.

If Domino knows only the user's Kerberos name, a session login debug message may be similar to this:
    SessionManager::LoginUser HTTP Sessions> Logging in user ( jdoe@ AD.EAST.RENOVATIONS.COM )
[
Step 5. If the Kerberos authentication portion of Windows single sign-on has succeeded, a failure to map the user's Kerberos name to a Domino name may be the result of consistency errors in the directory records associated with the user's Kerberos name. You can collect more detailed information about name mapping.
  • Set the following server notes.ini variables:
    LOGLEVEL_NAME_MAPPING=1
    WIDE_SEARCH_FOR_KERBEROS_NAMES=1
    WEBAUTH_VERBOSE_TRACE = 1
    Note: The server needs to be restarted to have these notes.ini variables work
  • Repeat the step to bring up a browser and enter the Domino URL.
Look in the server console log for name mapping errors.

Step 6. If the user's Kerberos name is being correctly mapped to a Domino name, you might still encounter other name mapping issues related to the user's LTPA token that represents the user after Kerberos authentication is complete. You can investigate name in the user's LTPA token, to assist your diagnosis of further name mapping issues.
  • Set the following server notes.ini variable: DEBUG_SSO_TRACE_LEVEL = 1
  • Repeat the step to bring up a browser and enter the Domino URL.
Look in the server console log for SSO API messages showing the user's SSO information.

Related information

A simplified Chinese translation is available

Document information

More support for: IBM Domino
Web Server

Software version: 8.5.1, 9.0, 9.0.1

Operating system(s): Windows

Software edition: Edition Independent

Reference #: 1394592

Modified date: 06 August 2015