Technote (troubleshooting)
Problem
A Domino Traveler server runs for an undetermined amount of time, then goes into a Red state. A "tell traveler status show" command results in a Red status, and is actually a hung thread.
Red Status Messages
User null on thread Alarm-316c has been running for 847 minutes.
Cause
Some subsystems in the Domino server perform searches against LDAP servers. Directory Assistance, for instance, uses and retains LDAP search results for a period of time, eliminating the need to obtain refreshed information from the LDAP servers. Domino allows these subsystems to quickly detect if entries in certain LDAP directories have changed, allowing Domino subsystems to flush stale search results and conduct another search for current LDAP information.
There are some third party LDAP servers which require additional configuration in order to support change detection, and/or the default level of access granted to the authenticated bind account used by LDAP may not have access to the third party LDAP server's change detection feature. This can cause the Traveler server thread to hang.
The poll for changes every 30 seconds can add extra load on the Domino server and the remote LDAP servers. This can cause CPU spike and slow response by Domino server.
@[ 3] 0x100362ee nnotes.OSDelayThread+50 (0,1005be70,5f8a3010,1)
@[ 4] 0x1005cb71 nnotes.OSLockSpin+493 (0,1005dd5e,61f41028,5f8a3170)
@[ 5] 0x1005cf32 nnotes.OSLockSem+82 (87980001,1005db69,2468002e,ed401)
@[ 6] 0x1151b151 nnotes.NetGetGlobalClosePending+25 (1,114c49b6,87980001,1)
@[ 7] 0x1151936a nnotes.FastNetAbortCheck+130 (87980001,114911b0,87980001,0)
@[ 8] 0x11490ff7 nnotes.CNetSession::MarkSessionInUse+707 (0,2468002e,61f41218,44)
@[ 9] 0x1148ceed nnotes.CNetSession::ReceiveAsync+237 (98c62a8,98c62a8,98c62a8,11d12289)
@[10] 0x110b17d9 nnotes.read1msg+553 (98c62a8,98c71a8,2,0)
@[11] 0x110b10cf nnotes.ND_ldap_result+695 (27840da8,0,0,5f8a34f0)
@[12] 0x11098087 nnotes.LDAPRootDSESearch+163 (8,5f8a34f0,0,0)
@[13] 0x110bc4c3 nnotes.CLDAPChangeBase::Detect+91 (102,10036a1a,0,1005be70)
@[14] 0x1107b8d0 nnotes.CLDAPChange::Detect+288 (1c,11dcbc0e,0,5f8a3a28)
@[15] 0x11da40bc nnotes.LDAPGatewayGetLastModifiedTime+516 (0,0,0,0)
@[16] 0x11d92a91 nnotes.GetNSModifiedTime+541 (15c8,100309dd,9,9a28001f)
@[17] 0x11d9170e nnotes.NAMEGetNSModifiedTimeExtended+1138 (0,48fa934a,5f8a4080,5f8a3c70)
@[18] 0x11e7524e nnotes.NAMELookupCacheImplementation2::CacheGetContainerModifiedTime+402 (0,0,0,10036a1a)
@[19] 0x11e75d24 nnotes.NAMELookupCacheImplementation2::CacheValidateContainer+1888 (0,14e09,0,0)
@[20] 0x11e2eba1 nnotes.gbiCache::FindView+417 (0,0,4c45828,0)
@[21] 0x11e2bc7b nnotes.gbiCache::CacheLookupContainer+367 (0,0,0,0)
@[22] 0x11dc223b nnotes.NAMELookupCache2LookupContainer+531 (4,4c45828,410e36c8,2000008e)
@[23] 0x11dc5ffe nnotes.NAMELookupCacheLookupContainer+138 (e5c,72a9a9,0,10036a1a)
@[24] 0x11d91937 nnotes.NAMEGetNSModifiedTimeExtended+1691 (0,5f8a5554,4,5f8a4d10)
@[25] 0x11d0a272 nnotes.NAMESwitchToNLThread+678 (0,1005c9a5,2,1)
@[26] 0x11d91543 nnotes.NAMEGetNSModifiedTimeExtended+679 (88cf644,1005dd96,12bcb008,12bcaf40)
@[27] 0x114b646f nnotes.GetNameModifiedTime+111 (0,11500001,17cbb500,0)
@[28] 0x114b5db5 nnotes.FindNameInGroupCache+49 (e24d030,1123e701,19342ad0,17)
@[29] 0x114b3bef nnotes.UAFGetAccessGroups+223 (FFFFFFFFFFFFFFE,28,0,27841c40)
@[30] 0x22b0d607 ninotes.UserInfo::AddGroupsFromDirectory+27 (FFFFFFFFFFFFFFE,1,27841828,0)
@[31] 0x22b0d470 ninotes.UserInfo::GetGroupList+680 (0,2784a058,5f8a74b0,22dd4dad)
@[32] 0x22b0d05f ninotes.AuthUser::SetAuthentic+463 (0,111d4ff7,4c30f50,1005c9a5)
@[33] 0x22b116e6 ninotes.AuthUser::Init+1198 (20006,10037a51,153d90e0,0)
@[34] 0x22d91743 ninotes.ReqContext::InitUserInternal+923 (2784a028,0,5f8a9168,ff)
@[35] 0x22d91351 ninotes.ReqContext::InitUser+177 (0,1002d1bc,27,0)
@[36] 0x22b885e3 ninotes.HAPICmdHandler::OpenDb+191 (1616,15c8,5f8a9168,0)
@[37] 0x22b86d86 ninotes.CmdHandlerBase::SetupCmdDb+18 (7e20ab0,173c,7e1fa10,5f8a9168)
@[38] 0x22d593e5 ninotes.CmdHandlerBase::HandleOpenFieldCmd+125 (c39,361bcbf8,4,5f8ae7e4)
@[39] 0x22b86581 ninotes.CmdHandlerBase::PrivHandle+661 (27840c50,5f8aee40,2048,11758f1f)
@[40] 0x22b87f97 ninotes.HAPICmdHandler::Handler+75 (27844028,27840c50,FFFFFFFFFFFFFFE,27848028)
@[41] 0x22b6b27d ninotes.Cmd::ExecuteHAPI+17 (27841228,27840c50,361bcbf8,22d883a5)
@[42] 0x22bddfe1 ninotes.HAPI_CmdHelperNormal::ExecuteCmd+53 (27848028,27840c50,60e506cc,3)
@[43] 0x22bdeecc ninotes.HAPI::ProcessCmd+416 (9f5a1f0,0,22978ddc,1002d1bc)
@[44] 0x22bdfcd0 ninotes.HAPI::ConvertItem+500 (20000501,0,80020072,13f)
@[45] 0x22c05b26 ninotes.iHTMLConvertItem+78 (60e506a8,0,5f8aed00,100)
@[46] 0x11f0c7b5 nnotes.HTMLConvertItem+77 (22afe720,0,18ddfe68,20)
Resolving the problem
To resolve the issue, LDAP Change detection can be disabled on the Domino server where Traveler is running. Enter the following line into the notes.ini and then restart Domino:
DisableLdapChangeDetection=1
Additional Information on Change Detection considerations for Domino:
http://publib.boulder.ibm.com/infocenter/domhelp/v8r0/index.jsp?topic=%2Fcom.ibm.help.domino.admin.doc%2FDOC%2FH_SPECIAL_CONSIDERATIONS_FOR_CHANGE_DETECTION_7432_REFERENCE.html
Rate this page:
Copyright and trademark information
IBM, the IBM logo and ibm.com are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml.