Skip to main content

Software  >  Lotus  >  

Domino Server hangs with error "Unable to create Server task: Insufficient memory - server executive pool is full"

 Technote (troubleshooting)
 
 
Problem

Your Lotus Domino server hangs intermittently at a rate of about once every 2-3 days after seeing the following error on the Domino Server console:


    "Unable to create Server task: Insufficient memory - server executive pool is full"
 
Symptom


An initial NSD will show the following memory usage for shared memory:

<@@ ------ Notes Memory -> Usage Summary -> Top 10 Memory Block Usage ->
16 bit Handles By Size :: (Shared) (Time 07:29:57) ------ @@>

Type TotalSize Count Typename
-----------------------------------------------------------
0x890b 134283242 2053 BLK_EXECPOOL
0x0a0a 4511458 138 ???
0x824b 3612320 60 BLK_OPENED_NOTE
0x8291 3466414 53 BLK_NSF_FOLDERPOOL
0x8a01 3139530 48 BLK_NETPOOL
0x8001 2229704 1549 ???
0x892e 1975000 1 BLK_TASK_TABLE
0x826c 1242676 19 BLK_EXTMGR
0x8141 1111868 17 BLK_STATICPOOL
0x8405 1089836 1862 BLK_IDTABLE

Notice that BLK_EXEC_POOL is at its limit of 128 MB and 2053 handles.

From the memory dumps, one sees that this block increased roughly at a rate of 3 million bytes per hour after a server restart.

This also occurs on Linux. This is an example of a Linux stack exhibiting this error:

Thread 190 (Thread 4002020256 (LWP 4420)):
#0 0xffffe405 in __kernel_vsyscall ()
#1 0xf61c4101 in __gxx_personality_v0 () from /lib/libc.so.6
#2 0xf64d0cbc in FRDoSleep ()
from /domino/lotus/notes/latest/linux/libnotes.so
#3 0xf64d1eb0 in OSRunExternalScript ()
from /domino/lotus/notes/latest/linux/libnotes.so
#4 0xf64d34ca in OSFaultCleanupExt ()
from /domino/lotus/notes/latest/linux/libnotes.so
#5 0xf64d35b4 in OSFaultCleanup ()
from /domino/lotus/notes/latest/linux/libnotes.so
#6 0xf64a4030 in fatal_error ()
from /domino/lotus/notes/latest/linux/libnotes.so
#7 <signal handler called>
#8 0xffffe405 in __kernel_vsyscall ()
#9 0xf635bfc1 in raise () from /lib/libpthread.so.0
#10 0xf64ec323 in Panic () from /domino/lotus/notes/latest/linux/libnotes.so
#11 0xf6521a9d in OSVBlockAddr ()
from /domino/lotus/notes/latest/linux/libnotes.so
#12 0x080caee7 in DequeueOpenCollection ()
#13 0x080a9daf in ServerCloseCollection ()
#14 0x080ab369 in ServerMiscFunction ()
#15 0x0809ab0b in DbServer ()
#16 0x080bb9c8 in WorkThreadTask ()
#17 0x0806f058 in Scheduler ()
#18 0xf64c8ec2 in ThreadWrapper ()
from /domino/lotus/notes/latest/linux/libnotes.so
#19 0xf63542ab in start_thread () from /lib/libpthread.so.0
#20 0xf61caa4e in clone () from /lib/libc.so.6
 
Cause
Investigated by the development team, this error was proven to be a memory leak.
 
Resolving the problem

This issue is now fixed in Lotus Notes and Domino 8.0.2 FP2, 8.5 FP1 and 8.5.1.

Fix details: SPR#TLAM7NGJXY

Refer to the Upgrade Central site for details on upgrading Notes/Domino.

For immediate relief, restart the Domino server.

If the server is unresponsive to server commands, issue the "NSD -kill" command and start the server back up to recycle the BLK_EXECPOOL memory block.

If you have not yet upgraded, as a workaround, you may add enable_collection_clean_queue=0 to the notes.ini. Then restart the Domino server.

 
 
 

Copyright and trademark information
IBM, the IBM logo and ibm.com are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml.
Rate this page
Please take a moment to complete this form to help us better serve you.
This material provides me with the information I need.




This material is clear and easy to understand.




Did the information help you to achieve your goal?
What updates, improvements, or related information would you like to see in this document?
Your response will be used to improve our document content. Requests for assistance, if applicable, should be submitted through your normal support channel as we cannot respond from this site.
Input the verification number to submit feedback:
Document information
 Product categories:
 Software
 Messaging Applications
 Advanced Messaging
 Lotus Domino
 Lotus Domino Server
 Operating system(s):
  AIX, IBM i, Linux, Solaris, Windows
 Software version:
  8.0, 8.5
 Software edition:
  All Editions
 Reference #:
  1376091
 IBM Group:
 Software Group
 Modified date:
 2009-10-11

Translate My Page
 
 

Rate this page

Help us improve this page. Your response will be used to improve our document content. Requests for assistance, if applicable, should be submitted through your normal support channel as we cannot respond from this site.