What is the new NOTES.INI parameter MAX_CONCURRENT_DBDIR_SCANS and how should it be used?
There are certain normal operations that Domino servers perform which scan the entire Domino data directory. However, when the load of this type of activity is excessive, Domino servers can experience high disk activity, high CPU, and long-running operations making connections to the server difficult or impossible. If administrators gather stack dumps/NSD, they would see the following type of procedures listed in multiple threads:
NOTE: These procedures are not listed in any order. To see complete call stacks, see 1426125, 1495812, or 1299812.
This issue has been reported to IBM Lotus Quality Engineering as SPR MLEE88NL2U and is fixed in 8.5.2 FP3, 8.5.3, and above.
Fix details: SPR MLEE88NL2U
Refer to the Upgrade Central site for details on upgrading Notes/Domino.
The fix for SPR MLEE88NL2U was to allow Domino administrators to control the high cost directory scans by adding new parameters in Lotus Domino. Those parameters are:
By default, MAX_CONCURRENT_DBDIR_SCANS is 50.
By default, CONCURRENT_DBDIR_SCAN_RETRY is 0.
What does this mean for your Domino server.
Most Domino servers (with a single network port enabled) have 40 threads in the Server thread pool. If you do not set the above parameters, then all 40 threads could be tied up doing high cost directory scans driving up CPU.
By setting MAX_CONCURRENT_DBDIR_SCANS, you can control how many threads can be doing directory scans at the same time.
NOTE: Development does not recommend setting MAX_CONCURRENT_DBDIR_SCANS to a value lower than 5.
CONCURRENT_DBDIR_SCAN_RETRY controls what happens to a thread if it requests a directory scan operation, but the maximum concurrency limit has been reached. By default, CONCURRENT_DBDIR_SCAN_RETRY is set to 0. This means that if a thread requests a directory scan operation and the maximum number of threads are already engaged in directory scans, the newly requesting thread gets an error that the server is not responding or unavailable.
CONCURRENT_DBDIR_SCAN_RETRY waits 0.5 seconds before trying the directory scan request again. Therefore, if CONCURRENT_DBDIR_SCAN_RETRY is set to 20, the thread will retry 20 times for a total possible wait time of 10 seconds.
NOTE: If you choose to set CONCURRENT_DBDIR_SCAN_RETRY, development recommends setting the value to 5 or less.
Administrators need to be aware that setting these parameters incorrectly can negatively impact their Domino server.
If you set MAX_CONCURRENT_DBDIR_SCANS too high, the CPU and/or the disk activity may be heavier than the hardware can handle.
If you set MAX_CONCURRENT_DBDIR_SCANS too low and CONCURRENT_DBDIR_SCAN_RETRY is not set, then you may get reports of background processing or administrative tool failure due to the server not responding.
If you set CONCURRENT_DBDIR_SCAN_RETRY too high and MAX_CONCURRENT_DBDIR_SCANS too low, then you could have situations where users report slow response to client requests due to threads being tied up waiting for the directory scan retry attempts.