Solving notification buffer overrun situations during JBB on Windows client

Resolving The Problem

The following set of messages can be found in jbberror.log:


<DATE> <TIME> ReadFsChangesThread(tid 24708): ReadDirectoryChangesW: Unable to process change notification buffer, the buffer size may be need to be adjusted to avoid this problem in the future.
<DATE> <TIME> ReadFsChangesThread(tid 24708): exiting thread.
<DATE> <TIME> psFsMonitorThread(tid 18064): Notification buffer overrun for monitored FS '<drive letter>', journal will be reset.
<DATE> <TIME> jnlDbCntrl(): Resetting journal for fs '<drive letter>'.

In general the tuning of the journal notification buffer size for files and directories could be some trouble if this computer runs out of system page pool memory or close to it. So as 32-bit Windows has the limited system page pool and Windows API allocates the notification buffers in system page pool, in some moment if the buffer has large requested size, Windows could be not able to allocate these buffers. So, from this point, it looks more reasonable to keep the notification buffers as small as possible. From the other point, the buffers should be large enough to keep all changes that occur on filesystem between the changes read. This looks as the main trick to choose the buffers enough small for available system page pool memory and large enough for filesystem activity.

Very often, if the filesystem activity is not very high, it could be enough to have e.g. 4 KB buffers.
If the journal is stable with the current notification buffers, it doesn't look reasonable to change them. If in some moments the 'buffer overflows' occur, you should most probably reduce the notification buffers, especially if they are at 1 MB or more.

Even though the "Tivoli Storage Manager (TSM) Journal Based Backup FAQ" recommends to increase the notification buffer size, it is not applicable in all cases.

Therefore the recommendation of the Tivoli Storage Manager support is to adjust the notification buffer so that the buffer overflow does not occur, increasing or (in most cases) DECREASING the notification buffer size.

As a starting point you should set the following values:

NotifyBufferSize=0x00001000
DirNotifyBufferSize=0x00001000

Note: You can specify these parameters separately for each local drive letter on your Windows client. There are dedicated stanzas in tsmjbbd.ini file for that. Please also note that there is no value for these parameters which will fit to all environments. You have to monitor the logs and adjust the parameters according to your needs.

There was a defect in Microsoft ReadDirectoryChangesW API description and behavior, influencing Tivoli Storage Manager Journal Based Backup buffers as well, so make sure to have latest MS hotfixes installed.