IBM Support

Tivoli Storage Manager Journal Based Backup FAQ

Question & Answer


Question

Journal Based Backup (JBB) Frequently Asked Questions (FAQ)

Cause

Tivoli Storage Manager Journal Based Backup Frequently Asked Questions

Answer


What is Journal Based Backup and how does it differ from traditional incremental backup?

Journal Based Backup is an alternate method of backup which utilizes a change journal maintained by the Tivoli Storage Manager Journal Daemon process.

The primary difference between traditional incremental backup and journal based backup is the method in which candidates for backup/expiration are obtained.

Traditional incremental backup obtains the list of backup/expiration candidates by building comprehensive lists of local objects and lists of active server objects for the file system being backed up. The local lists are obtained by scanning the entire local file system. The server list is obtained by querying the entire server inventory for all active objects. The two lists are then compared, and candidates are selected according to the following criteria:

  • An object is selected as a backup candidate if it exists in the local list but doesn't exist in the server list, or if the object exists in both lists but differs according to Tivoli Storage Manager incremental criteria (attribute changes, date/size changes, etc.).
  • An object is selected as an expiration candidate if it exists in the server list but doesn't exist in the local list.

Journal based backup obtains the candidates list of objects to backup/expire by querying the Tivoli Storage Manager Journal Service for the contents of the change journal of the file system being backed up. Change journal entries are cleared (marked as free) after they have been processed by the backup client and committed on the Tivoli Storage Manager server. Journal Backup is activated by configuring the journal daemon to monitor specified file systems for change activity and is enabled by successfully completing a full incremental backup.

Journal Based Backup is considered a viable solution for environments in which the amount of file system activity is light to moderate, and that the activity is somewhat well distributed. Running applications which touch every file (or a very large percentage of files) on the file system, or which flood the file system with changes in a very short period of time (such as copying a very large directory tree) may make journaling unusable.


What platforms and Tivoli Storage Manager client versions is Journal Backup available for?

Journal Based Backup (JBB) is supported on all Tivoli Storage Manager Windows x32, Windows x64, and AIX clients in active service.


General Recommend Usage and Limitations

Journal Backup is appropriate for backing up files systems with small or moderate amounts of change activity between backup cycles. Very large amounts file changes between backup cycles will result in very large change journals, and very large change journals pose memory and performance problems which may negate the benefits of journal backup. For example, creating, deleting, or renaming/moving very large directory trees may also negate the benefit of using journal backup instead of normal incremental backup.

Journal backup is not intended to be a complete replacement for traditional incremental backup. The goal of journal backup is to track all file system changes in the change journal and to make every attempt to keep the journal synchronized with what has been backed up. Limitations:
  • Since individual server attributes aren’t available during a journal backup, certain policy settings such copy frequency and copy mode may not be enforced.
  • Other platform specific idiosyncrasies may prevent objects from being processed properly, and other software which changes the default behavior of the file system may prevent file system changes from being detected.
  • It is possible for a small number of file system deletions to be missed when the file system is very active while a journal backup is in progress which will prevent the missed files from being expired during the journal backup.

Because of this and the stated limitation of journal backup, it is recommended that periodic full incremental backups be done along with more frequent journal backups. Traditional incremental backup compares the entire server inventory of files against the entire local file system and will therefore always be the most comprehensive backup method.


Limitations for non-Windows platforms (AIX, etc.)
  • Changes to NFS file systems by NFS clients may not be detected by a journal daemon running on a NFS file server.
  • Under some circumstances it is possible for directories to be redundantly processed during a journal backup until the directory is modified.
  • Journal backups with Open File System enabled may miss file changes generated by the pre and post snapshot commands.


What are change journals and how are they created/maintained ?

A change journal is a database of file system change information. The Tivoli Storage Manager Journal daemon creates and maintains a change journal for each file system being journaled, as specified in the Tivoli Storage Manager Journal Service configuration file tsmjbbd.ini. A change journal for a file system is activated when it is added to the list of journal file systems in the Journal daemon configuration file tsmjbbd.ini. See the Configuration section for examples.

The Tivoli Storage Manager Journal daemon monitors each journal file system for change activity and creates/updates entries in the corresponding change journal which correspond to the changes. Each change journal entry reflects the most recent change activity for a particular object so there can only be one journal entry per object.

A change journal must be initialized and validated for a particular Tivoli Storage Manager client node and server before backups of the file system will be journal based.


How are change journals initialized and validated ?

A change journal must be activated as previously described before it can be initialized and ready for use.

A traditional incremental backup of the file system must be completed while the journal for the file system is active. This backup must result in the Last Backup Completion Date being updated on the Tivoli Storage Manager server.

Once the full incremental backup has been completed, the change journal is marked as valid for the Tivoli Storage Manager client node and server the backup was performed with. Subsequent backups by the same client node to the same Tivoli Storage Manager server will be journal based provided the journal remains active and valid.

Backups by a different node and/or to a different server will not be journal based, but the change journal will remain valid to the original node and server until the change journal is deleted.


How is Journal Based Backup invoked and what indication is there that a backup is journal based ?

The backup will be use the journal when the following conditions are true:
- the journal daemon process in running
- the journal daemon is configured to monitor the file system being backed up
- all filespace integrity tests pass
- a full incremental backup has successfully completed when the above conditions are true. This backup:
- must specify a full file system (must update "Last Backup Completed " date on Tivoli Storage Manager server)
- may use different memoryefficient option settings including Disk Cache
- may be an incremental by date backup (note that this requires a previous full incremental backup as well)
- the change journal is in the valid state after the incremental backup completes

A journal based backup is started by issuing a normal backup command with a Tivoli Storage Manager backup client (command line, GUI, scheduler, or Web client) against either an entire file system or a directory. If any of these conditions isn't true, the client will revert to a traditional backup.

When a backup is journal based, the backup client will display the following messages when the journal backup begins:

"Querying Journal for '\\hostname\f$'"
Processing X Journal entries for '\\hostname\f$

Note that optimization processing by the journal daemon filters journal entries sent to the client, and the number of entries processed by the client may be much less than the actual number of entries in the journal.

Also note that journal backup uses special logic to process rename directory journal entries. When a directory is renamed, the files system reports two changes: one for the old directory which was deleted as the result of the rename, and one for the directory which was created as the result of the rename. Since changes aren’t reported for underlying files and subdirectories, journal backup processing must force a full recursive incremental backup on rename journal entries in order to ensure that all underlying files and subdirectories are backed up and expired correctly. The result is that the journal backup client output generated while processing rename entries appears as a “backup within a backup” as “Successful incremental completed” message will be displayed for each directory which is processed.


How is Journal Based Backup Installed/Enabled?

On Windows, Journal Based Backup is enabled by installing the Tivoli Storage Manager Journal Service. This may be done either by launching the Tivoli Storage Manager Journal Setup Wizard from the Tivoli Storage Manager GUI Client, or by using the Tivoli Storage Manager dsmcutil command line utility. Once the service has been installed, any file systems to journal are added to the list of journal file systems in the journal service configuration file tsmjbbd.ini. Please see the Configuration Sample section for examples. The Journal Setup Wizard is capable of installing and configuring the journal service and making basic journal configuration changes, but many more advanced settings require editing the journal configuration file directly.

On AIX, the Tivoli Storage Manager journal daemon can be configured by editing the journal daemon configuration sample file, tsmjbbd.ini.smp, and saving it as tsmjbbd.ini. Both the configuration sample file and the saved file should be in the default install directory. The journal configuration file, (tsmjbbd.ini), needs as a minimum a list of the file systems to monitor. These two lines are sufficient:
[JournaledFilesystemSettings]
JournaledFileSystems=/home

After the configuration file is created, start the journal daemon using the script file:/usr/tivoli/tsm/client/ba/bin/rc.tsmjbb. The journal will write initialization information to the errorlog. When you are satisfied that the journal is working correctly, you should run the script file, /usr/tivoli/tsm/client/ba/bin/jbbinittab. Running the script file will make entries in /etc/inittab, so that the journal will begin running when you restart your system.

On both AIX and Windows systems once the journal service has been installed and started, backups will be journal based after a journal for a particular file system has been initialized and validated as described in the conditions section above. Backups will remain journal based for a particular file system as long as the journal service is running, the journal remains valid, and the backup is being performed by the same client node to the same Tivoli Storage Manager server for which the journal is valid. Journal Based Backup is always the default backup method used, meaning that if the above conditions are met the backup will always be journal based.


Why isn't a backup journal based when the journal daemon is running?

There are a number of reasons why a backup may not be journal based, but most result from one of the conditions in the conditions section above not being met:
- the journal daemon is not correctly configured to monitor the file system being backed up. Check the tsmjbbd.ini to verify the journal file system configuration.

- all filespace integrity tests did not pass:
- The files space must exist on the server
- The last backup start and completion dates for the file space must be set on the server
- The node policy date must not have been updated since the last backup completed
- The file space must not have been deleted since the last backup completed
Note: The deletion date is set by the server whenever some non-client action (server restore, rollback, audit, etc.) modifies the server version of data in a file space such that it can no longer be guaranteed to represent what was on the client at the last incremental backup.

- a full incremental backup did not successfully complete when the above conditions were true. This backup either:
- did not specify a full file system ("Last Backup Completed " date on Tivoli Storage Manager server was not updated).
Use the QUERY FILESPACE command on the Tivoli Storage Manager server to confirm if the date was updated
- was an incremental by date backup, but no prerequisite full incremental backup was previously performed
A full incremental backup is required before an incremental by date backup can be performed.

Other errors which occur in the journal daemon may cause the journal to be reset and require a successful incremental backup to complete before the journal will be available for backup again. These include:
- disk errors (out of space, inaccessible journal, etc.) or other platform specific errors
- on Windows, very high volumes of file system activity may also cause a journal to be reset and invalided due to journal buffer overflow errors. This situation is discussed in detail in the Notification Buffer Overflow and Maximum Journal Size Exceeded topics under the Common Problems section.

If journal backup isn't possible, the client will always revert to a normal non-journal based backup. By default, journals are always invalidated and deleted when the journal service terminates, which means that a full incremental backup must be done when the service restarts in order to re-validate the journal. This behavior may be overridden with the PreserveDbOnExit setting (described in the Configuration Section below). Note that Journal Based Backup improvements made at certain client release levels require the previous journal to be invalidated and deleted when the client is upgraded to at least that level for the first time, even if PreserveDbOnExit has been specified.


What types of backup operations will use the journal ?

The following table describes when the journal will be used and interaction between the backup client and the journal:

Backup Typeinactive journalvalid journal with integrityinvalid active journalvalid journal without integrity
full file system incremental

example: dsmc incr c:
no interaction with journaljournal is used for backup

client sends journal delete entry notifications when objects are committed on server
client sends journal reset notification

journal is not used for backup

client sends journal delete entry notifications when objects are committed on server

client sends incremental complete notification when backup successfully completes which places journal in valid state
same as invalid active journal
full file system incremental by date

example: dsmc incr c: -incrbydate
no interaction with journaljournal is not used for backup

client sends journal delete entry notifications when objects are committed on server
client sends journal reset notification

journal is not used for backup

client sends delete entry notifications to journal when object are committed on server

client sends incremental complete notification when backup successfully completes which places journal in valid state
same as invalid active journal
partial incremental backup

example: dsmc incr c:\testdir\* -sub=yes
no interaction with journaljournal is used for backup, client will query journal for all entries under specified directory only if -subdir=yes is specified

client sends journal delete entry notifications l when objects are committed on server
no interaction with journal
journal is not used for backup

client sends journal delete entry notifications when object are committed on server
partial incremental backup by date

example: dsmc incr c:\testdir\* -incrbydate
no interaction with journaljournal is not used for backup

client sends journal delete entry notifications when objects are committed on server
no interaction with journaljournal is not used for backup

client sends journal delete entry notifications when objects are committed on server


How is Journal Based Backup configured to run in a cluster environment ?

Cluster exploitation is achieved by configuring journal service instances on each node in the cluster to
journal both local and shared resources. Shared resources are defined with the DeferFsMonStart setting enabled so that they may move to different nodes in the cluster when a failover or resource move occurs.

The location of the journal directory for shared resource journals must be accessible by all nodes in the cluster which may own the shared resource. It is necessary to use the JournalDir setting to define different journal directory locations for shared and local resources.

The PreserveDbOnExit setting may be used to allow journals for shared resources to remain valid when a resource moves to a different node in the cluster.

On AIX, two sample perl scripts are provided with the journal: jbbadd.pl and jbbdelete.pl. The invocation of these scripts can be added to StartClusterTsmClient.sh and StopTsmClient.sh. The arguments for these scripts are just the file system names of the shared resources.

Cluster configuration settings are described in more detail in the Configuration section. More information about setting up multiple journal service instances can be found in the section "Installing Multiple Journal Services on the same machine (Windows)."


Does the Journal support long file path/names? (Windows only)

All currently supported versions of the Tivoli Storage Manager Windows client is capable of backing up and restoring files that have path lengths greater than 1 K. It is not recommended to use any older versions that do not support these paths.




Journal Service Configuration Settings

Journal service configurations settings are specified in the journal service stanza based configuration file tsmjbbd.ini.

Note that journal service configuration processing is totally separate from backup client options processing.

JournalSettings stanza

General journal service process settings include:

Tracefile - jbb process tracefile
Traceflags - space delimited set of traceflags
TraceSegMax - maximum size in meg of tracefile segments
TestFlags - space delimited set of testflags
NlsRepos - nls message repository
Errorlog - jbb process errorlog, default is jbberror.log
JournalPipe - session manager pipe name client initially connects to,
used for running multiple instances of the journal service on Windows,
corresponds to client option of the same name

Under most circumstances these settings shouldn't need to be changed.

Journaldir

The directory where journal database files are stored and written.

On Windows, the default is the Journal Service install directory.

On AIX, the default is a directory named .tSm_JoUrNaL
(used within each file system being journaled).

By default this setting applies to all journaled file systems but may be overridden via an override stanza for each journal file system.

If the default value is a fully qualified path (for example c:\tsmjournal or /tsmjournal) all journal db files will be written to the specified directory.

If the default value does not specify a drive letter (for example \tsmjournal) or start with a directory delimiter (for example ./tmsjournal), the journal db files for each journal file system will be written to the specified directory on each journal file system.

Not specifying a drive letter or fully qualified name guarantees that a journal will always be accessible from any node in a cluster if the resource being journal is moved.


JournalExcludeList stanza

Specifies objects not to insert into an active journal. Filesystem changes to objects which match exclude criteria are not recorded in the journal.

Patterns may contain simple wildcard characters and environment variable specifications as follows:

%EnvVar% - expand environment variable

Note that environment variable expansions are done prior to pattern matching.

Pattern matching characters:

* - match zero or more characters
% - match one character

Note that backup client exclude processing and journal daemon exclude processing are totally separate.


JournaledFileSystemSettings stanza

Default settings for all file systems being journaled. These settings may be overridden by override stanzas for individual journaled file systems.

JournaledFileSystems

The space delimited list of filesystems to monitor and journal.

Only local, fixed drives are supported, network and removable filesystems are not.

AIX virtual mount points or Windows 2000 mount points may also be specified (i.e.c:\mountpntdir).

Journals for specific file systems may be brought online or offline while the journal service is running by modifying this option and saving the file.


JournalDBSize

Maximum journal database size (in bytes) for a journaled filesystem.

A size of 0 indicates that the journal database size will be restricted only by the capacity of the drive where journal db directory resides or by the supported journal db size of 2 gigabytes.


NotifyFilter (Windows only)

Specifies what types of filesystem activity to monitor for a journaled filesystem. Multiple activities may be monitored by adding values together. Specifying a less comprehensive filter value may reduce the size of journals and improve performance.

The qfilter tool is useful for calculating different filter values, and the filemon file system profiling utility may be useful in evaluating the amount of change activity generated by different filter values. These tools are described in the Useful Tools and Utilities section.

The default value of this setting may be overridden for individual file systems via an override stanza. The default value is 0x117 (File and Dir name changes, last write changes, attribute changes, size changes, and security changes).

Notification Type Filter Value

File name changes, including create, 0x00000001
delete and rename

Attribute changes 0x00000004

Size changes, notification is deferred 0x00000008
until cache is flushed

Last written time changes, notification 0x00000010
is deferred until cache is flushed

Last access time changes 0x00000020

Creation time changes 0x00000040

Security (acl) changes 0x00000100


DirNotifyBufferSize
Change notification buffer for directory name changes.

By definition a directory name changes occurs when a directory is created or deleted.

On Windows, the default value of this setting 0x100000 (1 Meg), and in general the value should be between 4kb and 2MB. Values of less than 1MB are allowed.

On AIX, the default is 128k.

The default value specified in this setting may be overridden for individual journaled file systems by an override stanza.

In general this buffer size can be smaller than the buffer size for non-directory change notifications as specified below. On Windows too large a value may cause notification buffer overflows to occur which will result in journals being invalidated. Directory notification buffer overflows are described in more detail in the Common Problems section.


NotifyBufferSize

Change notification buffer size for non-directory name changes on a journaled filesystem.

A large amount of change activity on a journaled filesystem may require this to be increased.

On Windows, the default value of this setting 0x100000 (1 Meg), and in general the value should be between 1 and 10 megabytes.

On AIX, the default is 128k.

On AIX, filesystem activity may also be momentarily blocked.

The default value specified in this setting may be overridden for individual journaled file systems by an override stanza.

Note that a notification buffer is allocated for each file system being journaled, so increasing the value of this setting will increase the memory utilization of the journal service process.

On Windows too small a value may cause notification buffer overflows to occur which will result in journals being invalidated. Notification buffer overflows are described in more detail in the Common Problems section.


PreserverDBOnExit

This setting allows a journal to remain valid when it is brought offline. Journals go offline when the journal service shuts down, or when the file system is removed from the list of journal file systems in the journal configuration file while the journal service is running. A value of 1 indicates that the journal will not be deleted when the journal file system goes offline and will be valid when the journal file system comes back online.

This setting is intended to be used when bringing a journal offline in a controlled manner (such as in the case of a cluster failover) and should be used with extreme caution as any file system change activity which occurs while the journaled file system is offline will not be reflected in the journal database when it is brought back online. Unrecoverable database disk errors which force a journal offline may also cause a journal to be brought back online in a corrupt state.

The default value of this setting is 0 (delete journal db on exit).

When the value of this setting is 0 the journal is automatically reset when it is brought online.

The default value of this setting may be overridden for individual file systems with an override stanza.


DeferFsMonStart

This setting allows the journal service to defer bringing a journal online when the file system being journaled isn't available. The journal service will also monitor online journaled filesystems for availability and will bring the filesystem offline if it becomes unavailable.

This setting is most commonly used in a cluster environment where shared resources may move to different nodes in the cluster. A value of 1 will defer bringing a journaled file system online until the specified journaled filesystem is valid/available and the specified journal directory can be created/accessed. Resources are re-checked at the interval specified by the DeferRetryInterval setting.

The default setting is 0, meaning that unavailable or inaccessible journaled file systems will not be restarted until the journal service is recycled.

The default value of this setting may be overridden for individual file systems with an override stanza.


DeferRetryInterval

The interval in seconds the journal service will check journaled filesystems configured with the DeferFsMonStart setting enabled for availability.

The default value is 5 seconds.

The default value of this setting may be overridden for individual file systems with an override stanza.


LogFsErrors

Usually used in conjunction with the deferFsMonStart setting to identify/eliminate excessive File System unavailable messages from being written to the logs when bringing a journaled filesystem online is deferred.

A default value of 1 indicates that all errors encountered accessing a journaled filesystem or journal directory should be logged (both to the jbb errorlog and the Windows eventlog).

A value of zero indicates that logging of errors encountered while checking deferred file systems and journal directories will be suppressed.

The default value of this setting may be overridden for individual file systems with an override stanza.


Sample Journal Configuration File

;
; JournalSettings - General Journal Service Settings
;
; Tracefile - jbb process tracefile
; Traceflags - space delimited set of traceflags
; TestFlags - space delimited set of testflags
; NlsRepos - nls message repository, default is dscameng.txt
; Errorlog - jbb process errorlog, default is jbberror.log
;
;
; Journaldir - Directory where journal database files are stored and
; written.
;
; Default is the Journal Service install directory.
;
; By default this setting applies to all journaled
; file systems but may be overridden via an override
; stanza for each journal file system.
;
; If the default value is a fully qualified path
; (for example c:\tsmjournal) all journal db files
; will be written to the specified directory.
;
; If the default value does not specifiy a drive
; letter (for example \tsmjournal) the journal db
; files for each journal file system will be written
; to the specified directory on each journal file system.
;
;----------------------------------------------------------------------------
;
[JournalSettings]
Errorlog=jbberror.log
;
;
;
; JournalExcludeList
;
; Specifies objects which will be filtered from being inserted/process
; to/from the journal database.
;
; Filesystem changes to objects which match exclude criteria are not
; recorded in the journal.
;
; Patterns may contain simple wildcard characters and environment variable
; specifications as follows:
;
; %EnvVar% - expand environment variable
;
; Note that environment variable expansions are done prior to
; pattern matching.
;
; Pattern matching characters:
;
; * - match zero or more characters
; % - match one character
;
;
;
; Note that this list contains system files which should always be excluded.
; Please do not remove any of these entries.
;
[JournalExcludeList]
%SystemRoot%\System32\Config\*
%SystemDrive%\Adsm.Sys\*
%TEMP%\*
%TMP%\*
%USERPROFILE\*\index.dat
*.jdb.dir
*.jdb.pag
*.jdbinc.dir
*.jdbinc.pag
*:\pagefile.sys
*:\*.crmlog
*:\hiberfil.sys
%SystemRoot%\csc\*
%SystemRoot%\System32\DTCLog\MSDTC.LOG
%SystemRoot%\netlogon.chg
%SystemRoot%\schedlgu.txt
%SystemRoot%\registration\*.clb
%WINDIR%\debug\*
%SystemDrive%\Documents and Settings\NetShowServices\ntuser.dat
%SystemDrive%\Documents and Settings\NetShowServices\Local Settings\Application Data\Microsoft\Windows\UsrClass.dat
%USERPROFILE%\ntuser.dat
%USERPROFILE%\Local Settings\Application Data\Microsoft\Windows\UsrClass.dat
;
;
;
;
;
; JournaledFileSystemSettings
;
; JournaledFileSystems
;
; Space delimited list of filesystems to monitor and journal.
;
; Only local, fixed filesystems are suppored, network and removable
; filesystems are not.
;
; Windows 2000 mountpoints may also be specified (i.e.c:\mountpntdir).
;
; There must be at least one entry in this list or the Journal service
; will not start. If the service does not start, the reason may be
; found in the event viewer application log.
;
;
;
; Note the following settings specified under the JournaledFileSystemSettings
; stanza will apply to all journaled filesystems unless overriden by a
; JournaledFileSystems override stanza setting of the same name.
;
; For example, the override stanza name for C:\ would be
; JournaledFileSystemSettings.C:\ .
;
;
; JournalDBSize
;
; Maximum journal database size for a journaled filesystem.
;
; A size of 0 indicates that the journal database size will be
; restricted only be the capacity of the drive where journal db
; directory resides.
;
; NotifyFilter
;
; Specifies what types of filesystem activity to monitor for
; a journaled filesystem.
;
; Multiple activities may be monitored by combining (logical OR)
; values.
;
; The default value is 0x117 (File and Dir name changes,
; last write changes, attribute changes, size changes, and
; security changes).
;
;
; Notification Type Filter Value
;
; File name changes, including create, 0x00000001
; delete and rename
;
;
; Attribute changes 0x00000004
;
; Size changes, notification is deferred 0x00000008
; until cache is flushed
;
; Last written time changes, notification 0x00000010
; is defered until cache is flushed
;
; Last access time changes 0x00000020
;
; Creation time changes 0x00000040
;
; Security changes 0x00000100
;
;
;
; NotifyBufferSize
;
; Change notification buffer size for a journaled filesystem.
;
; Large amount of change activity on a journaled filesystem may
; require that this to be increased. The default is 0x100000 (1 Meg).
;
;
; JournalDir
;
; See description under JournalSettings
;
;
; PreserverDBOnExit
;
; A value of 1 indicates that the journaled file system journal database
; will not be deleted when the journal file system goes offline and
; will be valid when the journal file system comes back online.
;
; This value should be used with caution as any file system change
; activity which occurs while the journaled file system is offline
; will not be reflected in the journal database.
;
; The default setting is 0 (delete journal db on exit).
;
;
; DeferFsMonStart
;
; A value of 1 will defer bringing a journaled file system online until the
; specified journaled filesystem is valid/available and the specified journal
; directory can be created/accessed.
;
; Also will monitor online journaled filesystems for availability
; and will bring the filesystem offline if it becomes unavailable.
;
; Resources are checked at the interval specified by the deferRetryInterval
; setting.
;
; This setting is most commonly used in a cluster environment where shared
; resources may move to different machines in the cluster.
;
; The default setting is 0, meaning that unavailable or inaccessible journaled
; file systems will not be restarted until the journal service is recycled.
;
;
; DeferRetryInterval
;
; The value in seconds journaled filesystems with the deferFsMonStart setting
; enabled will be checked for availability.
;
; The default value is 60 seconds.
;
;
; logFsErrors
;
; A value of 1 indicates that all errors encountered accessing
; a journaled filesystem or journal directory should be logged
; (both to the jbb errorlog and the NT eventlog).
;
; A value of zero indicates that logging of errors encountered
; while checking deferred file systems and journal directories
; will be suppressed.
;
; Usually used in conjunction with the deferFsMonStart setting to
; eliminate excessive File System unavailable messages from being
; written to the logs when bringing a journaled filesystem online
; is deferred.
;
; The default value is 1 (log all errors).
;
;
[JournaledFileSystemSettings]
;
; List of journaled filesystems: c:, d:, and W2K mounted volume c:\mountpnt
;
JournaledFileSystems=c: d: c:\mountpnt
;
; Default journal db size
; Allows journal to grow, limited only by space available
JournalDBSize=0x00000000
;
; What events to monitor
;
; File & Dir Name, Attrib, Size, Security
; (no caching, immediate notification)
;NotifyFilter=0x00000107
;
;
; Notification Buffer Size
;
NotifyBufferSize=0x00200000
;
;
; Override settings for individual journaled filesystems.
;
; These settings override settings in JournaledFileSystemSettings
;
; Example of configuring a shared cluster resource override stanza
;
[JournaledFileSystemSettings.D:\]
JournalDBSize=0x0020000
NotifyFilter=0x00000107
DirNotifyBufferSize=0x00100000
NotifyBufferSize=0x00500000
JournalDir=d:\tsmjournal
; Don't delete the journal when a journaled file system goes offline
PreserveDBOnExit=1
; allow journals to move between cluster nodes
DeferFsMonStart=1
DeferRetryInterval=30
logFsErrors=0
;
;



What are common problems using the Journal?

Notification Buffer Overflow Errors

The mechanism used to monitor file system change activity requires a pre-allocated buffer to store file change notification entries for each file system actively being journaled. Buffer overflows occur when the file system is flooded with a very large volume of change activity in a short window in time. If change notifications arrive faster they can be processed by the file system monitor portion of the journal service the notification buffer will overflow, and the file system journal must be invalidated because the integrity of the journal has been compromised since change activity has been missed.

For example, copying or deleting a large directory tree (large in terms of number of objects and directory depth) may cause buffer overflow errors to occur.

Increasing the size of the NotifyBufferSize setting may eliminate this problem, but increasing the size beyond 10 megabytes isn't recommended as it increases memory utilization and degrades system performance.

Decreasing the size of the DirNotifyBufferSize setting may eliminate this problem, but increasing the size beyond 2 megabytes may increase the risk of this failure occurring.


Deleting files from a Windows command prompt

Files deleted from a Windows command prompt always generate change notifications for long file names with the compressed DOS style 8.3 name. This presents several problems to journal based backup. Since the file has been deleted there is no method of obtaining the original long name of the file. Any previous backup of the file would have been under the long name, and since there is no reliable method of deriving the original name from the compressed name, the individual file can't be expired. Since the individual file can't be expired, journal based backup must force a non-journal based backup to be performed on the path portion of the file.

Another problem with deleting files from a command prompt may cause a journal database to exceed the maximum supported size of 2 GB. Using the Windows "rd" (or "rmdir") command to delete a large number of files may cause the journal database to grow to its maximum size of 2 GB. This problem seems to be very rare, but can occur if the files being deleted have very similar names, or if the 8.3 names are very similar.

     Both these problems may be circumvented by disabling NTFS 8.3 naming for long file names. This can be done by changing the NtfsDisable8dot3NameCreation value from 0 to 1 in the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem

A reboot is necessary to complete the change. Note that this change does not eliminate existing 8.3 file names, but suppresses the automatic creation of new 8.3 names. Also note that many legacy DOS and 16-bit Windows applications rely on 8.3 file names. Disabling 8.3 file name creation may make new files with names longer than 8.3 inaccessible to these legacy applications.


Installing Multiple Journal Services on the same machine (Windows)

Multiple journal services are installed by using the dsmcutil client utilitity. Each instance must specify a unique service name and specify a unique journal configuration file via the /JBBCONFILE dsmcutil option. In addition, each tsmjbbd.ini/dsm.opt config file "pair" must specify a unique value of the JournalPipe setting. Backup clients specify the specific journal service to connect to by using the JournalPipe client option to point to a specific journal service instance.

Note: Configuring multiple journal services to journal the same file system currently isn't supported.

Example of the JournalPipe setting/option:

in dsm.opt (client options file):

JournalPipe \\.\pipe\journalService1

in tsmjbbd.ini (journal config file):

JournalPipe=\\.\pipe\journalService1


Example of installing multiple services via dsmcutil:

dsmcutil install journal /name:"TSM Journal Service 1" /jbbconfigfile:"f:\journal1\journal1.ini"

TSM Windows NT Client Service Configuration Utility
Command Line Interface - Version 5, Release 4, Level 0.0
(C) Copyright IBM Corporation, 1990, 2007, All Rights Reserved.
Last Updated Jan 9 2007
TSM Api Verison 5.4.0

Command: Install TSM Client Service
Machine: BARKENSTEIN(Local Machine)

Installing TSM Client Service:

Machine : BARKENSTEIN
Service Name : TSM Journal Service 1
Client Directory : C:\program files\tivoli\tsm\baclient
Automatic Start : no
Logon Account : LocalSystem

The service was successfully installed.

Creating Registry Keys ...

Updated registry value 'ImagePath' .
Updated registry value 'EventMessageFile' .
Updated registry value 'TypesSupported' .
Updated registry value 'TSM Journal Service 1' .
Updated registry value 'ADSMClientKey' .
Updated registry value 'jbbConfigFile' .

Starting the 'TSM Journal Service 1' service ....

The service was successfully started.


JournalSettings stanza configuration settings

SettingPlatformDefaultDescription
dbKeySize All1281Size in characters of maximum allowable path name which can be recorded in the journal
TraceFileAllTrace.outTrace file
TraceFlagsAllNoneList of space delimited trace flags
TraceMaxAllNoneMaximum size of the aggregate trace file
TraceSegMaxAllNoneMaximum size of trace file segments
ErrorLogAllJbberror.log Fully qualified journal daemon error log
ErrorLog MaxAllNoneMaximum size of the error log before wrapping will be done
JournalDirAllDirectory where journal daemon residesLocation of journal database, may be overridden for particular journals with override stanzas
JournalPipeAllDepends on platformDefault session manager pipe clients connect to.

Should only be changed when configuring multiple journal daemons on the same machine.
(Not allowed on AIX)
Corresponds to the client option of the same name.
SessionTimeoutAll1200Time in seconds a journal daemon client session will wait for a client response
JournaledFileSystemsAllnoneSpace delimited list of file systems to monitor



JouraledFileSystem stanza configuration settings

SettingPlatformDefaultDescription
JournalDbSizeAll0 (unlimited size)Size journal databases may grow to in bytes
notifyFilterWindows0x117Type of change activity to monitor
notifyBufferSizeWindows0x200000 (2 meg)Notify buffer size for non-directory name changes
dirNotifyBufferSizeWindows0x100000 (1 meg)Notify Buffer size for directory name changes
PreserveDbOnExitAll0 (disabled)Prevents journal from being deleted and reset when they are brought offline
deferFsMonStartAll0 (disabled)Allow file systems which aren’t available to be placed in defer mode
deferRetryIntervalAll5 secondsInterval in seconds in which deferred file systems will be checked for availability
logFsMonErrorsAll1 (enabled)Log error messages when file systems are not available



Windows Notify Filter Flag values
FlagValueChanges to monitor
File Name0x01File creations, deletions, and renames
Directory Name **0x02Directory creations, deletions, and renames
Attribute0x04Attribute changes including NTFS metadata
Size *0x08File size
Last Write timestamp *0x10Last modified timestamp
Last Access timestamp0x20Last access timestamp
Creation timestamp0x40Creation timestamp
NTFS Security0x100NTFS Security descriptor

* change notifications deferred until system write cache is flushed
** mutually exclusive of other flags

[{"Product":{"code":"SSGSG7","label":"Tivoli Storage Manager"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":"Client","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Supported Versions","Edition":"Edition Independent","Line of Business":{"code":"LOB26","label":"Storage"}}]

Product Synonym

TSM

Document Information

Modified date:
17 June 2018

UID

swg21681523