IBM Tivoli Log File Agent Configuration

You can use either the IBM® Tivoli® 6.3 Log File Agent or the REST client in the data collector to load data into the IBM Operations Analytics - Log Analysis.

For detailed information about how to configure the loading of data into IBM Operations Analytics - Log Analysis, see the topic about loading data into IBM Operations Analytics - Log Analysis in the Configuring IBM Operations Analytics - Log Analysis section.

For more information about how to use the REST client to load data into IBM Operations Analytics - Log Analysis, see the topic about using the REST client to load log file information in the Configuring IBM Operations Analytics - Log Analysis section of the IBM Operations Analytics - Log Analysis documentation.

If you use the IBM Tivoli 6.3 Log File Agent to load data into the IBM Operations Analytics - Log Analysis server, you must install the configuration files into the agent. This configuration ensures that the agent knows where the log files for a data source are located, how to process the records in the log file, and the server to which records are sent.

When you define your custom Insight Pack, include the LFA configuration files in the lfa folder within the project. When you install the custom Insight Pack, the files are installed into the LFA that is installed with IBM Operations Analytics. The files are installed in the ../config/lo subdirectory under the root directory where the LFA is installed. For example, /home/unityadm/IBM/LogAnalysis/IBM-LFA-6.30/config/lo.

The LFA configuration for a particular data source is defined in the following files:
  • A <name>.conf file that contains the properties that are used by the Log File Agent (LFA) for processing the log files.
  • A <name>.fmt file that contains an expression and format that is used by the agent to identify matching log file records and to identify the properties to include in the Event Integration Format (EIF) record. The EIF is sent from the agent to the receiving server. The receiving server is the server where the IBM Operations Analytics server is installed. The <name>.fmt file uses a regular expression to determine matching records in the log file and to send each matching record to the IBM Operations Analytics server in an EIF event.

If you want to use the LFA to send your log files to IBM Operations Analytics server, you must customize the regular expression and define your own stanza in the <name>.fmt file to capture the log records that are to be sent. The event record format must include the host name, file name, log path, and text message. The IBM Operations Analytics server uses these values to process the logs. For more information about the IBM Tivoli 6.3 Log File Agent and the configuration files and properties, see Tivoli Log File Agent User's Guide.

The file names must be identical for both files. For example, WASContentPack_v1.1.0-lfawas.conf and WASContentPack_v1.1.0-lfawas.fmt.

LFA configuration file examples

The following example shows the files that are installed as part of the WebSphere® Insight Pack that is included as standard with IBM Operations Analytics - Log Analysis.

The WASContentPack_v1.1.0-lfawas.conf file contains many properties, including the following examples:
# Files to monitor.  The single file /tmp/regextest.log, or any file like 
/tmp/foo-1.log or /tmp/foo-a.log.
      LogSources=/home/unityadm/IBM/LogAnalysis/logsources
			/WASInsightPack/*

      # Our EIF receiver host and port. 
      ServerLocation=<EIF Receiver host name>
      ServerPort=5529
The WASContentPack_v1.1.0-lfawas.fmt file contains the following regular expression that matches any record within a monitored log file. In this example, the regular expression matches all the log records in the file and to the Operations Analytics server as an EIF event. The EIF event contains the host name where the agent is running, the file name of the log file, the log file path of the log file, and the log file record itself.
 // Matches records for any Log file:
     //

     REGEX AllRecords
     (.*)
     hostname LABEL
     -file FILENAME
     logpath PRINTF("%s",file)
     text $1
     END