Overview

This section provides an overview of IBM® Operations Analytics - Log Analysis and outlines how you can extend IBM Operations Analytics using tools and techniques outlined in this guide. You can extend IBM Operations Analytics to ingest new data and to develop Custom Search Dashboards to visualize the indexed data.

You can extend IBM Operations Analytics to ingest new data and to develop Custom Search Dashboards to visualize the indexed data. A set of related artifacts to ingest data or to develop applications will be packaged together as an installable package called an Insight Pack.

The information contained in this section is intended for developers who want to understand how to extend IBM Operations Analytics - Log Analysis to provide support for a new data source, modify support for an existing data source, or to develop a Custom Search Dashboard. An Insight Pack is a set of artifacts packaged together to allow IBM Operations Analytics - Log Analysis to ingest data or used to develop Custom Search Dashboards. An Insight Pack contains a complete set of artifacts required to process a data source. You can install, uninstall, or upgrade an Insight Pack as a stand-alone package.

The Insight Pack defines:
  • The type of data that is to be consumed.
  • How data is annotated. The data is annotated to highlight relevant information.
  • How the annotated data is indexed. The indexing process allows you to manipulate search results for better problem determination and diagnostics.
  • How to render the data in a chart.

IBM Operations Analytics - Log Analysis includes the Insight Packs:

The following Insight Packs are now installed with the product:

WASInsightPack
The WebSphere Application Server Insight Pack includes support for ingesting and performing metadata searches against WebSphere Application Server V7 and V8 log files. Updates to WAS index configuration will improve indexing performance. The field logsourceHostname has been changed to datasourceHostname.
WASAppInsightPack
The Websphere Application Server (WAS) Applications Insight Pack provides troubleshooting dashboards for WebSphere Application Server Logs. A new authentication mechanism eliminates the need to specify userid and password in the application script. The field logsourceHostname has been changed to datasourceHostname.
DB2InsightPack
The DB2 Insight Pack includes support for ingesting and performing metadata searches against DB2 version 9.7 and 10.1 db2diag.log files. The field logsourceHostname has been changed to datasourceHostname.
DB2AppInsightPack
The DB2 Applications Insight Pack provides troubleshooting dashboards for DB2 Logs. A new authentication mechanism eliminates the need to specify userid and password in the application script. The field logsourceHostname has been changed to datasourceHostname.
Syslog Insight Pack
The Syslog Insight Pack includes support for ingesting and performing metadata searches against syslog data logging. The field logsourceHostname has been changed to datasourceHostname.
WebAccessLogInsightPack
The Web Access Logs Insight Pack provides the capability to ingest and perform metadata searches against Web Access Logs such as Apache IHS, JBoss, Apache Tomcat. The pack now includes a Web Health Check Dashboard example that provides summaries of key metrics.
WindowsOSEventsInsightPack
You can use the Windows OS Event Insight pack and the IBM Tivoli Monitoring Log File Agent to load and search Windows OS events. New support for data collection using Logstash provides an alternative to the IBM Tivoli Monitoring Log File Agent.
JavacoreInsightPack
The Java™ Core Insight Pack provides the capability to ingest and search metadata that originates in Java Core files in IBM Operations Analytics - Log Analysis. The field logsourceHostname has been changed to datasourceHostname.
GAInsightPack
The Generic Annotation Insight Pack is not specific to any particular log data type. It can be used to analyze log files for which a log-specific Insight Pack is not available

The following tooling will be placed in the unity_content/tools directory:

Insight Pack Tooling
The Insight Pack Tooling Eclipse Plugin allows you to create custom Insight Packs.
DSV Toolkit
The DSV toolkit is used to create Insight Packs that allow you to load Delimiter Separated Value (DSV) data into IBM Operations Analytics - Log Analysis.
logstash integration
IBM Operations Analytics - Log Analysis includes logstash. You can use it to extend IBM Operations Analytics - Log Analysis functions so that it can ingest and perform metadata searches against log data acquired by logstash. The toolkit now supports for logstash 2.2.1 and for running logstash on Windows OS.