Matching and searching data using the Probabilistic Matching Engine for InfoSphere BigInsights

Important update: If you are downloading the IBM® InfoSphere® MDM Probabilistic Matching Engine for IBM InfoSphere BigInsights™ for the first time, you can choose to download only the iFix released in December 2013. The iFix is a complete replacement for the previous version of the offering.

With Probabilistic Matching Engine for InfoSphere BigInsights, you can efficiently derive master data, compare members, resolve members into entities, and do probabilistic searches.

To provide the master data underpinnings for such scenarios, the Probabilistic Matching Engine for InfoSphere BigInsights capability is available with the Enterprise Edition of IBM InfoSphere MDM. Hereafter, this capability is referred to by the shorter name big data matching.

The big data matching capability runs as a set of InfoSphere BigInsights applications within the InfoSphere BigInsights framework in order to derive, compare, and link large volumes of records, for example 1 billion records or more. In general, the applications can either run automatically as you load data into your HBase tables or as batch processes after the data is loaded.

To make the most of big data, you have to start with data you trust. But the sheer volume and complexity of big data means that the traditional manual methods of discovering, governing and correcting information are no longer feasible.

Transaction details, multichannel interactions, social media, syndicated data from sources such as loyalty cards, and other customer-related information are powerful new tools for creating a complete picture of customers’ preferences and demands. They are the keys to understanding and predicting customer behavior.

Scalability and performance in resolving master data records can become an issue particularly when loading large lead lists and matching those lists with known customers and prospects, or when matching records against large watch lists to combat threat and fraud. Analyzing large volumes of stored data enables organizations to discover previously hidden patterns and insights that allow them to optimize processes and profitability.

Consider an example. Social media postings might tell a hotel chain that a high proportion of its business guests have too many children for standard reward rooms. The hotel can then respond by providing reward privileges on larger suites for these high-value customers, potentially increasing their loyalty.