Upgrades to 4.0.1 create a significantly larger, incorrect index - testfix is required before the upgrade

Updates:
April 1, 2013: Changed version referenced in the following statement from 4.0.1.1 to 4.0.2:
This problem is addressed in CLM 4.0.2 and later releases.
February 4, 2013: Clarified information about requesting a testfix.

Cause

The 4.0.1 and 4.0.2 RDF index providers keep the index size in the RDF index itself. However, in 4.0 and before the index size was only kept in an external meta properties file. The effect is that in 4.0.1, Jazz will believe the 4.0 (or older) index is empty, causing it to reinsert the old data. More details of the cause can be found in defect 245220.

Symptoms

The most immediate symptom is a significantly increased disk size of the index. The problem only affects resources which existed before the migration. Newly added resources after migration are not affected.

A secondary symptom is an initial sluggishness for a number of operations when 4.0.1 starts for the first time after the upgrade. Although queries will keep responding when only dealing with pre migration data, queries on newly added data may either take a significantly longer time or even time out. If queries time out, the corresponding application log files will contain warning or error messages in this regard. Defect 244504 shows an instance of this symptom.

In some rare cases, the 4.0.1 index will actually be invalid and cause some queries to return the wrong results. If this happens, the effect will be duplicate results. Defect 36270 shows an instance of this symptom.

Resolution

This problem can be avoided or resolved in the following two ways dependant upon your environment:

1. If you are running 3.x, 4.0 or 4.0.0.x and are planning to upgrade to 4.0.1, you should contact IBM Rational Support to request a testfix for defect 245220 to apply to the newly installed 4.0.1 instance prior to the data migration from the previous 3.x, 4.0 or 4.0.0.x release.
   
2. If you have already upgraded to 4.0.1 and notice the symptoms described in this technote, then the problem can only be fixed by running repotools-application reindex all. Ensure the CLM server is completely shut down before running the command.
   

This problem is addressed in CLM 4.0.2 and later releases.

Leverage the Jazz Community

Jazz and Rational Team Concert have an active community that can provide you with additional resources. Browse and contribute to the User forums, contribute to the Team Blog and review the Team wiki. Refer to technote 1319600 for details and links.