# UNIANOVA, GLM and MEANS fail numerical accuracy test with NIST data set

## Problem(Abstract)

To verify the computational accuracy of some SPSS procedures, I been performing various analyses with the reference data sets provided by the National Institute of Standards and Technology (http://www.itl.nist.gov/div898/strd/). During my tests I was able to replicate most of the "certified" results, but SPSS failed in the case of ANOVA models with a "higher" level of difficulty (i.e., dependent variables with a large number of "significant" digits). An example where SPSS produced totally different results can be found under
http://www.itl.nist.gov/div898/strd/anova/SmLs07.html. The dependent variable, Response, has values 1*10**12 +k, where k*10**(-1), where k consists of single digit integers. All of the variance in Response is in the values of k.

I ran ONEWAY, UNIANOVA, GLM and MEANS on the data with Treatment as the group variable/factor and Response as the dependent variable.

The ONEWAY procedure produced an ANOVA table which was very similar to the certified values. Whereas the NIST-certified SSb and SSw were 1.68000000000000 and 1.8000000000000, respectively, ONEWAY produced an SSb of 1.6832828521728516 and SSw of 1.8002539909031667. There were consequently similar variations from the certified Mean Squares Between (2.1000000000000E-01) and MSw (1.000000000000E-02) and F (21.0000000000000). UNIANOVA, and GLM, on the other hand, reported an SSb of 1.707 and an SSw of 0.000. The ANOVA table was incomplete as a result of the SSw of 0. The Means procedure produced an empty box for the ANOVA table with the footnote "a. No variance within groups - statistics for Response * Treatment cannot be computed.", despite the non-zero standard deviations reported for each group in the Report table of the MEANS output.

I have replicated the above results in SPSS Version 15 and Statistics 17.0 and 18.0 on PC systems running under MS Windows XP Prof., with both AMD and Intel processors.

## Resolving the problem

A defect was filed with SPSS Development to examine numerical accuracy issues with GLM/UNIANOVA and MEANS procedures as reflected by analysis with this NIST data set.

Because truncation errors are introduced into the input data simply by encoding it in 64-bit binary floating point, four good digits in the results is the best anybody can do using 64-bit arithmetic. The NIST canonical results were produced using 500-digit arithmetic.

UNIANOVA gave a zero result for the error sum of squares. If the difficulty of the problem is reduced slightly, UNIANOVA gives satisfactory results.

A bug was discovered in the MEANS procedure wherein an ANOVA table was not printed if computations indicate a negative sum of squares for error. This event occurs with the test problem, and it happens even if the problem difficulty is substantially reduced. The MEANS procedure bug has been fixed for Statistics 18.0.2, which is expected to be released in the first half of 2010. MEANS will now produce results for the NIST test case which are similar to those of ONEWAY .

A workaround for a data set like this, where all of the variation is in the small k portion of the values, is to rescale the dependent variable so that the coefficient of variation (the ratio of the standard deviation to the mean) is much larger. For example, you can create a version of Response, resp_scaled, which equals Response - 1*10**12 and reran the 3 procedures.

Using the resp_scaled DV, where 1*10**12 was removed from the values, the ANOVA tables for the three procedures agree with each other. Although they do not exactly match the certified results or the ONEWAY results with the original Response DV, they are much closer to the certified results and the ANOVA tables for UNIANOVA and MEANS are complete.

We apologize for any resulting inconvenience

## Historical Number

87316

### Document information

More support for: SPSS Statistics

Software version: 18.0

Operating system(s): Platform Independent

Reference #: 1485458

Modified date: 05 January 2010