Troubleshooting
Problem
I performed a Kruskal-Wallis test in the NPAR TESTS procedure and now wish to perform a post hoc analysis to see which pairs of groups differ significantly. How can this be done?
Resolving The Problem
The newer NPTESTS procedure offers post hoc tests for the Kruskal-Wallis omnibus test. In the menus, select Analyze>Nonparametric Tests>Independent Samples. (Note that in versions through 21, you can only specify Scale dependent variables, while beginning with Version 22 you can specify Ordinal or Scale dependent variables.) You will get a Kruskal-Wallis test and will also get post hoc tests automatically if the omnibus test is significant if your grouping variable has more than two levels. Note that the full test results for the K-W test and the post-hoc tests are contained in the Model Viewer in the output, if you have your settings to show Model Viewer output. You need to double-click on this object in the output to see the omnibus test results, which will be on the right-hand side of the Viewer when it opens. Select the View drop down at the bottom of the screen and Pairwise Comparisons to see the post-hoc results. For the pairwise comparisons, adjusted significance levels are given by multiplying the unadjusted significance values by the number of comparisons, setting the value to 1 if the product is greater than 1.
Post hoc testing is not offered in the NPAR TESTS procedure, but a series of Mann-Whitney tests can be performed to ascertain which pairs of groups differ significantly from one another. For example, if the test variable is called Y and the grouping variable GROUP has four levels numbered 1 to 4, run the syntax:
NPAR TESTS
/M-W= y BY group(1 2).
NPAR TESTS
/M-W= y BY group(1 3).
NPAR TESTS
/M-W= y BY group(1 4).
NPAR TESTS
/M-W= y BY group(2 3).
NPAR TESTS
/M-W= y BY group(2 4).
NPAR TESTS
/M-W= y BY group(3 4).
In post hoc analyses, one wishes to control the experiment-wise type-1 error rate in order to be assured, for example, of having no more than a five percent chance of making a type-1 error for ANY of the hypotheses being tested. To accomplish this in the present situation, one can apply either a Bonferroni or Dunn-Sidak correction to the significances obtained from the series of Mann-Whitney test results, and, if a corrected significance remains under .05, declare that pair of groups to be significantly different on the test variable Y.
The Dunn-Sidak adjustment is a bit more powerful than the Bonferroni, but a little more difficult to compute. Both will be described here. Let px denote the experiment-wise type-1 error rate, ps a single test type-1 error rate, and k be the number of pairwise comparisons that can be made. The Bonferroni adjustment calculates
px = k * ps ,
and declares that pair of groups to be significantly different at the .05 level if
px < .05. The Dunn-Sidak adjustment calculates
px = 1 - (1 - ps)**k .
Both expressions are upper bounds for the chances of making a type-1 error on any of the k comparisons. Px calculated using the Dunn-Sidak expression will always be a little smaller than if it is calculated using the Bonferroni adjustment.
Note that the analysis described here does not produce results identical to those obtained in Release 18 or later in NPTESTS. The reason for this is that the method used in NPTESTS compares pairs of groups based on rankings created using data from all groups, as opposed to just the two groups being compared. The tests in NPTESTS are known as Dunn-Bonferroni tests and are based on:
Dunn, O. J. 1964. Multiple comparisons using rank sums. Technometrics, 6, 241-252.
Related Information
Historical Number
25165
Was this topic helpful?
Document Information
Modified date:
16 April 2020
UID
swg21477370