Grafen & Hails: Modern Statistics for the Life Sciences
16.01.03 Rajah Roy, St Peter's, Oxford
Chapter 10, page 186
Q: Hi! I was hoping you might be able to clear something up for me which is proving a bit confusing. It concerns when to use Seq SS instead of Adj SS for performing F-ratio tests in regard to models involving interactions. For example, in Chapter 10, pp186-188, using Adj SS for the F-ratio tests provides the incorrect p-values for the lower order terms. Fine. There are no issues with orthogonality either. On p195 the same model is run, but Seq SS are used for the tests, providing the correct p-values. On pp198-200 a model involving 3 categorical variables is analysed using Seq SS, due to loss of orthogonality. So surely it would be correct to use Seq SS for tests involving interactions when there is either loss of orthogonality or when there is a continuous variable present (and hence the test cannot be orthogonal). Otherwise using Adj SS for tests you would be analysing the main effects having already taken into account the variability explained by the interaction terms. For example, p127 contains a simple interaction, but with a continuous variable (BACBEF). Tests are run using Adj SS, and there is also a substantial difference between the Seq SS and Adj SS terms (e.g. Treatmt drops from 83.35 to 5.83, presumably having something to do with the interaction term). Does this mean that the main effect has been analysed having already taken into consideration the variability of the interaction term? Similarly, on p131 Adj SS are used to test a continuous variable when an interaction term is present. Is this still valid? Hopefully there's a simple explanation, and if you could clarify it I would be most grateful!
A: Rajah is right in his conclusion that Seq SS should be tested for main effects when there are interactions and we have non-orthogonality. He is naturally puzzled because Boxes 7.7 and 7.9 on pages 127 and 131 display tests of the Adj SS. Notice, though, that those invalid tests are not used in the text. It is an unfortunate fact of life that statistical packages provide tests which are invalid, and perhaps we should campaign to have them print *** in place of the F-ratios and p-values! But for the moment, readers must realise that not all tests provided by packages are valid tests, and that the reader must be careful, as we were in the text, about which tests to use from any given analysis.
It is true we could have asked for Seq SS throughout the analyses, and this might have been a logical thing to do. However, we knew that we were interested only in testing the interaction, and that for the interaction the Seq and Adj SS would be equal. In many packages (including Minitab and SPSS) the default is to produce Adj SS, and so it is 'natural', or at least easiest, to view the Adj SS unless you actually need the Seq SS.
From a didactic point of view, the passages explaining when to use Seq SS don't appear until later in the book. Thus we couldn't have explained at pages 127 and 131 that it was better to use Seq SS, without bringing the whole discussion forward. Rajah is right to notice that we gloss over the problem on pages 127 and 131, but we were very careful not to fall into an error. It is really necessary to explain interactions before marginality, so it is almost inevitable that readers who have fully understood the later points will re-read the earlier parts with a certain sense that they were not told the whole truth first time round!