Interaction without the main effects

[Miina Kovanen]

I’m a bit stuck in a question among appliers without some clever statistician to guide us to the theoretical reasoning. Many people I’ve talked to state that it is fundamentally wrong to consider a model including an interaction without the main effects, whereas I have thought this is true basically when the modeling is based on hypotheses testing or likelihood ratios (since to my understanding these methods use the variance components of the main effects in estimating the coefficient of the interaction term). Am I a way out here, or is it statictically correct, in principle, to take interaction without main effects as a valid parameterization in information theoretic model selection (via maximum likelihood estimation)?

Furthermore,
1) In case it is statistically correct and also biologically reasonable to hypothesize (a*b), when is it really necessary? Comparing (a + b + a*b) and (a + b) already provides information on the relative importance of the interaction. If the main effects are a priori thought to be possibly meaningless, whereas the interaction would make biological sense, should one also include (a*b)?

(This model would have smaller K, moreover would the difference in the structure of the model as a whole matter?)

2) Vice versa, say, if b is a priori suspected not to play a biologically important role independently of a, what is the reason to exclude model (a + a*b) ?

3) Does it matter if other parameters are also involved? For example, comparing models:
(a + b + c)
(a + b + c + a*b)
(c + a*b)

I would appriciate anybody revealing to me the general reasoning for a model including an interaction without the main effects to be a stupid one or discussing the use of this kind of parametrization.

Best regards, Miina

[cooch]

Depends to a fair degree on whether the 'factors' in your linear models are classification (class) variables, or continuous covariates, or mixtures of the two.

[Miina Kovanen]

Interesting, thank you for your reply. In the case I'm thinking, they are classification variables (for example age clas, season).

[bacollier]

Minna,
You might find this interesting. This question pop's up on the R and SPlus lists with fair regularity.

http://www.stats.ox.ac.uk/pub/MASS3/Exegeses.pdf

Bret

[mcmelnychuk]

Minna,

First, I'd suggest a change in your nomenclature, that a:b should represent the interaction term. That is, (a*b) = (a + b + a:b).

What matters, whichever way the model is specified for categorical factor variables, is that all possible combinations of a:b are accounted for by model coefficients. Consider a simple model, with two levels of age and two levels of season, represented by a & b. There are (at least) three ways of specifying the same model, all of which have the same number of parameters in total:

1) y ~ overall intercept + a + b + a:b
total of 4 parameters: 1 for int (representing one of the four a:b combinations), 1 for a (representing another a:b combination, which differs from the first due to a), 1 for b (representing another a:b combination, which differs from the first due to b), and 1 for a:b (representing the fourth a:b combination, which differs from the first due to a, b, and a:b)

2) y ~ overall intercept + a:b
total of 4 parameters: 1 for int (representing one of the four a:b combinations), 3 for a:b (representing the other three a:b combinations, which differ from the first due to the particular a:b values).

3) y ~ remove intercept + a:b
total of 4 parameters: 4 for a:b (directly representing the intercept for each of the four a:b combinations).


R is smart enough to account for all possible combinations of a & b, so if you use RMark and specify a model like 2 or 3 without main effects, they will just be accounted for instead by the multiple estimated a:b terms.

If you manually specify the models with design matrices, whether or not you parameterize with main effects or with interactions-only, just make sure that you have enough interaction terms to cover the possible combinations of a & b. If in a design matrix you only specify a single interaction parameter to go with an overall intercept, that will not provide enough information to reconstruct biological parameters for each of the four combinations. Others can correct this interpretation if I'm off-track.

Regarding your question 3, this should not change if other factor variables (c) are involved as additive terms. Whether c is categorical or continuous, the four parameters for intercept/a/b will take on different values, but will still represent intercepts for each of the four a:b combinations after accounting for c.

Cheers, Mike

[Miina Kovanen]

Thank you for the replies, both very useful! Also they somewhat challenge my skills in statistics in English, so to be able to differentiate between not fully understanding the language from not necessarily understanding the explanations on the reasoning itself , I might come back with some additional questions after another day of reading and thinking.

[Miina Kovanen]

Mike,

"If you manually specify the models with design matrices, whether or not you parameterize with main effects or with interactions-only, just make sure that you have enough interaction terms to cover the possible combinations of a & b. If in a design matrix you only specify a single interaction parameter to go with an overall intercept, that will not provide enough information to reconstruct biological parameters for each of the four combinations."

Just to make sure I read this right, by "parametrization" in saying "whether or not you parameterize with main effects or with interactions-only", do you refer to the nomenclature of the model, or the actual effects included in the probability structure the model is suggesting (despite the way the model parametrization is written)?

Just a note. "If in a design matrix you only specify a single interaction parameter to go with an overall intercept, that will not provide... " I see, this is the DM column structure for the interaction between two categorial variables, both with two levels. In my case, one of these categorial variables is actually an individual time variant constraint with two levels - since some individuals need to be transferred from the age class 0 to the age class 1 during the individual encounter histories. Thus I have built the DM with only 1 column for the interaction term and so far I have understood that's correct. Maybe you let me know if this sounds weird, see also viewtopic.php?f=11&t=816&hilit=interaction+term

[mcmelnychuk]

Minna,

When I referred to different ways of parameterizing a model, I meant alternative ways of constructing the design matrix (therefore leading to different model coefficients) which lead to the same estimates of biological parameters. Depending how you construct the design matrix, model coefficients may take on different meanings (and therefore different estimated values).

In short, I would not recommend incorporating an interaction term between two categorical variables if the main effects of one or both of those variables are not included (see the Venables paper posted by Bret for a good discussion of this). The interaction term does not provide enough information by itself to describe the effects of those explanatory variables on the biological response variable. The exception to this, which I tried to highlight with the 3 examples, is that it's possible to properly specify a model using just the "interaction term" (a:b) if there is more than one level represented by a:b. In example (3) above, the model
y ~ -1 + a:b should result in the independent coefficients a1b1, a1b2, a2b1, a2b2. This model doesn't represent a single interaction term without main effects, it's just that the term a:b is used to represent 4 different combinations of possible a & b values.

Hope that helps, and good luck, Mike