Effect size question

[jbruggin]

The following verbiage appears in Chapter 6:

Suppose we consider a relative difference of 15% or greater to be biologically important. Suppose the estimated effect size for the difference in survival between the colonies was 19.3%, with a CI of 1.7%-36.9%. As such we would consider the results statistically 'significant,' since the CI doesn't include 0, but biologically inconclusive, because the CI includes the value of 10%.

Would it be correct to say that the results are biologically inconclusive because the CI includes values <15%? If not, could someone clarify where this 10% value comes from?

Thanks.

John

[cooch]

The following verbiage appears in Chapter 6:

Suppose we consider a relative difference of 15% or greater to be biologically important. Suppose the estimated effect size for the difference in survival between the colonies was 19.3%, with a CI of 1.7%-36.9%. As such we would consider the results statistically 'significant,' since the CI doesn't include 0, but biologically inconclusive, because the CI includes the value of 10%.

Would it be correct to say that the results are biologically inconclusive because the CI includes values <15%? If not, could someone clarify where this 10% value comes from?

Thanks.

John

Fairly straightforward. There are two 'kinds' of significance - statistical, and biological. They are not one in the same (which may come as a shock to some). In the example in the book, we assume, based on our biological insight (whatever that means) that a difference of 15% or greater in survival is 'biologicaly' meaningful (in some undefined context). The estimated effect size is 19.3%, with a frequentist 95% CI of 1.7-36.9.

OK, so, the 95% CI doesn't bound 0, meaning the estimate is 'statistically' significant (i.e., significantly greater than zero based on a nominal alpha level of 0.05). But, we've decided a priori that only differences of 15% or greater are 'biologically' significant. Now, in this case the 95% CI include values below this critical cut-off (e.g., includes values below 15%, including the mentioned 10%), such that the results are 'biologically inconclusive'. For the results to 'biologically' significant, the lower CI would have to be >15%.

I admit the wording in the chapter isn't particular clear on this point. I'll address that in the next revision.

[jbruggin]

OK, great, thanks.

My next effect size question relates to the example on page 7-53 which indicates:

We want to estimate B1 – the slope for the colony term in the design matrix, which we understand to be the effect size – in this case, the difference between the good and poor colonies. All you need to do is look at the ‘beta estimates’ for the model with the identity link. The estimated value for B1 is 0.0845, with a 95% CI of 0.064 to 0.105. For completeness, the estimate of the intercept (B0) is 0.7897.[/i]

Is there a step missing here? When I look at the beta estimates, I see the following:

Code: Select all

Parameter      Beta         Lower           Upper
1: Phi          0.7897        0.7729          0.8066
2: Phi          0.8743        0.8611          0.8874
3:p              0.7499        0.7352          0.7647

I see that I can get the B1 from page 7-53 by subtracting 0.7897 from 0.8743 but where does the CI of 0.064 to 0.105 come from?

[cooch]

Is there a step missing here? When I look at the beta estimates, I see the following:

Code: Select all

Parameter      Beta         Lower           Upper
1: Phi          0.7897        0.7729          0.8066
2: Phi          0.8743        0.8611          0.8874
3:p              0.7499        0.7352          0.7647

I see that I can get the B1 from page 7-53 by subtracting 0.7897 from 0.8743 but where does the CI of 0.064 to 0.105 come from?

No - nothing missing. You just looked at the wrong estimates. The beta estimates you show (above) are those from the model fit using PIMs, not the design matrix. You need to use the design matrix, shown on p. 7-52 (at the bottom). Thats really the point of the exercise - you need to be using the design matrix to estimate effect sizes.