Design, Conduct and Analysis: Division E

1. Distinguish between Design structure and Treatment structure
   1. Introduction

i.      There is a great deal of diversity in studies and it is very useful to categorize them in a variety of ways. The design structure is basically the physical layout of the study. For example, if we compare two weightings of each of ten animals with those of twenty animals, in each case there are twenty data points but the design structure is different-and the analysis will differ. The design structure must be known to correctly interpret the results and their implications. The treatment structure of a study describes the possible relationships among factors. For example, factorial treatment structure will involve the application of all possible factor combinations (or fractions thereof) to experimental units.

1. Rule of Thumb

i.      Distinguish between the design structure and the treatment structure of a study.

1. Illustration

i.      The introduction to this rule presented a simple example. A richer example is given in Rule VI, with a factorial design (treatment structure) with each treatment combination assigned to different subjects or the same subjects (design structure). In that example the treatment structure was the same but the design structure of Design A differed from that of Design B.

ii.      The example discussed in the previous rule has a factorial treatment structure, the design structure involves randomly allocating animals to the treatment combinations-identical to what would be done of the study involved six unrelated treatments.

1. Basis of the Rule

i.      The design structure determines the layout of the analysis. The treatment structure determines the finer partitioning of the degrees of freedom.

1. Discussion and Extensions

i.      The concepts of treatment and design structure are proven to be helpful in thinking about the development of designs for studies. They indicate that there are constraints imposed by the study units and the treatment units. Suppose that in the example above it was not possible to assign all treatment combinations to the same subjects (possibly because of carry-over effects or a practical problem of not being able to get the subjects to come in four times). The treatment structure is still of interest but the design structure restricts the design of experiments. A great deal of work has been done to make treatment structures and design structures compatible. For example, in factorial studies it may not be feasible (because of cost or time constraints) to study all possible factorial combinations-a four factor experiment with three levels per factor requires 3₄=81 experimental units per replication. This leads to consideration of fractional factorial designs mentioned in Rule IV. See Cox and Reid (2000) for more examples and further discussions. The treatment structure is the driver in the planning experiments, the design structure is the vehicle.

ii.      Design and treatment structures in experimental designs often require trade-offs. It is important to understand the differences between these strictures, to know about possible compromises and to be able to apply the best statistical strategies to the design and analysis of the proposed research.

IX.            Make hierarchical analysis the default analysis

1. Introduction

i.      This section assumes a factorial treatment structure with each factor at two levels such as presence or absence, low dose or high dose. Treatment effects in this kind of study can be classified as main effects (attributable to a single factor) or interaction effects (attributable to two or more factors). The interaction effects can be sorted into two-way, three-way or up to k-way interactions if there are k factors. An effect is of higher order if it involves more treatment combinations. With k factors there are 2_k possible methods to consider-very quickly a large number. The following rule gives a rationale for assessing a smaller number of models. In addition, it provides a guide to the ordering of the effects in the analysis.

1. Rule of Thumb

i.      Ordinarily, plan to do a hierarchical analysis of treatment effects by including all lower order effects associated with the higher order effects.

1. Illustration

i.      Consider a two-factor design consisting of factors A and B. Each factor can be present or absent. The possible different impacts of treatment are A (i.e., the effect of A present compared with A absent), B (present or absent) and A X B (both A and B present or both absent). If each of these can be in or out of the model there will be 2₃=8 possible models. If there are 3 factors, A, B and C, there are 2₇=128 possible models to analyse. The rule states that only hierarchical models should be considered. For example, a model consisting solely of the interaction A X B is not allowed. If A X B is to be examined, the rule states that A and B should also be in the model. Besides the null model (no treatment effects), in the three-factor example, there are 18 hierarchical models-a substantial reduction from 128. Not all of these 18 models are of equal interest. For example, one of the 18 models involves only the factors B, C, and the interaction B X C. why analyse only these two factors when A is also in the experiment? If the decision is made that only hierarchical models using all factors will be examined then only 9 models need to be assessed.

1. Basis of the Rule

i.      There are three basis for the rule. First, the concept of parsimony in connection with Ockham’s razor applies here. Second, a hierarchical analysis is more straightforward. Third, interpretation will be less convoluted.

1. Discussion and Extensions

i.      One rule sometimes stated is that if the interaction in a hierarchical model is significant then there is no need to examine the associated main effects. The basis for this agreement is that the presence of interaction implies differential effects of one factor at the different levels of a second factor. There is merit to this rule but there may be situations where there is both an interactive effect and a main effect, and it may be interesting to compare the magnitude of the two effects.

ii.      Bryk and Raudenbush (1992) provide a thorough review of hierarchical models and the advantage of using this analytic strategy.

iii.      Interpreting experimental results is difficult. Using the hierarchical inference principle provides an initial simplification and guidance through the mass of data to be analysed. If a non-hierarchical analysis is contemplated, there should be a deliberate justification for this approach.

Click here for government certification in Statistics and Quality