Completely Randomized Design

CR-p -- Fixed Effects Model

AKA - One-way Analysis of variance, One-way ANOVA.

Schematic with Example Data

Level a1 a2 a3 a4 Total
4
6
3
3
1
3
2
2 4
5
4
3
2
3
4
3 5
6
5
4
3
4
3
4 3
5
6
5
6
7
8
10
Mean 3.0 3.5 4.25 6.25 4.25
sd 1.51 0.93 1.04 2.12 1.88

Or in abbreviated form:

Level a1 a2 a3 a4 Total
S₁
n=8 S₂
n=8 S₃
n=8 S₄
n=8
Mean 3.0 3.5 4.25 6.25 4.25
sd 1.51 0.93 1.04 2.12 1.88
Where each S_j is an independent randomly assigned group of subjects.

Linear Model

Y_ij = μ + α_j + ε_i(j)

The prediction model is

Y'_j = μ + α_j

where,

Y_ij is the score for the ith observation in the jth treatment level
Y'_j is the predicted value for the jth treatment level and is equal to the mean of the group
μ is the overall population mean (grand mean)
α_j is the effect of A treatment level j which is equal to μ_j - μ and is subject to the restriction that Σα_j = 0 over j
ε_i(j) is the error effect associated with Y_ij and is equal to Y_ij - μ - α_j . The error effect is a random variable that is distributed NID(0,s²_ε)

Hypotheses

Treatment Effect

Assumptions

1. The linear model reflects all sources of variation.
2. The experiment contains all the treatment levels of interest.
3. The ε_i(j) are independent of each other.
4. The ε_i(j) are normally distributed.
5. The ε_i(j) have equal variance in the population.

Notes:
Assumptions 1 & 2 are concerned with model specification.
Because μ and α_j are constants, the following holds

ANOVA Summary Table

Source SS df MS F p-value

Between Groups 49.0 3 16.333 7.50 0.0008

Within Groups 61.0 28 2.179

Total 110.0 31

The ANOVA Summary Table may also look like this:

Source SS df MS F p-value

Treatment 49.0 3 16.333 7.50 0.0008

Error 61.0 28 2.179

Total 110.0 31

Table of the F-distribution

Between Groups = Treatment or use the variable name.

Within Groups = Error or Residual

Expected Mean Squares

Correctly Formed F-ratios

Table of Group Means and Variances

a1 a2 a3 a4
Mean 3.00 3.50 4.25 6.25
Variance 2.29 0.86 1.07 4.50

Std Dev 1.51 0.93 1.04 2.12

A Measure of Strength of Association

Omega-squared (ω²) is the recommended measure of strength of association for fixed-effects analysis of variance models.

From the Example:

      49 - (3)2.179
ω² = ---------------  = 0.3785
       110 + 2.179

Approximately 38% of the variability of the dependent variable can be explained by the independent variable, that is, by the differences among the four levels of the categorical variable.

The following guidelines are suggested by Cohen (1989):

ω² = .01 is a small association
ω² = .06 is a medium association
ω² = .14 is a large association

By these guidelines the ω² = .38 is very large, but this is because the example an artificial classroom dataset.

In terms of the fhat index of effect size:

fhat = .10 is a small effect
fhat = .25 is a medium effect
fhat = .40 is a large effect

These are very rough guidelines.

Note: The fhat index of effect size should not be confused with Cohen's d index of effect size. The fhat index is derived directly form the ω².

Model for Orthogonal Coding

G  X1  X2  X3
1   1   1   1
2  -1   1   1
3   0  -2   1
4   0   0  -3

Stata Computer Example

input y grp x1 x2 x3
 4 1  1  1  1
 6 1  1  1  1
 3 1  1  1  1
 3 1  1  1  1
 1 1  1  1  1
 3 1  1  1  1
 2 1  1  1  1
 2 1  1  1  1
 4 2 -1  1  1
 5 2 -1  1  1
 4 2 -1  1  1
 3 2 -1  1  1
 2 2 -1  1  1
 3 2 -1  1  1
 4 2 -1  1  1
 3 2 -1  1  1
 5 3  0 -2  1
 6 3  0 -2  1
 5 3  0 -2  1
 4 3  0 -2  1
 3 3  0 -2  1
 4 3  0 -2  1
 3 3  0 -2  1
 4 3  0 -2  1
 3 4  0  0 -3
 5 4  0  0 -3
 6 4  0  0 -3
 5 4  0  0 -3
 6 4  0  0 -3
 7 4  0  0 -3
 8 4  0  0 -3
10 4  0  0 -3
end

tabstat y, by(grp) stat(n mean sd var)

Summary for variables: y
     by categories of: grp 

     grp |         N      mean        sd  variance
---------+----------------------------------------
       1 |         8         3  1.511858  2.285714
       2 |         8       3.5  .9258201  .8571429
       3 |         8      4.25  1.035098  1.071429
       4 |         8      6.25   2.12132       4.5
---------+----------------------------------------
   Total |        32      4.25  1.883716  3.548387
--------------------------------------------------

display 2.12132/.9258201

2.2912875

histogram y, by(grp) normal

parcord2 grp y, by(grp) 



robvar y, by(grp) /* W0 is Levene's test of homoscedasticity */

            |            Summary of y
        grp |        Mean   Std. Dev.       Freq.
------------+------------------------------------
          1 |           3   1.5118579           8
          2 |         3.5    .9258201           8
          3 |        4.25   1.0350983           8
          4 |        6.25   2.1213203           8
------------+------------------------------------
      Total |        4.25   1.8837163          32

W0  = 1.292876   df(3, 28)     Pr > F = .29625408

W50 = 1.037037   df(3, 28)     Pr > F = .39138742

W10 = 1.292876   df(3, 28)     Pr > F = .29625408

anova y grp 

                           Number of obs =      32     R-squared     =  0.4455
                           Root MSE      =   1.476     Adj R-squared =  0.3860

                  Source |  Partial SS    df       MS           F     Prob > F
              -----------+----------------------------------------------------
                   Model |       49.00     3  16.3333333       7.50     0.0008
                         |
                     grp |       49.00     3  16.3333333       7.50     0.0008
                         |
                Residual |       61.00    28  2.17857143   
              -----------+----------------------------------------------------
                   Total |      110.00    31   3.5483871 

omega2  /* download from ATS via the Internet */

 omega squared for grp = 0.3785
 fhat effect size   = 0.7805

fhcomp grp  /* download from ATS via the Internet */

Fisher-Hayter pairwise comparisons for variable grp
studentized range critical value(.05, 3, 28) = 3.4994064

                                      mean     critical
grp vs grp       group means          dif        dif
-------------------------------------------------------
  1 vs   2     3.0000     3.5000     0.5000    1.8261
  1 vs   3     3.0000     4.2500     1.2500    1.8261
  1 vs   4     3.0000     6.2500     3.2500*   1.8261
  2 vs   3     3.5000     4.2500     0.7500    1.8261
  2 vs   4     3.5000     6.2500     2.7500*   1.8261
  3 vs   4     4.2500     6.2500     2.0000*   1.8261

prcomp y grp, tukey test  /*  findit prcomp -- STB-47 sg101 */


                   Pairwise Comparisons of Means

Response variable (Y): y           
Group variable (X):    grp         

    Group variable (X): grp           Response variable (Y): y
-------------------------------    -------------------------------
      Level                            n         Mean         S.E.
------------------------------------------------------------------
          1                            8            3     .5345225
          2                            8          3.5     .3273268
          3                            8         4.25     .3659625
          4                            8         6.25          .75
------------------------------------------------------------------

Simultaneous significance level: 5%    (Tukey wsd method)
Homogeneous error SD = 1.475998, degrees of freedom = 28

          (Row Mean - Column Mean) / (Critical Diff)
Mean(Y) |        3       3.5      4.25 
Level(X)|        1         2         3 
--------+------------------------------
     3.5|       .5 
       2|    2.015 
        |
    4.25|     1.25       .75 
       3|    2.015     2.015 
        |
    6.25|     3.25*     2.75*        2 
       4|    2.015     2.015     2.015 

oneway y grp, noanova sidak bonferroni scheffe

                            Comparison of y by grp
                                (Bonferroni)
Row Mean-|
Col Mean |          1          2          3
---------+---------------------------------
       2 |         .5
         |      1.000
         |
       3 |       1.25        .75
         |      0.608      1.000
         |
       4 |       3.25       2.75          2
         |      0.001      0.005      0.068

                            Comparison of y by grp
                                  (Scheffe)
Row Mean-|
Col Mean |          1          2          3
---------+---------------------------------
       2 |         .5
         |      0.927
         |
       3 |       1.25        .75
         |      0.427      0.794
         |
       4 |       3.25       2.75          2
         |      0.002      0.009      0.085

                            Comparison of y by grp
                                   (Sidak)
Row Mean-|
Col Mean |          1          2          3
---------+---------------------------------
       2 |         .5
         |      0.985
         |
       3 |       1.25        .75
         |      0.474      0.900
         |
       4 |       3.25       2.75          2
         |      0.001      0.005      0.066
       
regress y x1 x2 x3

      Source |       SS       df       MS              Number of obs =      32
-------------+------------------------------           F(  3,    28) =    7.50
       Model |       49.00     3  16.3333333           Prob > F      =  0.0008
    Residual |       61.00    28  2.17857143           R-squared     =  0.4455
-------------+------------------------------           Adj R-squared =  0.3860
       Total |      110.00    31   3.5483871           Root MSE      =   1.476

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
          x1 |       -.25   .3689996    -0.68   0.504    -1.005861    .5058614
          x2 |  -.3333333    .213042    -1.56   0.129    -.7697301    .1030635
          x3 |  -.6666667   .1506435    -4.43   0.000    -.9752458   -.3580875
       _cons |       4.25   .2609221    16.29   0.000     3.715525    4.784475
------------------------------------------------------------------------------ 

xi3: regress y r.grp
r.grp             _Igrp_1-4           (naturally coded; _Igrp_1 omitted)

      Source |       SS       df       MS              Number of obs =      32
-------------+------------------------------           F(  3,    28) =    7.50
       Model |       49.00     3  16.3333333           Prob > F      =  0.0008
    Residual |       61.00    28  2.17857143           R-squared     =  0.4455
-------------+------------------------------           Adj R-squared =  0.3860
       Total |      110.00    31   3.5483871           Root MSE      =   1.476

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     _Igrp_2 |         .5   .7379992     0.68   0.504    -1.011723    2.011723
     _Igrp_3 |          1   .6391261     1.56   0.129    -.3091904     2.30919
     _Igrp_4 |   2.666667   .6025738     4.43   0.000      1.43235    3.900983
       _cons |       4.25   .2609221    16.29   0.000     3.715525    4.784475
------------------------------------------------------------------------------

Some Formulas

Recall the linear model,

The grand mean is the general level of scores,

The treatment effect is the elevation or depression of scores due to the jth treatment,

The error effect is unique to subject i in treatment level j,

The above implies,

From the prediction model (way above),

Partitioning Sums of Squares

Linear Statistical Models Course

Phil Ender, 11apr06, 12Feb98

Level	a1	a2	a3	a4	Total
	4 6 3 3 1 3 2 2	4 5 4 3 2 3 4 3	5 6 5 4 3 4 3 4	3 5 6 5 6 7 8 10
Mean	3.0	3.5	4.25	6.25	4.25
sd	1.51	0.93	1.04	2.12	1.88

Level	a1	a2	a3	a4	Total
	S₁ n=8	S₂ n=8	S₃ n=8	S₄ n=8
Mean	3.0	3.5	4.25	6.25	4.25
sd	1.51	0.93	1.04	2.12	1.88

Source	SS	df	MS	F	p-value
Between Groups	49.0	3	16.333	7.50	0.0008
Within Groups	61.0	28	2.179
Total	110.0	31

Source	SS	df	MS	F	p-value
Treatment	49.0	3	16.333	7.50	0.0008
Error	61.0	28	2.179
Total	110.0	31

	a1	a2	a3	a4
Mean	3.00	3.50	4.25	6.25
Variance	2.29	0.86	1.07	4.50
Std Dev	1.51	0.93	1.04	2.12