Plan Of Investigation

We consider maximum likelihood estimation for the normal-normal model, equations (2) and (3), as well as for the normal-t model, equations (2) and (4). The estimation procedure is based on a reliable variable-metric algorithm which supports linear equality constraints on point estimators. Estimation of the normal-multivariate t model involves numerical integration by a one-dimensional Gauss-Laguerre quadrature. In all cases, exact standard errors are computed from analytic expressions of the Hessian evaluated at the minima (again, by an approximation for models with a multivariate t prior). We also consider estimation procedures for the model when some outcomes are missing at random.

As a multivariate two-stage model, the MHLM adds to several previous formulations that have appeared in Goldstein and McDonald (1988), Muthén (1989), and Schmidt (1969). More recently, McDonald and Goldstein (1989) and Longford and Muthén (1992) attempt to restate the general LISREL model (Jöreskog & Sörbom, 1984) and Schmidt's (1969) early results for multivariate random effects models for the unbalanced case. Another approach to multivariate multi-level data can be formulated in terms of a univariate three-stage model, as it should be obvious that a three-stage model with no residual variance at any one level is formally equivalent to a two-stage model. As such, the normal-normal formulation of the MHLM is a straightforward simplification of any three-stage hierarchical linear model, such as that of Raudenbush and Bryk (1986). Note, however, that while a three-stage model is multivariate by virtue of an additional level of variation, the MHLM is multivariate in the usual sense that its Stage 1 model is simply the familiar general linear model with a less restrictive error structure.

To assess the usefulness of our two-stage approach in understanding behavioral processes, we begin in Chapter 5 with three illustrative applications of the MHLM emphasizing three common characteristics of behavioral data. To recapitulate briefly, behavior is first of all invariably multivariate in its conceptualization and communication. Separate univariate analyses of related outcome variables are fraught with potential interpretive blindspots for the researcher. This practice also suffers, from an inferential standpoint, because it fails to take advantage of any redundant information in the outcomes. Second, studies of behavior, especially in experimental research, employ smaller samples. This situation raises issues of robustness of inference with respect to outlying individuals. Third, the outcome variable may have observations missing because of accidents or by design. The model permits the estimation of the full spectrum of plausible measurement error structures while using all the available information. Maximum likelihood estimates are obtained, in the context of these examples, for various members of the multivariate hierarchical linear model (MHLM).

Chapter 6 discusses the first of two applications in psychology. We use the MHLM to isolate a generalized learning curve in order to demonstrate its advantages as a formal means for aggregating individual models. Using data from Tucker (1966), the interpretation afforded by this model provides a interesting comparison to the exploratory factor analytic approach for determining generalized learning curves for probabilistic learning. The MHLM analysis for this problem is essentially a growth curve analysis with heteroscadestic errors.

In another application, we use the MHLM to separate method from trait effects and gauge their variability in the Kelley and Fiske (1951) test validation data in Chapter 7. We compare these results with a covariance structure analysis by Bock and Bargmann (1966) of test validation data in the form of the multitrait-multimethod (MTMM) matrix. Against a backdrop of inconclusive analyses for the bulk of MTMM data, and for the Kelley-Fiske data in particular, fixed-scale analyses of the raw data using MHLMs give sensible estimates for a main-effects model with method-trait inter-correlations as well as a model with two-way interactions. We draw an important methodological lesson for this line of reseach. We question the routine choice of the sample second moments, correlations or covariances, as the finest grade of input data for covariance components analysis. We contend that currently accepted methods for such data which are based on fitting the observed sample variance-covariance matrix by maximum Wishart likelihood are restricted in the level of inferential detail.

Finally, Chapter 8 provides an evaluation of the efficacy of our two-stage conception of behavioral phenomena as it is applied to the above research problems in psychology.

SELECTED REFERENCES

Bock, R. D. (1989) Addendum - Measurement of human variation: A two-stage model. In R. D. Bock (Ed.) Multilevel Analysis of Educational Data (pp. 319-342). New York: Academic Press.

Bock, R. D. & Bargmann, R. E. (1966) Analysis of covariance structures. Psychometrika, 31, 507-534.

Brower, D. (1949) The problem of quantification in psychological science. Psychological Review, 58, 325-333.

Bryk, A. S. & Raudenbush, S. W. (1987) Application of hierarchical linear models to assessing change. Psychological Bulletin, 104, 147-158.

Bryk, A. S. & Raudenbush, S. W. (1988) Toward a more appropriate conceptualization of research on school effects: A three-level hierarchical linear model. American Journal of Education, 95, 65-108.

Burstein, L. (1980) The analysis of multi-level data in educational research and evaluation. Review of Research in Education, 8, 158-233.

Camerer, C. (1981) General conditions for the success of bootstrapping models. Organizational Behavior and Human Performance, 27, 411-422.

Cattell, R. B. (1952) Factor Analysis. New York: Harper.

Cronbach, L. J. (1957) The two disciplines of scientific psychology. American Psychologist, 12, 671-684.

Cronbach, L. J. & Snow, R. E. (1977) Aptitudes and Instructional Methods. New York: Irvington.

Dawes, R. M., Faust, D., & Meehl, P. E. (1989) Clinical versus actuarial judgement. Science, 243, 1668-1674.

DeLeeuw, J. & Kreft, I. (1986) Random coefficient models for multilevel analysis. Journal of Educational Statistics, 11, 57-85.

Fiske, D. W. (1979) Two worlds of psychological phenomena. American Psychologists, 34, 733-739.

Goldstein, H. (1987) Multilevel Models in Education and Social Research. London: Oxford University Press.

Goldstein, H. & McDonald, R. P. (1988) A general model for the analysis of multilevel data. Psychometrika, 53, 455-467.

Gulliksen, H. (1974) Looking back and ahead in psychometrics. American Psychologist, 29, 251-261.

Hull, C. L. (1945) The place of innate individual and species differences in a natural-science theory of behavior. Psychological Review, 52, 55-60.

Jöreskog, K. G. & Sörbom, D. (1984) LISREL VI: Analysis of linear structural relationships by maximum likelihood, instrumental variables and least squares methods. Mooresville, IN.: Scientific Software.

Kelly, E. L. & Fiske, D. W. (1951) The Prediction of Performance in Clinical Psychology. Ann Arbor: University of Michigan Press.

Longford, N. T. (1987) A fast scoring algorithm for maximum likelihood estimation in unbalanced mixedmodels with nested effects. Biometrika, 74, 817-827.

Longford, N. T. & Muthén, B. O. (1992) Factor analysis for clustered observations. Psychometrika, 57,581-597.

McDonald, R. P. & Goldstein, H. (1989) Balanced versus unbalanced designs for linear structural relations in two-level data. British Journal of Mathematical and Statistical Psychology, 42, 215-232.

Mischel, W. (1968) Personality and assessment. New York: Wiley.

Muthén, B. (1989) Latent variable modelling in heterogeneous populations. Psychometrika, 54, 557-585.

Ogilvie, J. R. & Schmitt, N. (1979) Situational influences on linear and nonlinear use of information. Organizational Behavior and Human Performance, 23, 292-306.

Piaget, J. (1950) Psychology of intelligence. M. Piercy & D. E. Berlyne (Trans.). London: Routledge and Kegan Paul.

Phillips, B. N. (1985) New directions in aptitude-treatment interaction research. In C. R. Reynolds & V. L. Willson (Eds.) Methodological and Statistical Advances in the Study of Individual Differences, pp. 241-273. New York: Plenum Press.

Raudenbush, S. W. & Bryk, A. S. (1986) A hierarchical Model for studying school effects. Sociology of Education, 59, 1-17.

Raudenbush, S. W. (1988) Educational applications of hierarchical models: a review. Journal of Educational Statistics, 13, 85-116.

Raudenbush, S. W., Rowan, B. & Kang, S. J. (1991) A multilevel, multivariate model for studying school climate with estimation via the EM algorithm and application to U. S. high-school data. Journal of Educational Statistics, 16, 295-330.

Reinsel, G. (1984) Estimation and prediction in a multivariate effects generalized linear model. Journal of the American Statistical Association, 79, 406-414.

Rogosa, D. R. & Ghandour, G. (1991) Statistical models for behavioral observations. Journal of Educational Statistics, 16, 157-252.

Rogosa, D. R. & Willett, J. B. (1985a) Understanding correlates of change by modelling individual differences in growth. Psychometrika, 50, 203-228.

Schmidt, W. H. (1969) Covariance structure analysis of the multivariate random effects model. Unpublished doctoral dissertation, University of Chicago.

Seltzer, M. H. (1993) Sensitivity analysis for fixed effects in hierarchical models: A Gibbs sampling approach. Journal of Educational Statistics, 18, 207-235

Shweder, R. A. (1975) How relevant is an individual difference theory of personality? Journal of personality, 43, 455-485.

Sternberg, R. J. (1985) Applying componential theory to the study of individual differences in cognitive styles. In C. R. Reynolds & V. L. Willson (Eds.) Methodological and Statistical Advances in the Study of Individual Differences, pp. 209-239. New York: Plenum Press.

Tucker, L. R. (1964) Learning theory and multivariate experiment: illustration by determination of generalized learning curves. In R. B. Cattell (Ed.) Handbook of Multivariate Psychology, pp. 476-501. Chicago: Rand McNally & Company.

Underwood, B. J. (1975) Individual differences as a crucible in theory construction. American Psychologist, 12, 128-134.

updated