Login
In Cooperation with:

American Society for Quality Statistics Division

American Statistical Association

Bernoulli Society for Mathematical Statistics and Probability

Institute of Mathematical Statistics

International Biometric Society

International Chinese Statistical Association

International Society for Bayesian Analysis

International Statistical Institute

Royal Statistical Society

Statistical Society of Canada / Société statistique du Canada
Random Coefficient Models
ref
Random Coefficient Models 1Nicholas T. Longford SNTL and UPF, Barcelona, Spain SummaryRandom coefficient models are intended for settings with two or more sources of random variation. The widest range of applications is found for them when observational units form natural clusters, such that the units within a cluster are more similar than units in general. Models for independent observations have to be extended to allow for within- and between-cluster variation.Keywords: Analysis of variance; borrowing strength; clusters; correlation structure; empirical Bayes analysis; longitudinal analysis; maximum likelihood; ordinary regression. Independence of the observations is a key assumption of many standard statistical methods, such as analysis of variance (ANOVA) and ordinary regression, and some of its extensions. Common examples of data structures that do not fit into such a framework arise in longitudinal analysis, in which observations are made on subjects at subject-specific sequences of time points, and in studies that involve subjects (units) ocurring naturally in clusters, such as individuals within families, schoolchildren within classrooms, employees within companies, and the like. The assumption of independence of the observational units is not tenable when observations within a cluster tend to be more similar than observations in general. Such similarity can be conveniently represented by a positive correlation (dependence). This article describes an adaptation of the ordinary regression for clustered observations. Such observations require two indices, one for elements within clusters, with the usual assumptions of normality, independence and equal variance (homoscedasticity) of the deviations where In the model in (2), the within-cluster regressions are parallel -- their intercepts are ![]() because they share the same deviation where We refer to and this quadratic function has a unique minimum at
The model in (3) is fitted by maximum likelihood (ML) which maximizes the log-likelihood function where The Fisher scoring algorithm for maximising the log-likelihood in (5) is described in the Appendix; for details and applications, see see Longford (1993), and for an alternative method Goldstein (2000). These and other algorithms are implemented in most standard statistical packages. A key to their effective implementation are closed-form expressions for the inverse and determinant of patterned matrices (Harville, 1997). Model selection entails two tasks, selecting a set of variables to form Random coefficients can be applied to a range of models much wider than ordinary regression. In principle, we can conceive any basis model, characterized by a vector of parameters, which applies to every cluster. A subset of these parameters is constant across the clusters and the remainder varies according to a model for cluster-level variation. The latter model need not be a multivariate normal distribution, although suitable alternatives to it are difficult to identify. The basis model itself can be complex, such as a random coefficient model itself. This gives rise to three- or, generally, multilevel models, in which elements are clustered within two-level units, these units in three-level units, and so on. Generalized linear mixed models (GLMM) have generalized linear models (McCullagh and Nelder, 1989) as their basis; see Pinheiro and Bates (2000). For cluster with a (monotone) link function Without conditioning, the likelihood for non-normally distributed Random coefficient models are well suited for analysing surveys in which clusters arise naturally as a consequence of the organisation (design) of the survey and the way the studied population is structured. They can be applied also in settings in which multiple observations are made on subjects, as in longitudinal studies (Molenberghs and Verbeke, 2000). In some settings it is contentious as to whether the clusters should be regarded as fixed or random. For example, small-area estimation (Rao, 2003) is concerned with inferences about districts or another partition of a country when some (or all) districts are represented in the analysed national survey by small subsamples. In one perspective, district-level quantities, such as their means of a variable, should be regarded as fixed because they are the inferential targets, fixed across hypothetical replications. When they are assumed to be random the (random coefficient) models are often more parsimonious than their fixed-effects (ANCOVA) counterparts, because the number of parameters involved does not depend on the number of clusters. Borrowing strength (Robbins, 1955, Efron and Morris, 1972) is a general principle for efficient inference about each cluster (district) by exploiting the similarity of the clusters. It is the foundation of the empirical Bayes analysis, in which the between-cluster variance matrix plays a role similar to the Bayes prior for the within-cluster regression coefficients. The qualifier `empirical' refers to using a data-based estimator ReferencesEfron, B., and Morris, C. N. (1972). Limiting the risk of Bayes and empirical Bayes estimators -- Part II: empirical Bayes case. Journal of the American Statistical Association 67, 1286-1289. Goldstein, H. (2000). Multilevel Statistial Models. 2nd Edition. London: Edward Arnold. Harville, D. (1997). Matrix Algebra from a Statistician's Perspective. New York: Springer-Verlag. Lee, Y., and Nelder, J. A. (2001). Hierarchical generalised linear models: a synthesis of generalised linear models, random effect models and structured dispersions. Biometrika 88, 987-1004. Longford, N. T. (1993). Random Coefficient Models. Oxford: Oxford University Press. Magnus, J. R., and Neudecker, H. (1988). Matrix Differential Calculus with Applications in Statistics and Econometrics. New York: Wiley. McCullagh, P., and Nelder, J. A. (1989). Generalized Linear Models. 2nd Edition. London: Chapman and Hall. Pinheiro, J. C., and Bates, D. M. (2000). Mixed-Effects Models in S and Splus. New York: Springer-Verlag. Rao, J. N. K. (2003). Small Area Estimation. New York: Wiley and Sons. Robbins, H. (1955). An empirical Bayes approach to statistics. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1, 157-164. Berkeley, CA: University of California Press. Verbeke, G., and Molenberghs, G. (2000). Linear Mixed Models for Longitudinal Data. New York: Springer-Verlag. Appendix. Fisher scoring algorithmThis Appendix describes a method for fitting a random coefficient model by maximum likelihood. We prefer to use the scaled variance matrices where We have the following closed-form expressions for the inverse and determinant of where Assuming that the log-likelihood ![]() and so the maximum likelihood estimator of The matrices The elementary-level (residual) variance ![]() which is ![]() The elements of Let and ![]() where so we do not have to form the matrices 5pt ![]() for ![]() which are cluster-level totals of the cross-products of various elements of where
Based on an article from Lovric, Miodrag (2011), International Encyclopedia of Statistical Science. Heidelberg: Springer Science + Business Media, LLC. Footnotes
|










