In Cooperation with:

American Society for Quality Statistics Division

American Statistical Association

Bernoulli Society for Mathematical Statistics and Probability

Institute of Mathematical Statistics

International Biometric Society

International Chinese Statistical Association

International Society for Bayesian Analysis

International Statistical Institute

Royal Statistical Society

Statistical Society of Canada / Société statistique du Canada

Generalized Quasi-likelihood (GQL) Inference

GeneralizedQuasiLikelihoodGQLInferences

Generalized Quasi-likelihood (GQL) Inference*
by Brajendra C. Sutradhar
Memorial University
Email address: bsutradh@mun.ca

QL Estimation for Independent Data. For $i=1,\ldots,K,$ let denote the response variable for the th individual, and $x_i=(x_{i1},\ldots,x_{iv},\ldots,x_{ip})'$ be the associated dimensional covariate vector. Also, let $\beta$ be the dimensional vector of regression effects of on Further suppose that the responses are collected from independent individuals. It is understandable that if the probability distribution of is not known, then one can not use the well known likelihood approach to estimate the underlying regression parameter $\beta.$ Next suppose that only two moments of the data, that is, the mean and the variance functions of the response variable for all $i=1,\ldots,K,$ are known, and for a known functional form $a(\cdot)$ , these moments are given by

$\displaystyle E[Y_i]=a'(\theta_i)\;$ and $\displaystyle \;$ var $\displaystyle [Y_i]=a''(\theta_i),$

(1)

where for a link function $h(\cdot),$ $\theta_i=h(x'_i\beta),$ and $a'(\theta_i)$ and $a''(\theta_i)$ are the first and second order derivatives of $a(\theta_i),$ respectively, with respect to $\theta_i.$ For the estimation of the regression parameter vector $\beta$ under this independence set up, Wedderburn (1974) (see also McCullagh (1983)) proposed to solve the so-called quasi-likelihood (QL) estimating equation given by

$\displaystyle \sum^K_{i=1}[\frac{\partial a'(\theta_{i})}{\partial \beta}\frac{(y_{i}-a'(\theta_{i}))}{a''(\theta_i)}]=0.$

(2)

Let $\hat{\beta}_{QL}$ be the QL estimator of $\beta$ obtained from (2). It is known that this estimator is consistent and highly efficient. In fact, for Poisson and binary data, for example, $\hat{\beta}_{QL}$ is equivalent to the maximum likelihood (ML) estimator and hence it turns out to be an optimal estimator.

Illustration for the Poisson case: For the Poisson data, one uses

$\displaystyle a(\theta_{i})=\exp(\theta_{i})$

(3)

with identity link function $h(\cdot),$ that is, $\theta_{i}=x'_{i}\beta.$ This gives the mean and the variance functions as

var $\displaystyle (Y_{i})=a''(\theta_{i})=E(Y_i)=a'(\theta_{i})=\mu_{i}\;$ (say) $\displaystyle =\exp(x'_{i}\beta),$

yielding by (2), the QL estimating equation for $\beta$ as

$\displaystyle \sum^K_{i=1}x_i(y_i-\mu_i)=0.$

(4)

Note that as the Poisson density is given by $f(y_i\vert x_i)=\frac{1}{y_i!}\exp[y_ilog(\mu_i)-\mu_i],$ with $\mu_i=\exp(\theta_i)=\exp(x'_i\beta),$ it follows that the log likelihood function of $\beta$ has the form log $L(\beta)=-\sum^K_{i=1}log(y_i!)+\sum^K_{i=1}[y_{i}\theta_{i}-a(\theta_{i})],$ yielding the likelihood equation for $\beta$ as

$\displaystyle \frac{\partial \mbox{log} L}{\partial \beta}=\sum^K_{i=1}[y_{i}-a... ...a_{i})]\frac{\partial \theta_{i}}{\partial \beta}=\sum^K_{i=1}x_i(y_i-\mu_i)=0,$

(5)

which is the same as the QL estimating equation (4). Thus, if the likelihood function were known, then the ML estimate of $\beta$ would be the same as the QL estimate $\hat{\beta}_{QL}.$

Illustration for the binary case: For the binary data, one uses

$\displaystyle a'(\theta_{i})=\frac{\exp(\theta_{i})}{1+\exp(\theta_{i})}=\mu_i\;$ and $\displaystyle \;a''(\theta_i)=\mu_i(1-\mu_i),$

(6)

with $\theta_i=x'_i\beta.$ The QL estimating equation (2) for the binary data, however, provides the same formula (4) as in the Poisson case, except that now for the binary case $\mu_i= \frac{\exp(\theta_{i})}{1+\exp(\theta_{i})},$ whereas for the Poisson case $\mu_i=\exp(\theta_i).$

As far as the ML estimation for the binary case is concerned, one first writes the binary density given by $f(y_i\vert x_i)={\mu_i}^{y_i}(1-\mu_i)^{1-y_i}.$ Next by writing the log likelihood function as log $L(\beta)=\sum^K_{i=1}y_i\mu_i+\sum^K_{i=1}(1-y_i)(1-\mu_i),$ one obtains the same likelihood estimating equation as in (5), except that here $\mu_i= \frac{\exp(x'_i\beta)}{1+\exp(x'_i\beta)},$ under the binary model. Since the QL estimating equation (4) is the same as the ML estimating equation (5), it then follows that the ML and QL estimates for $\beta$ would also be the same for the binary data.

GQL Estimation: A Generalization of the QL Estimation to the Correlated Data.

As opposed to the independence set up, we now consider as a vector of repeated binary or count responses, collected from the th individual, for all $i=1,\ldots,K.$ Let $y_i=(y_{i1},\ldots,y_{it},\ldots,y_{iT})',$ where $y_{it}$ represents the response recorded at time for the th individual. Also, let $x_{it}=(x_{it1},\ldots,x_{itv},\ldots,x_{itp})'$ be the dimensional covariate vector corresponding to the scalar $y_{it},$ and $\beta$ be the dimensional regression effects of $x_{it}$ on $y_{it}$ for all $i=1,\ldots,K,$ and all $t=1,\ldots,T.$ Suppose that $\mu_{it}$ and $\sigma_{itt}$ be the mean and the variance of $Y_{it},$ that is $\mu_{it}=E[Y_{it}]$ and var $[Y_{it}]=\sigma_{itt}.$ Note that both $\mu_{it}$ and $\sigma_{itt}$ are functions of $\beta.$ But, when the variance is a function of mean, it is sufficient to estimate $\beta$ involved in the mean function only, by treating $\beta$ involved in the variance function to be known. Further note that since the repeated responses of an individual are likely to correlated, the estimate of $\beta$ to be obtained by ignoring the correlations, that is, the solution of the independence assumption based QL estimating equation

$\displaystyle \sum^K_{i=1}\sum^T_{t=1}[\frac{\partial \mu_{it}}{\partial \beta}\frac{(y_{i}-\mu_{it})}{\sigma_{itt}}]=0,$

(7)

for $\beta,$ will be consistent but inefficient. As a remedy to this inefficient estimation problem, Sutradhar (2003) has proposed a generalization of the QL estimation approach, where $\beta$ is now obtained by solving the GQL estimating equation given by

$\displaystyle \sum^K_{i=1} \frac{\partial \mu'_i}{\partial \beta}{\Sigma_i}^{-1}(\rho )(y_i-\mu_i)=0,$

(8)

where $\mu_i=(\mu_{i1},\ldots,\mu_{it},\ldots,\mu_{iT})'$ is the mean vector of

and $\Sigma_i(\rho)$ is the covariance matrix of

that can be expressed as $\Sigma_i(\rho )=A^{\frac{1}{2}}_iC_i(\rho )A^{\frac{1}{2}}_i$ , with

diag $[\sigma_{i11},\ldots,\sigma_{itt},\ldots,\sigma_{iTT}]$ and $C_i(\rho)$ as the correlation matrix of

$\rho$ being a correlation index parameter.

Note that the use of the GQL estimating equation (8) requires the structure of the correlation matrix $C_i(\rho)$ to be known, which is, however, unknown in practice. To overcome this difficulty, Sutradhar (2003) has suggested a general stationary auto-correlation structure given by

$\displaystyle C_i(\rho)=\left[ \begin{array}{ccccc} 1 & \rho_1 & \rho_2 & \cdot... ...s \\ \rho_{T-1} & \rho_{T-2} & \rho_{T-3} & \cdots & 1 \\ \end{array} \right] ,$

(9)

(see also Sutradhar and Das (1999, Section 3)), for all $i=1,\ldots,K,$ where for $\ell=1,\ldots,T-1,$ $\rho_\ell$ represents the lag $\ell$ auto-correlation. As far as the estimation of the lag correlations is concerned, they may be consistently estimated by using the well known method of moments. For $\ell =\vert u-t\vert$ , $u\neq t$ , $u, t=1,\ldots ,T$ , the moment estimator for the autocorrelation of lag $\ell$ , $\rho_{\ell}$ , has the formula

$\displaystyle \hat{\rho}_\ell = \frac{\sum^K_{i=1}\sum^{T-\ell}_{t=1}\tilde{y}_... ...tilde{y}_{i,t+\ell} /K(T-\ell )}{\sum^K_{i=1}\sum^T_{t=1}\tilde{y}^2_{it}/KT} ,$

(10)

(Sutradhar and Kovacevic (2000, eqn. (2.18), Sutradhar (2003)), where $\tilde{y}_{it}$ is the standardized residual, defined as $\tilde{y}_{it}=(y_{it}-\mu_{it})/\{\sigma_{itt} \}^{\frac{1}{2}}$ .

The GQL estimating equation (8) for $\beta$ and the moment estimate of $\rho_\ell$ by (10) are solved iteratively until convergence. The final estimate of $\beta$ obtained from this iterative process is referred to as the GQL estimate of $\beta,$ and may be denoted by $\hat{\beta}_{GQL}.$ This estimator $\hat{\beta}_{GQL}$ is consistent for $\beta$ and also highly efficient, the ML estimator being fully efficient which is however impossible or extremely complex to obtain in the correlated data set up.

With regard to the generality of the stationary auto-correlation matrix $C_i(\rho)$ in (9), one may show that this matrix, in fact, represents the correlations of many stationary dynamic such as stationary auto-regressive order 1 (AR(1)), stationary moving average order 1 (MA(1)), and stationary equi-correlations (EQC) models. For example, consider the stationary AR(1) model given by

$\displaystyle y_{it}=\rho * y_{i,t-1}+d_{it},$

(11)

(McKenzie (1988), Sutradhar (2003)) where it is assumed that for given $y_{i,t-1}$ , $\rho * y_{i,t-1}$ denotes the so-called binomial thinning operation (McKenzie, 1988). That is,

$\displaystyle \rho * y_{i,t-1} = \sum^{y_{i,t-1}}_{j=1}b_j(\rho ) = z_{i,t-1}, {\mbox{say}},$

(12)

with $\Pr [b_j(\rho )=1]=\rho$ and $\Pr [b_j(\rho )=0]=1-\rho$ . Furthermore, it is assumed in (11) that $y_{i1}$ follows the Poisson distribution with mean parameter $\mu_{i\cdot},$ that is, $y_{i1}\sim Poi(\mu_{i\cdot}),$ where $\mu_{i\cdot}=\exp(x'_{i\cdot}\beta)$ with stationary covariate vector $x_{i\cdot}$ such that $x_{it}=x_{i\cdot}$ for all $t=1,\ldots,T.$ Further, in (11), $d_{it} \sim P(\mu_{i\cdot}(1-\rho ))$ and is independent of $z_{i,t-1}.$ This model in (11) yields the mean, variance and auto-correlations of the data as shown in Table 1. The Table 1 also contains the MA(1) and EQC models and their basic properties including the correlation structures.

Table 1. A class of stationary correlation models for longitudinal count data and basic properties.

Model	Dynamic relationship	Mean-variance
		& Correlations
AR(1)	$y_{it}=\rho * y_{i,t-1}+d_{it}, t=2,\ldots$	$E[Y_{it}]=\mu_{i\cdot}$
	$y_{i1}\sim Poi(\mu_{i\cdot})$	var $[Y_{it}]=\mu_{i\cdot}$
	$d_{it} \sim P(\mu_{i\cdot}(1-\rho )), t=2,\ldots$	corr $[Y_{it},Y_{i,t+\ell}]=\rho_{\ell}$
		$=\rho^{\ell}$
MA(1)	$y_{it}=\rho * d_{i,t-1}+d_{it}, t=2,\ldots$	$E[Y_{it}]=\mu_{i\cdot}$
	$y_{i1}=d_{i1} \sim Poi(\mu_{i\cdot}/(1+\rho))$	var $[Y_{it}]=\mu_{i\cdot}$
	$d_{it} \sim P(\mu_{i\cdot}/(1+\rho )), t=2,\ldots$	corr $[Y_{it},Y_{i,t+\ell}]=\rho_{\ell}$
		$= \left\{ \begin{array}{ll} \frac{\rho}{1+\rho} & \mbox{for } \ell=1\ 0 & \mbox{otherwise}, \end{array} \right.$
EQC	$y_{it}=\rho * y_{i1}+d_{it}, t=2,\ldots$	$E[Y_{it}]=\mu_{i\cdot}$
	$y_{i1}\sim Poi(\mu_{i\cdot})$	var $[Y_{it}]=\mu_{i\cdot}$
	$d_{it} \sim P(\mu_{i\cdot}(1-\rho )), t=2,\ldots$	corr $[Y_{it},Y_{i,t+\ell}]=\rho_{\ell}$
		$=\rho$

It is clear from Table 1 that the correlation structures for all three processes can be represented by $C_i(\rho)$ in (9). By following Qaqish (2003), one may write similar but different dynamic models for the repeated binary data, with their correlation structures represented by $C_i(\rho).$ Thus, if the count or binary data follow this type of auto-correlations model, one may then certainly estimate the regression vector consistently and efficiently by solving the general auto-correlations matrix based GQL estimating equation (8), where the lag correlations are estimated by (10) consistently.

[* Reprinted with permission from Lovric, Miodrag (2011), International Encyclopedia of Statistical Science. Heidelberg: Springer Science & Business Media, LLC]

References

McCullagh, P. (1983). Quasilikelihood functions. Ann. Statist. 11, 59-67.

McKenzie, E. (1988). Some ARMA models for dependent sequences of Poisson counts. Advances in Applied Probability 20, 822-35.

Qaqish, B. F. (2003). A family of multivariate binary distributions for simulating correlated binary variables with specified marginal means and correlations. Biometrika 90, 455-463.

Sutradhar, B. C. (2003). An overview on regression models for discrete longitudinal responses. Statistical Science 18, 377-93.

Sutradhar, B. C. & Das, K. (1999). On the efficiency of regression estimators in generalized linear models for longitudinal data. Biometrika 86, 459-65.

Sutradhar, B. C. & Kovacevic, M. (2000). Analyzing ordinal longitudinal survey data: Generalized estimating equations approach. Biometrika 87, 837-848.

Wedderburn, R. W. M. (1974). Quasi-likelihood functions, generalised linear models, and the Gauss-Newton method. Biometrika 61, 439-447.

Generalized Quasi-likelihood (GQL) Inference is owned by Brajendra C Sutradhar.

Note

NOTE ON THE PROPER CITATION OF THIS ARTICLE: The proper citation for this article is Brajendra Sutradhar, "Generalized Quasi-likelihood (GQL)Inference." In: Lovric, Miodrag (Editor), (2010). International Encyclopedia of Statistical Science. Heidelberg: Springer Science +Business Media, LLC, reprinted and freely available at StatProb: The Encyclopedia Sponsored by Statistics and Probability Societies.

How to Cite This Entry

Brajendra C Sutradhar. "Generalized Quasi-likelihood (GQL) Inference" (version 8). StatProb: The Encyclopedia Sponsored by Statistics and Probability Societies. Freely available at http://statprob.com/encyclopedia/GeneralizedQuasiLikelihoodGQLInferences.html

Classification

AMS MSC:	62F10 (Statistics :: Parametric inference :: Point estimation)
	62H20 (Statistics :: Multivariate analysis :: Measures of association )

Pending Errata and Addenda

None.

Discussion

No messages.

Username:
Password:

Register
I've forgotten my login details

Login