Congeneric reliability

For broader coverage of this topic, see Reliability (statistics).

In statistical models applied to psychometrics, congeneric reliability ρ C {\displaystyle \rho _{C}} ("rho C")[1] a single-administration test score reliability (i.e., the reliability of persons over items holding occasion fixed) coefficient, commonly referred to as composite reliability, construct reliability, and coefficient omega.  ρ C {\displaystyle \rho _{C}} is a structural equation model (SEM)-based reliability coefficients and is obtained from a unidimensional model.  ρ C {\displaystyle \rho _{C}} is the second most commonly used reliability factor after tau-equivalent reliability( ρ T {\displaystyle \rho _{T}} ; also known as Cronbach's alpha), and is often recommended as its alternative.

## History and names

[edit]

A quantity similar (but not mathematically equivalent) to congeneric reliability first appears in the appendix to McDonald's 1970 paper on factor analysis, labeled  θ {\displaystyle \theta } .[2] In McDonald's work, the new quantity is primarily a mathematical convenience: a well-behaved intermediate that separates two values.[3][4] Seemingly unaware of McDonald's work, Jöreskog first analyzed a quantity equivalent to congeneric reliability in a paper the following year.[4][5] Jöreskog defined congeneric reliability (now labeled ρ) with coordinate-free notation,[5] and three years later, Werts gave the modern, coordinatized formula for the same.[6] Both of the latter two papers named the new quantity simply "reliability".[5][6] The modern name originates with Jöreskog's name for the model whence he derived  ρ C {\displaystyle \rho _{C}} : a "congeneric model".[1][7][8]

Applied statisticians have subsequently coined many names for  ρ C {\displaystyle {\rho }_{C}} . "Composite reliability" emphasizes that  ρ C {\displaystyle {\rho }_{C}} measures the statistical reliability of composite scores.[1][9] As psychology calls "constructs" any latent characteristics only measurable through composite scores,[10] ρ C {\displaystyle {\rho }_{C}} has also been called "construct reliability".[11] Following McDonald's more recent expository work on testing theory, some SEM-based reliability coefficients, including congeneric reliability, are referred to as "reliability coefficient  ω {\displaystyle \omega } ", often without a definition.[1][12][13]

## Formula and calculation

[edit]

Congeneric measurement model

Congeneric reliability applies to datasets of vectors: each row X in the dataset is a list Xi of numerical scores corresponding to one individual. The congeneric model supposes that there is a single underlying property ("factor") of the individual F, such that each numerical score Xi is a noisy measurement of F. Moreover, that the relationship between X and F is approximately linear: there exist (non-random) vectors λ and μ such that  X i = λ i F + μ i + E i , {\displaystyle X_{i}=\lambda _{i}F+\mu _{i}+E_{i}{\text{,}}} where Ei is a statistically independent noise term.[5]

In this context, λi is often referred to as the factor loading on item i.

Because λ and μ are free parameters, the model exhibits affine invariance, and F may be normalized to mean 0 and variance 1 without loss of generality. The fraction of variance explained in item Xi by F is then simply  ρ i = λ i 2 λ i 2 + V [ E i ] . {\displaystyle \rho _{i}={\frac {\lambda _{i}^{2}}{\lambda _{i}^{2}+\mathbb {V} [E_{i}]}}{\text{.}}} More generally, given any covector w, the proportion of variance in wX explained by F is  ρ = ( w λ ) 2 ( w λ ) 2 + E [ ( w E ) 2 ] , {\displaystyle \rho ={\frac {(w\lambda )^{2}}{(w\lambda )^{2}+\mathbb {E} [(wE)^{2}]}}{\text{,}}} which is maximized when w ∝ 𝔼[EE*]-1λ.[5]

ρC is this proportion of explained variance in the case where w ∝ [1 1 ... 1] (all components of X equally important):  ρ C = ( ∑ i = 1 k λ i ) 2 ( ∑ i = 1 k λ i ) 2 + ∑ i = 1 k σ E i 2 {\displaystyle \rho _{C}={\frac {\left(\sum _{i=1}^{k}\lambda _{i}\right)^{2}}{\left(\sum _{i=1}^{k}\lambda _{i}\right)^{2}+\sum _{i=1}^{k}\sigma _{E_{i}}^{2}}}}

### Example

[edit]

Fitted/implied covariance matrix  |  X 1 {\displaystyle X_{1}} |  X 2 {\displaystyle X_{2}} |  X 3 {\displaystyle X_{3}} |  X 4 {\displaystyle X_{4}}

|
|
|
|

X 1 {\displaystyle X_{1}} |  10.00 {\displaystyle 10.00}
X 2 {\displaystyle X_{2}} |  4.42 {\displaystyle 4.42} |  11.00 {\displaystyle 11.00}
X 3 {\displaystyle X_{3}} |  4.98 {\displaystyle 4.98} |  5.71 {\displaystyle 5.71} |  12.00 {\displaystyle 12.00}
X 4 {\displaystyle X_{4}} |  6.98 {\displaystyle 6.98} |  7.99 {\displaystyle 7.99} |  9.01 {\displaystyle 9.01} |  13.00 {\displaystyle 13.00}
Σ {\displaystyle \Sigma } |  124.23 = Σ d i a g o n a l + 2 × Σ s u b d i a g o n a l {\displaystyle 124.23=\Sigma _{diagonal}+2\times \Sigma _{subdiagonal}}

These are the estimates of the factor loadings and errors:

Factor loadings and errors  |  λ ^ i {\displaystyle {\hat {\lambda }}_{i}} |  σ ^ e i 2 {\displaystyle {\hat {\sigma }}_{e_{i}}^{2}}

|
|

X 1 {\displaystyle X_{1}} |  1.96 {\displaystyle 1.96} |  6.13 {\displaystyle 6.13}
X 2 {\displaystyle X_{2}} |  2.25 {\displaystyle 2.25} |  5.92 {\displaystyle 5.92}
X 3 {\displaystyle X_{3}} |  2.53 {\displaystyle 2.53} |  5.56 {\displaystyle 5.56}
X 4 {\displaystyle X_{4}} |  3.55 {\displaystyle 3.55} |  .37 {\displaystyle .37}
Σ {\displaystyle \Sigma } |  10.30 {\displaystyle 10.30} |  18.01 {\displaystyle 18.01}
Σ 2 {\displaystyle \Sigma ^{2}} |  106.22 {\displaystyle 106.22}

ρ ^ C = ( ∑ i = 1 k λ ^ i ) 2 σ ^ X 2 = 106.22 124.23 = .8550 {\displaystyle {\hat {\rho }}_{C}={\frac {\left(\sum _{i=1}^{k}{\hat {\lambda }}_{i}\right)^{2}}{{\hat {\sigma }}_{X}^{2}}}={\frac {106.22}{124.23}}=.8550}

ρ ^ C = ( ∑ i = 1 k λ ^ i ) 2 ( ∑ i = 1 k λ ^ i ) 2 + ∑ i = 1 k σ ^ e i 2 = 106.22 106.22 + 18.01 = .8550 {\displaystyle {\hat {\rho }}_{C}={\frac {\left(\sum _{i=1}^{k}{\hat {\lambda }}_{i}\right)^{2}}{\left(\sum _{i=1}^{k}{\hat {\lambda }}_{i}\right)^{2}+\sum _{i=1}^{k}{\hat {\sigma }}_{e_{i}}^{2}}}={\frac {106.22}{106.22+18.01}}=.8550}

Compare this value with the value of applying tau-equivalent reliability to the same data.

## Related coefficients

[edit]

Tau-equivalent reliability ( ρ T {\displaystyle \rho _{T}} ), which has traditionally been called "Cronbach's  α {\displaystyle \alpha } ", assumes that all factor loadings are equal (i.e.  λ 1 = λ 2 = . . . = λ k {\displaystyle \lambda _{1}=\lambda _{2}=...=\lambda _{k}} ). In reality, this is rarely the case and, thus, it systematically underestimates the reliability. In contrast, congeneric reliability ( ρ C {\displaystyle \rho _{C}} ) explicitly acknowledges the existence of different factor loadings. According to Bagozzi & Yi (1988),  ρ C {\displaystyle \rho _{C}} should have a value of at least around 0.6.[14] Often, higher values are desirable. However, such values should not be misunderstood as strict cutoff boundaries between "good" and "bad".[15] Moreover,  ρ C {\displaystyle \rho _{C}} values close to 1 might indicate that items are too similar. Another property of a "good" measurement model besides reliability is construct validity.

A related coefficient is average variance extracted.

## References

[edit]

1. ^ a b c d Cho, E. (2016). Making reliability reliable: A systematic approach to reliability coefficients. Organizational Research Methods, 19(4), 651–682. https://doi.org/10.1177/1094428116656239
2. ^ Although McDonald, R. P. (1985). Factor analysis and related methods. Hillsdale, NJ: Lawrence Erlbaum and (1999). Test theory. Mahwah, NJ: Lawrence Erlbaum claim that McDonald 1970 invented congeneric reliability, there is a subtle difference between the formula given there and the modern one. As discussed in Cho & Chun 2018, McDonald's denominator totals observed covariances, but the modern definition divides by the sum of fitted covariances.
3. ^ McDonald, R. P. (1970). Theoretical canonical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23, 1-21. doi:10.1111/j.2044-8317.1970.tb00432.x.
4. ^ a b Cho, E. and Chun, S. (2018), Fixing a broken clock: A historical review of the originators of reliability coefficients including Cronbach’s alpha. Survey Research, 19(2), 23–54.
5. ^ a b c d e Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109–133. https://doi.org/10.1007/BF02291393
6. ^ a b Werts, C. E., Linn, R. L., & Jöreskog, K. G. (1974). Intraclass reliability estimates: Testing structural assumptions. Educational and Psychological Measurement, 34, 25–33. doi:10.1177/001316447403400104
7. ^ Graham, J. M. (2006). Congeneric and (Essentially) Tau-Equivalent Estimates of Score Reliability What They Are and How to Use Them. Educational and Psychological Measurement, 66(6), 930–944. https://doi.org/10.1177/0013164406288165
8. ^ Lucke, J. F. (2005). “Rassling the Hog”: The Influence of Correlated Item Error on Internal Consistency, Classical Reliability, and Congeneric Reliability. Applied Psychological Measurement, 29(2), 106–125. https://doi.org/10.1177/0146621604272739
9. ^ Werts, C. E., Rock, D. R., Linn, R. L., & Jöreskog, K. G. (1978). A general method of estimating the reliability of a composite. Educational and Psychological Measurement, 38(4), 933–938. https://doi.org/10.1177/001316447803800412
10. ^ Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
11. ^ Hair, J. F., Babin, B. J., Anderson, R. E., & Black, W. C. (2018). Multivariate data analysis (8th ed.). Cengage.
12. ^ Padilla, M. (2019). A Primer on Reliability via Coefficient Alpha and Omega. Archives of Psychology, 3(8), Article 8. https://doi.org/10.31296/aop.v3i8.125
13. ^ Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. https://doi.org/10.1007/s11336-008-9102-z
14. ^ Bagozzi & Yi (1988), https://dx.doi.org/10.1177/009207038801600107
15. ^ Guide & Ketokivi (2015), https://dx.doi.org/10.1016/S0272-6963(15)00056-X

## External links

[edit]

* RelCalc, tools to calculate congeneric reliability and other coefficients.
* Handbook of Management Scales, Wikibook that contains management related measurement models, their indicators and often congeneric reliability.