Effects of Ignoring Clustered Data Structures in Factor Analysis

In psychiatric research, data for analysis originate principally from two sources: directly from the patients themselves and from interviews conducted by health care professionals. In the latter case, statistical theory indicates that clustering by interviewers or raters needs to be considered when performing any analyses including regression, factor analysis (FA) or item response theory (IRT) modelling of binary or ordinal data. We use simulated data to study the bias of factor analytic estimates and model fit indices when data clustering is fully or partly ignored. Robustness of different estimators, such as maximum likelihood, weighted least squares and Markov chain Monte Carlo is also presented. In the second part, we analyse two real datasets containing responses to the Positive and Negative Syndrome Scale (PANSS) to show the differences when the data are analysed using the correct multilevel approach rather than a traditional aggregated analysis.