I. J. of Business & Management Research
Research Article | 
Factor Analysis: Dealing with Response Bias
Author(s) : Robert Goedegebuure1*, Manorama Adhikari2
Publisher : FOREX Publication
Published : 20 February 2023
e-ISSN :2347-4696
Page(s) : 25-33
Abstract
This paper proposes an innovative method for factor analyzing data that potentially contains individual response bias. Past methods include the use of “ipsative” data, or, related to that, “ipsatized” data. Unfortunately, factor analysis as the main method used for analyzing the dimensionality of data, cannot be applied to ipsative data. In contrast, normalization of data as an alternative method to filter out response bias, is not hampered by the technical statistical issues inherent to applying multivariate techniques to ipsative data. Using high-quality data from a survey in Nepal that makes use of – among others – the High-Performance Organizations (HPO) framework, this paper shows that the traditional approach of directly applying Confirmatory Factor Analysis (CFA) starting from an existing model or theory, is inferior to our approach. Even applying Exploratory Factor Analysis (EFA) to the raw (non-normalized) data before using CFA, is unable to detect the optimal dimensionality, or structure, in the data. A better structure can be obtained by performing EFA on normalized data that corrects for response bias in the raw data. This paper convincingly shows that the newly identified structure is superior to the original structure suggested by the HPO framework. Applying a CFA using the newly detected structure on the raw data, gives excellent goodness-of-fit statistics, with more items retained, and no need of forced methods to improve the model fit. The findings suggest that existing models and questionnaires based on these models, are not necessarily as valid and reliable as empirical studies that make use of traditional analyses seem to suggest. When adopting existing instruments, researchers are advised to critically check the validity and reliability of these instruments – especially those vulnerable to response bias - and to apply the procedures laid out in this paper, in order to enhance the quality of their research, and to inform future researchers who consider using the same instruments or to warn them about the potential shortcomings of these instruments.
Keywords: Exploratory Factor Analysis, Confirmatory Factor Analysis, Ipsative Data, Ipsatized Data,Normalized Data,Response Bias,High-Performance Organizations.
Robert Goedegebuure*, Professor and Research Director at Swiss School of Management Research Center; Email: robert.goedegebuure@ssmresearch.com
Manorama Adhikari , Deputy Chief of Party in Monitoring, Evaluation, and Learning Project of USAID in Nepal
[1] Baron, H. (1996). Strengths and limitations of ipsative measurement. Journal of Occupational and Organizational Psychology, 69, 49-56.
[2] Bowen, C.-C., Martin, B. A., & Hunt, S. T. (2002). A comparison of ipsative and normative approaches for ability to control faking in personality questionnaires. The International Journal of Organizational Analysis, 10, 240-259.
[3] Cerny, C.A., & Kaiser, H.F. (1977). A study of a measure of sampling adequacy for factor-analytic correlation matrices. Multivariate Behavioral Research, 12(1), 43-47.
[4] Chan, W. & Bentler, P. M. (1993). Covariance structure analysis of ipsative data. Sociological Method and Research, 22, 214-247.Closs, S. J. (1996). On the factoring and interpretation of ipsative data. Journal of Occupational and Organizational Psychology, 69, 41-47 . [Cross Ref]
[5] Closs, S.J. (1976). Ipsative vs. Normative Interpretation of Interest Test Scores or “What Do You Mean by Like”. Bulletin of the British Psychological Society, 28, 289-299
[6] Closs, S.J. (1996). On the Factoring and Interpretation of Ipsative Data. Journal of Occupational and Organizational Psychology, 69, 41-47
[7] Chan, W. (2003). Analyzing ipsative data in psychological research. Behaviormetrika, Vol. 30, No. 1, 99-121 [Cross Ref]
[8] Cornwell, J.M. & Dunlap, W.P. (1994). On the Questionable soundness of Factoring Ipsative Data: A Response to Saville & Wilson. Journal of Occupational and Organizational Psychology, 67, 89-100 [Cross Ref]
[9] Cunningham, W.H., Cunningham, I.C. M. & Green R.T. (1977). The Ipsative Process to Reduce Response Set Bias, The Public Opinion Quarterly. Vol. 41, No. 3 (Autumn), 379-384 [Cross Ref]
[10] Davis, T.M. & Chissom, B. (1981). Factor Analysis (R-Technique) of Ipsatized Data May Be Misleading, Psychological Reports. Vol 49, Issue 2
[11] De Waal, A.A. (2007), The Characteristics of a High Performance Organization. Business Strategy Series, August
[12] De Waal, A.A., Duong, H. & Ton V. (2009), High Performance in Vietnam: The Case of the Vietnamese Banking Industry, Journal of Transnational Management, 14, 179–201 [Cross Ref]
[13] De Waal, A.A. (2010), Achieving high performance in the public sector. What needs to be done? Public Performance & Management Review, 34, 1, 81–103
[14] De Waal, A.A. & Akaraborworn, C.T. (2013). Is the high performance organization framework suitable for Thai organizations?, Measuring Business Excellence, Vol. 17 Issue: 4, 76-87
[15] De Waal, A.A.(2012). What Makes a High Performance Organization: Five Factors of Competitive Advantage that Apply Worldwide 2nd Edition. HPO Center
[16] Van Eijnatten, F.M., Van der Ark, L.A. & Holloway, S.S. (2015). Ipsative Measurement and the Analysis of Organizational Values: an Alternative for Data Analysis. Qual Quant, 49, 559-579
[17] Field, A., Miles, J. & Field, Z. (2012). Discovering Statistics Using R. Sage Publications. [Cross Ref]
[18] Goedegebuure, R.V. (2018). Sampling items from lengthy scales using Item Response Theory (IRT); The case of the High-Performance Organizations (HPO) Scale. StatMind Working Paper Series 2018:01. Retrievable from http://statmind.org/Publications/ (last accessed 18 February 2018) [Cross Ref]
[19] Gordon, L. V. (1951). Validities of the forced-choice and questionnaire methods of personality measurement. Journal of Applied Psychology, 35, 407-412. [Cross Ref]
[20] Hayton, J.C., Allen, D.G. & Scarpello, V. (2004). Factor Retention Decisions in Exploratory Factor Analysis: A Tutorial on Parallel Analysis. Organizational Research Methods, Vol. 7 No. 2, April 2004 191-205 [Cross Ref]
[21] Hicks, L.E. (1970). Some properties of ipsative, normative, and forced-choice normative measures. Psychological Bulletin, 74, 167-184 [Cross Ref]
[22] Hofstee, W.K.B., Ten Berge, J.M.F. & Hendriks, A.A.J. (1998). How to score questionnaires? Personality and Individual Differences, 25, 897-909 [Cross Ref]
[23] Hooper, D., Coughlan, J. & Mullen, M. R. (2008). Structural Equation Modelling: Guidelines for Determining Model Fit. The Electronic Journal of Business Research Methods, Volume 6, Issue 1, 53 – 60 [Cross Ref]
[24] Johnson, C. E., Wood R. & Blinkhorn, S. F. (1988). Spuriouser and spuriouser: the use of ipsative personality tests. Journal of Occupational Psychology, 61, 153-162. [Cross Ref]
[25] Ledesma, R.D. & Valero-Mora, P. (2007). Determining the Number of Factors to Retain in EFA: an easy-to-use computer program for carrying out Parallel Analysis. Practical Assessment, Research & Evaluation, Volume 12, Number 2, February [Cross Ref]
[26] Matsunaga, M. (2010). How to Factor-Analyze Your Data Right: Do’s, Don’ts and How-to’s. International Journal of Psychological Research, 3(1), 97-110 [Cross Ref]
[27] Moeller J. (2015). A word on standardization in longitudinal studies: don't. Frontiers in Psychology 6. [Cross Ref]
[28] Moskowitz, D.S. (2005). Unfolding Interpersonal Behavior. Journal of Personality 73:6, December 2005 [Cross Ref]
[29] Saville, P. & Willson, E. (1991). The reliability and validity of normative and ipsative approaches in the measurement of personality. Journal of Occupational Psychology, 64, 219-238. [Cross Ref]
Robert Goedegebuure and Manorama Adhikari (2023), Factor Analysis: Dealing with Response Bias. IJBMR 11(1), 25-33. DOI: 10.37391/IJBMR.110103.