Common method bias in research designs using self-report tools: literature overview and recommended remedies
Published:
Jul 21, 2022
Keywords:
Common method variance self-report questionnaire systematic measurement error procedural remedies statistical remedies
Abstract
One of the most discussed and controversial methodological and statistical issues that concerns empirical survey-based research is common method bias, which may occur when the data for the predictor and criterion variables come from the same person using the same response method. Uncontrolled method variance may produce biased estimates of the reliability and validity of the examined constructs and erroneous parametric estimates of the relationships between two constructs. The aim of the study is the literature overview on the concepts of common method bias and variance. The 143 articles that derived from the advanced search of relevant studies in four databases demonstrated the main thematic sections of the present study. Classical test theory was used to explain the sources of measurement errors and the effects of common method variance. Subsequently, the prevailing procedural and statistical remedies for controlling common method variance were described. The potential effects of common method variance on self-report research are complex and difficult to understand, but despite the debate about its nature and extent, researchers from psychology and social or organizational disciplines should take steps to minimize method bias.
Article Details
- How to Cite
-
Kaltsonoudi, K., Tsigilis, N., & Karteroliotis, K. (2022). Common method bias in research designs using self-report tools: literature overview and recommended remedies. Psychology: The Journal of the Hellenic Psychological Society, 27(1), 210–231. https://doi.org/10.12681/psyhps.30843
- Section
- THEORETICAL REVIEWS
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The journal PSYCHOLOGY adopts a Platinum open-access policy. Submission, processing or publication costs are waived by the Hellenic Psychological Society. Papers published in the journal PSYCHOLOGY are licensed under a 'Creative Commons Attribution-ShareAlike 4.0 International' licence. The authors reserve the copyright of their work and grant the journal the right of its first publication. Third-party licensees are allowed to use the published paper immediately after publication as they wish, provided they retain the defined by the license copyright formalities, regarding the reference to its author(s) and its initial publication in the journal PSYCHOLOGY. Moreover, any adjusted work should be shared under the same reuse rights, so with the same CC license.
Downloads
References
Αγγούσης, Ι. (2020). Ο διαμεσολαβητικός μηχανισμός των τεχνολογιών πληροφορικής και επικοινωνιών στη σχέση μεταξύ κοινωνικοοικονομικών παραγόντων και διαφθοράς [Αδημοσίευτη Διδακτορική Διατριβή]. Τμήμα Εφαρμοσμένης Πληροφορικής, Πανεπιστήμιο Μακεδονίας, Θεσσαλονίκη. Ανακτήθηκε στις 21 Ιανουαρίου 2021 από https://www.didaktorika.gr/eadd/handle/10442/47746
Allen, M. J., & Yen, W. M. (1979). Introduction to Measurement Theory. Brooks/Cole Pub. Co.
Antonakis, J. (2017). On doing better science: From thrill of discovery to policy implications. The Leadership Quarterly, 28(1), 5-21. http://dx.doi.org/10.1016/j.leaqua.2017.01.006
Anusic, I., Schimmack, U., Pinkus, R. T., & Lockwood, P. (2009). The nature and structure of correlations among Big Five ratings: The halo-alpha-beta model. Journal of Personality and Social Psychology, 97(6), 1142–1156. https://doi.org/10.1037/a0017159
Ashkanasy, N. M. (2008). Submitting your manuscript. Journal of Organizational Behavior, 29, 263-264. https://doi.org/10.1002/job.524
Ashkanasy, N. M. (2010). Editorial: Publishing today is more difficult than ever. Journal of Organizational Behavior, 1-3. https://doi.org/10.1002/job.676
Bachrach, D. G., Powell, B. C., Collins, B. J., & Richey, R. G. (2006). Effects of task interdependence on the relationship between helping behavior and group performance. Journal of Applied Psychology, 91(6), 1396-1405. https://doi.org/10.1037/0021-9010.91.6.1396
Bäckström, M., Björklund, F., & Larsson, M. R. (2009). Five-factor inventories have a major general factor related to social desirability which can be reduced by framing items neutrally. Journal of Research in Personality, 43(3), 335-344. https://doi.org/10.1016/j.jrp.2008.12.013
Bagozzi, R. P. (1984). Expectancy-value attitude models an analysis of critical measurement issues. International Journal of Research in Marketing, 1(4), 295-310. https://doi.org/10.1016/0167-8116(84)90017-X
Bagozzi, R. P. (2011). Measurement and meaning in information systems and organizational research: Methodological and philosophical foundations. MIS Quarterly, 35(2), 261-292. https://doi.org/10.2307/23044044
Bagozzi, R. P., & Yi, Y. (1990). Assessing method variance in multitrait-multimethod matrices: The case of self-reported affect and perceptions at work. Journal of Applied Psychology, 75(5), 547-560. https://doi.org/10.1037/0021-9010.75.5.547
Baumgartner, H., & Steenkamp, J. B. E. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38(2), 143-156. https://doi.org/10.1509/jmkr.38.2.143.18840
Baumgartner, H., & Weijters, B. (2021). Dealing with Common Method Variance in International Marketing Research. Journal of International Marketing. Advance online publication. https://doi.org/10.1177/ 0022242921995871
Baumgartner, H., Weijters, B., & Pieters, R. (2021). The biasing effect of common method variance: some clarifications. Journal of the Academy of Marketing Science, 49(2), 221-235. https://doi.org/10.1007/ s11747-020-00766-8
Berendt, J., & Uhrich, S. (2016). Enemies with benefits: The dual role of rivalry in shaping sports fans’ identity. European Sport Management Quarterly, 16(5), 613-634. https://doi.org/10.1080/ 16184742.2016.1188842
Biderman, M. D., Nguyen, N. T., Cunningham, C. J. L., & Ghorbani, N. (2011). The ubiquity of common method variance: The case of the Big Five. Journal of Research in Personality, 45(5), 417-429. https://doi.org/10.1016/j.jrp.2011.05.001
Campbell, J. P. (Ed.). (1982). Editorial: Some remarks from the outgoing editor. Journal of Applied Psychology, 67(6), 691–700. https://doi.org/10.1037/h0077946
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105. https://doi.org/10.1037/h0046016
Chang, L., Connelly, B. S., & Geeza, A. A. (2012). Separating method factors and higher order traits of the Big Five: A meta-analytic multitrait–multimethod approach. Journal of Personality and Social Psychology, 102(2), 408-426. https://doi.org/10.1037/a0025559
Chang, S., J., van Witteloostuijn, A., & Eden, L. (2010). From the editors: Common method variance in international business research. Journal of International Business Studies, 41(2), 178-184. https://doi.org/10.1057/jibs.2009.88
Chen, P. Y., & Spector, P. E. (1991). Negative affectivity as the underlying cause of correlations between stressors and strains. Journal of Applied Psychology, 76(3), 398-407. https://doi.org/10.1037/0021-9010.76.3.398
Conway, J. M., & Lance, C. E. (2010). What reviewers should expect from authors regarding common method bias in organizational research. Journal of Business and Psychology, 25(3), 325-334. https://doi.org/10.1007/s10869-010-9181-6
Cote, J. A., & Buckley, M. R. (1987). Estimating trait, method, and error variance: Generalizing across 70 construct validation studies. Journal of Marketing Research, 24(3), 315-318. https://doi.org/10.2307/3151642
De Vries, R. (2011). No evidence for a general factor of personality in the HEXACO Personality Inventory. Journal of Research in Personality, 45(2), 229–232. https://doi.org/10.1016/j.jrp.2010.12.002
Doty, D. H., & Glick, W. H. (1998). Common methods bias: Does common methods variance really bias results? Organizational research methods, 1(4), 374-406. https://doi.org/10.1177/109442819814002
Fisher, D. M. (2014). A multilevel cross-cultural examination of role overload and organizational commitment: Investigating the interactive effects of context. Journal of Applied Psychology, 99(4), 723-736. https://doi.org/10.1037/a0035861
Fuller, C. M., Simmering, M. J., Atinc, G., Atinc, Y., & Babin, B. J. (2016). Common methods variance detection in business research. Journal of Business Research, 69(8), 3192-3198. https://doi.org/10.1016 /j.jbusres.2015.12.008
Gerstner, C. R., & Day, D. V. (1997). Meta-Analytic review of leader–member exchange theory: Correlates and construct issues. Journal of Applied Psychology, 82(6), 827-844. https://doi.org/10.1037/0021-9010.82.6.827
Harrison, D. A., McLaughlin, M. E., & Coalter, T. M. (1996). Context, cognition, and common method variance: Psychometric and verbal protocol evidence. Organizational Behavior and Human Decision Processes, 68(3), 246-261. https://doi.org/10.1006/obhd.1996.0103
Huang, Y. S. S., Greenbaum, R. L., Bonner, J. M., & Wang, C. S. (2019). Why sabotage customers who mistreat you? Activated hostility and subsequent devaluation of targets as a moral disengagement mechanism. Journal of Applied Psychology, 104(4), 495-510. https://doi.org/10.1037/apl0000359
Jakobsen, M., & Jensen, R. (2015). Common method bias in public management studies. International Public Management Journal, 18(1), 3-30. https://doi.org/10.1080/10967494.2014.997906
Johnson, R. E., Rosen, C. C., & Djurdjevic, E. (2011). Assessing the impact of common method variance on higher order multidimensional constructs. Journal of Applied Psychology, 96(4), 744-761. https://doi.org/10.1037/a0021504
Jordan, P. J., & Troth, A. C. (2020). Common method bias in applied settings: The dilemma of researching in organizations. Australian Journal of Management, 45(1), 3-14. https://doi.org/10.1177/0312896219871976
Kaltsonoudi, K., Tsigilis, N., & Karteroliotis, K. (2022). Critical review of the literature and current tendencies of the common method variance in sport management research. Measurement in Physical Education and Exercise Science, 26(2), 103-115. http://dx.doi.org/10.1080/1091367X.2021.1949322
Karatepe, O. M. (2012). Job resources, work engagement, and hotel employee outcomes: A time-lagged analysis. Economic Research, 25(3), 644-665. https://doi.org/10.1080/1331677X.2012.11517527
Kock, F., Berbekova, A., & Assaf, A. G. (2021). Understanding and managing the threat of 612 common method bias: Detection, prevention, and control. Tourism Management, 86, Article e104330. https://doi.org/10.1016/j.tourman.2021.104330
Kwan, H. K., Zhang, X., Liu, J., & Lee, C. (2018). Workplace ostracism and employee creativity: An integrative approach incorporating pragmatic and engagement roles. Journal of Applied Psychology, 103(12), 1358-1366. http://dx.doi.org/10.1037/apl0000320
Lance, C. E., Dawson, B., Birkelbach, D., & Hoffman, B. J. (2010). Method effects, measurement error, and substantive conclusions. Organizational Research Methods, 13(3), 435-455. https://doi.org/10.1177/1094428109352528
Le, H., Schmidt, F. L., & Putka, D. J. (2009). The multifaceted nature of measurement artifacts and its implications for estimating construct-level relationships. Organizational Research Methods, 12(1), 165-200. https://doi.org/10.1177/1094428107302900
Lindell, M. K., & Whitney, D. J. (2001). Accounting for common method variance in cross-sectional research designs. Journal of Applied Psychology, 86(1), 114-121. https://doi.org/10.1037/0021-9010.86.1.114
Lishner, D. A., Hong, P. Y., Jiang, L., Vitacco, M. J., & Neumann, C. S. (2015). Psychopathy, narcissism, and borderline personality: A critical test of the affective empathy-impairment hypothesis. Personality and Individual Differences, 86, 257-265. https://doi.org/10.1016/j.paid.2015.05.036
Lord, F. M., & Novick, M. R. (2008). Statistical theories of mental test scores. Information Age Publishing.
Lowe, K. B., Kroeck, K. G., & Sivasubramaniam, N. (1996). Effectiveness correlates of transformational and transactional leadership: A meta-analytic review of the MLQ literature. The Leadership Quarterly, 7(3), 385-425. https://doi.org/10.1016/S1048-9843(96)90027-2
MacKenzie, S. B., & Podsakoff, P. M. (2012). Common method bias in marketing: causes, mechanisms, and procedural remedies. Journal of Retailing, 88(4), 542–555. https://doi.org/10.1016/j.jretai.2012.08.001
Malhotra, N. K., Kim, S. S., & Patil, A. (2006). Common method variance in IS research: A comparison of alternative approaches and a reanalysis of past research. Management Science, 52(12), 1865-1883. https://doi.org/10.1287/mnsc.1060.0597
Malhotra, N. K., Schaller, T. K., & Patil, A. (2017). Common method variance in advertising research: When to be concerned and how to control for it. Journal of Advertising, 46(1), 193-212. https://doi.org/10.1080/00913367.2016.1252287
Mathieu, C., Fabi, B., Lacoursière, R., & Raymond, L. (2016). The role of supervisory behavior, job satisfaction and organizational commitment on employee turnover. Journal of Management & Organization, 22(1), 113–129. https://doi.org/10.1017/jmo.2015.25
McDermott, M. S., & Sharma, R. (2017). Evaluating the impact of method bias in health behaviour research: Α meta-analytic examination of studies utilising the theories of reasoned action and planned behaviour. Health psychology review, 11(4), 358-373. https://doi.org/10.1080/17437199.2017.1339568
Meier, K. J., & O’Toole, L. J. (2013). Subjective organizational performance and measurement error: Common source bias and spurious relationships. Journal of Public Administration Research and Theory, 23(2), 429-456. https://doi.org/10.1093/jopart/mus057
Min, H., Park, J., & Kim, H. J. (2016). Common method bias in hospitality research: A critical review of literature and an empirical study. International Journal of Hospitality Management, 56, 126-135. https://doi.org/10.1016/j.ijhm.2016.04.010
Musek, J. (2007). A General Factor Of Personality: Evidence for the Big One in the five-factor model. Journal of Research in Personality, 41(6), 1213–1233. https://doi.org/10.1016/j.jrp.2007.02.003
Ostroff, C., Kinicki, A. J., & Clark, M. A. (2002). Substantive and operational issues of response bias across levels of analysis: An example of climate-satisfaction relationships. Journal of Applied Psychology, 87(2), 355-368. https://doi.org/10.1037/0021-9010.87.2.355
Παπάζογλου, Σ. (2018). Ιδιότυποι Τρόποι Απόκρισης: Σύγκριση μεταξύ παραδοσιακών συνθηκών μέτρησης και συνθηκών συμμετοχής σε διαδικτυακή έρευνα [Αδημοσίευτη Διδακτορική Διατριβή]. Φιλοσοφική σχολή, Τμήμα Ψυχολογίας, ΕΚΠΑ, Αθήνα. Ανακτήθηκε στις 21 Ιανουαρίου 2021 από: https://www.didaktorika.gr/eadd/handle/10442/43675
Pejtersen, J. H., Kristensen, T. S., Borg, V., & Bjorner, J. B. (2010). The second version of the Copenhagen Psychosocial Questionnaire. Scandinavian Journal of Public Health, 38(3_suppl), 8-24. https://doi.org/10.1177/1403494809349858
Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. http://dx.doi.org/10.1037/0021-9010.88.5.879
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63(1), 539-569. http://dx.doi.org/10.1146/annurev-psych-120710-100452
Podsakoff, P. M., & Organ, D. W. (1986). Self-reports in organizational research: Problems and prospects. Journal of Management, 12(4), 531-544. https://doi.org/10.1177/014920638601200408
Pugh, S. D., Groth, M., & Hennig-Thurau, T. (2011). Willing and able to fake emotions: A closer examination of the link between emotional dissonance and employee well-being. Journal of Applied Psychology, 96(2), 377-390. https://doi.org/10.1037/a0021395
Richardson, H. A., Simmering, M. J., & Sturman, M. C. (2009). A tale of three perspectives: Examining post hoc statistical techniques for detection and correction of common method method variance. Organizational Research Methods, 12(4), 762–800. https://doi.org/10.1177/1094428109332834
Robertson, R., Gockel, C., & Brauner, E. (2013). Trust your teammates or bosses? Differential effects of trust on transactive memory, job satisfaction, and performance. Employee Relations, 35(2), 222-242. https://doi.org/10.1108/01425451311287880
Schmidt, L. F., & Hunter, E. J. (1999). Theory testing and measurement error. Intelligence, 27(3), 183-198. https://doi.org/10.1016/S0160-2896(99)00024-0
Sharma, R., Yetton, P., & Crawford, J. (2009). Estimating the effect of common method variance: The method—method pair technique with an illustration from TAM Research. MIS Quarterly, 33(3), 473-490. https://doi.org/10.2307/20650305
Siemsen, E., Roth, A., & Oliveira, P. (2010). Common method bias in regression models with linear, quadratic, and interaction effects. Organizational Research Methods, 13(3), 456-476. https://doi.org/10.1177/1094428109351241
Spector, P. E. (1987). Method variance as an artifact in self-reported affect and perceptions at work: Myth or significant problem?. Journal of Applied Psychology, 72(3), 438-443. https://doi.org/10.1037/0021-9010.72.3.438
Spector, P. E. (2006). Method variance in organizational research: truth or urban legend?. Organizational Research Methods, 9(2), 221-232. http://dx.doi.org/10.1177/1094428105284955
Spector, P. E., & Brannick, M. T. (2010). Common method issues: An introduction to the feature topic in organizational research methods. Organizational Research Methods, 13(3), 403-406. https://doi.org/10.1177/1094428110366303
Thompson, M. J., Carlson, D. S., Kacmar, K. M., & Vogel, R. M. (2020). The cost of being ignored: Emotional exhaustion in the work and family domains. Journal of Applied Psychology, 105(2), 186-l95. https://psycnet.apa.org/doi/10.1037/apl0000433
Traber, D., Le Barbenchon, E., Hot, P., & Pellissier, S. (2020). Perspectives about the PTSD prevalence rate in the case of multiple traumatic events exposure among mountain workers. European Journal of Trauma & Dissociation, 4(4), 1-6. https://doi.org/10.1016/j.ejtd.2019.05.002
Van der Linden, D., te Nijenhuis, J., & Bakker, A. B. (2010). The General Factor of Personality: A meta-analysis and a criterion-related validity study. Journal of Research in Personality, 44(3), 315–327. https://doi.org/10.1016/j.jrp.2010.03.003
Van der Linden, D., Tsaousis, I., & Petrides, K. V. (2012). Overlap between General Factors of Personality in the Big Five, Giant Three, and trait emotional intelligence. Personality and Individual Differences, 53(3), 175-179. https://doi.org/10.1016/j.paid.2012.03.001
Williams, L. J., Cote, J. A., & Buckley, M. R. (1989). Lack of method variance in self-reported affect and perceptions at work: Reality or artifact?. Journal of Applied Psychology, 74(3), 462-468. https://doi.org/10.1037/0021-9010.74.3.462
Williams, L. J., Edwards, J. R., & Vandenberg, R. J. (2003). Recent advances in causal modeling methods for organizational and management research. Journal of Management, 29(6), 903-936. https://doi.org/10.1016/S0149-2063(03)00084-9
Williams, L. J., Hartman, N., & Cavazotte, F. (2010). Method variance and marker variables: A review and comprehensive CFA marker technique. Organizational research methods, 13(3), 477-514. https://doi.org/10.1177/1094428110366036
Williams, L. J., & McGonagle, A. K. (2016). Four research designs and a comprehensive analysis strategy for investigating common method variance with self-report measures using latent variables. Journal of Business and Psychology, 31(3), 339-359. https://doi.org/10.1007/s10869-015-9422-9
