Evaluating the Factor Structure and Construct Validity of the Intensions Health-Related Masculine Values Scale: Is It Really a Measure of Health-Related Masculine Values?
Main Article Content
Abstract
The Intensions Health-Related Masculine Values Scale (IHRMVS) was recently developed to address the growing interest in examining the associations between strengths-based masculinity and men’s health. However, while the original development and validation work was promising, replication and extension of several aspects of the scale’s validity should be undertaken before the scale can be adopted for widespread use. Using a large sample of British men, aged 18-80 years, exploratory factor analyses revealed a single factor for both the original age grouping (18-29 years, n = 166) and an older group of men (? 30 years, n = 404), failing to replicate the two-factor structure proposed in the parent study. Additionally, correla-tions between scores on the IHRMVS and scores on a measure of traditional male role norms were mostly non-significant or very small. Similar findings emerged in our analyses of the associations between scores on the IHRMVS and indices of depression, anxiety, help-seeking attitudes, sleep disturbance, and smok-ing. These findings draw into question the utility of the IHRMVS as a measure of positive, health-related masculine values.
Downloads
Download data is not yet available.
Article Details
How to Cite
McCreary, D., Barron, D., & Swami, V. (2019). Evaluating the Factor Structure and Construct Validity of the Intensions Health-Related Masculine Values Scale: Is It Really a Measure of Health-Related Masculine Values?. International Journal of Mens Social and Community Health, 2(1), e55-e68. https://doi.org/10.22374/ijmsch.v2i1.24
Section
Articles
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright of articles published in all DPG titles is retained by the author(s). The author(s) grants DPG the rights to publish the article and identify itself as the original publisher. The author grants DPG exclusive commercial rights to the article. The author grants any party the rights to use the article freely for non-commercial purposes provided that the original work is properly cited.
References
1. Robertson S, Kilvington-Dowd L. Masculinity and men’s health disparities: Conceptual and theoretical challenges. In: Griffith DM, Bruce MA, Thorpe RJ, eds. Men’s Health Equity: A Handbook. London: Routledge; 2019:10–26.
2. Thompson Jr EH, Bennett KM. Measurement of masculinity ideologies: A (critical) review. Psychol Men Masculin 2015;16(2):135–40.
3. Oliffe JL. Three key notes from my address at the men, health and wellbeing conference. Int J Men’s Health 2015;14(3):204–13.
4. Bem SL. The measurement of psychological androgyny. J Consult Clin Psychol 1974;42(2):155–62.
5. Spence JT, Helmreich RL. Masculinity and Feminin-ity: Their Psychological Dimensions, Correlates, and Antecedents. Austin (TX): University of Texas Press; 1978.
6. Spence JT, Helmreich RL, Holahan CK. Negative and positive components of psychological masculinity and femininity and their relationships to self-reports of neurotic and acting out behaviors. J Pers Soc Psychol 1979;37(10):1673–82.
7. Orlofsky JL. Relationship between sex role attitudes and personality traits and the Sex Role Behavior Scale-1: A new measure of masculine and feminine role behaviors and interests. J Pers Soc Psychol 1981;40(5):927–40.
8. Orlofsky JL, O’Heron CA. Development of a short-form Sex Role Behavior Scale. J Pers Assess 1987; 51(2):267–77.
9. Levant RF, Hirsch L, Celentano E, et al. The male role: An investigation of norms and stereotypes. J Mental Health Counsel 1992:14(3):325–37.
10. Mahalik JR, Locke BD, Ludlow LH, et al. Develop-ment of the Conformity to Masculine Norms Inven-tory. Psychol Men Masculin 2003;4(1):3–25.
11. Thompson Jr EH, Pleck JH. The structure of male role norms. Am Behav Scient 1986;29(5):531–43.
12. Eisler RM. The relationship between masculine gender role stress and men’s health risk: The validation of a construct. In: Levant RF, Pollack WS, eds. A New Psy-chology of Men. New York (NY): Basic Books; 1995.
13. O’Neil JM. Summarizing 25 years of research on men’s gender role conflict using the Gender Role Conflict Scale: New research paradigms and clinical implications. Counsel Psychol 2008;36(3): 358–45.
14. McCreary DR, Newcomb MD, Sadava SW. Dimen-sions of the male gender role: Confirmatory analysis in men and women. Sex Roles 1998;39(1–2):81–95.
15. McCreary DR, Newcomb MD, Sadava SW. The male role, alcohol use, and alcohol problems: A structural modeling examination in adult women and men. J Counsel Psychol 1999;46(1):109–24.
16. Sloan C, Connor M, Gough B. How does masculinity impact on health? A quantitative study of masculin-ity and health behavior in a sample of UK men and women. Psychol Men Masculin 2015;16(2):206–17.
17. McCreary DR. Masculinity. In Zeigler-Hill V, Shackelford TK, eds. Encyclopedia of Personality and Individual Differences. New York (NY): Springer; 2016.
18. Pleck JH. The gender role strain paradigm: An update. In Levant RF, Pollack WS, eds. A New Psychology of Men. New York (NY): Basic Books; 1995.
19. Hammer JH, Good GE. Positive psychology: An em-pirical examination of beneficial aspects of endorse-ments of masculine norms. Psychol Men Masculin 2010;11(4):303–18.
20. Kiselica MS, Benton-Wright S, Englar-Carlson M. Accentuating positive masculinity: A new foundation for the psychology of boys, men, and psychology. In: Wong YJ, Wester SR, eds. APA Handbook of Men and Masculinities. Washington (DC): American Psycho-logical Association; 2015.
21. Oliffe JL, Rice S, Kelly MT, et al. A mixed-methods study of the health-related masculine values among young Canadian men. Psychol Men Masculin 2019;20(3):310–23.
22. Kaplan D, Rosenmann A, Schuhendler, S. What about nontraditional masculinities? Toward a quantitative model of therapeutic new masculinity ideology. Men Masculinities 2017;20(4):393–426.
23. Fabrigar LR, Wegener DT. Exploratory Factor Analy-sis. New York (NY): Oxford University Press; 2012.
24. Swami V, Barron D. Translation and validation of body image instruments: Challenges, good practice guidelines, and reporting recommendations for test
adaptation. Body Image 2019; in press.
25. Worthington R, Whittaker T. Scale development re-
search: A content analysis and recommendations for best practice. Counsel Psychol 2006;34(6):806–38.
26. Institute for Health Metrics and Evaluation. GDB Compare. Seattle (WA): IHME; https://vizhub.health data.org/gbd-compare/.
27. Courtenay WH. Dying to Be Men: Psychosocial, Environmental, and Biobehavioral Directions in Promoting the Health of Men and Boys. New York (NY): Routledge; 2011.
28. Bottorff JL, Haines-Saah R, Kelly MT, et al. Gender, smoking and tobacco reduction and cessation: A scop-ing review. Int J Equity Health 2014;13:114.
29. Addis ME, Mahalik JR. Men, masculinity, and the contexts of help seeking. Am Psychol 2003;58(1):5–14.
30. Grandner MA, Jackson NJ, Pak VM, et al. Sleep disturbance is associated with cardiovascular and metabolic disorders. J Sleep Res 2012;21(4):427–33.
31. Seidler ZE, Dawes AJ, Rice SM, et al. The role of masculinity in men’s help-seeking for depression: A systematic review. Clin Psychol Rev 2016;49(9):106–18.
32. Cavanagh A, Wilson CJ, Kavanagh DJ, et al. Differ-ences in the expression of symptoms in men versus women with depression: A systematic review and meta-analysis. Harvard Rev Psych 2017;25(1):29–38.
33. Wong YJ, Ho M-HR, Wang S-Y, et al. Meta-analyses of the relationship between conformity to masculine norms and mental health-related outcomes. J Counsel
Psychol 2017;64(1):80–93.
34. World Health Organization. Depression and Other
Common Mental Disorders: Global Health Estimates. Geneva: World Health Organization; 2017.
35. Office for National Statistics. Ethnicity and Na-tional Identity in England and Wales. London: ONS; https://www.ons.gov.uk/peoplepopulationand community/culturalidentity/ethnicity/articles/ ethnicityandnationalidentityinenglandandwales/2012-12-11
36. Parent MC, Moradi B. Confirmatory factor analysis of the Conformity to Masculine Norms Inventory and development of the Conformity to Masculine Norms Inventory-46. Psychol Men Masculin 2009;10(3):175–89.
37. Fischer E, Farina A. Attitudes toward seeking psy-chological professional help: A shortened form and considerations for research. J Col Student Dev 1995;36(4):368–73.
38. Whittlesey V. Diversity Activities for Psychology. Boston (MA): Allyn and Bacon; 2001.
39. Rice SM, Fallon BJ, Aucote HM, et al. Development and preliminary validation of the Male Depression Risk Scale: Furthering the assessment of depression in men. J Affect Disord 2013;151(3):950–58.
40. Rice SM, Ogrodniczuk JS, Kealy D, et al. Validity of the Male Depression Risk Scale in a representative Canadian sample: Sensitivity and specificity in iden-tifying men with recent suicide attempt. J Mental Health 2019;28(2):132–40.
41. Spitzer RL, Kroenke K, Williams JB, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arc Internal Med 2006;166(10):1092–97.
42. Buysse DJ, Reynolds III CF, Monk TH, et al. The Pittsburgh Sleep Quality Index: A new instrument for psychiatric practice and research. Psych Res 1989;28(2):193–13.
43. Statistics Canada. Canadian Tobacco Use Monitor-ing Survey (CTUMS): 2005 Cycle 1 Questionnaire. Ottawa, 2005; http://www23.statcan.gc.ca/imdb-bmdi/ instrument/4440_Q1_V11-eng.pdf.
44. Buhrmester M, Kwang T, Gosling SD. Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspect Psychol Sci 2011;6(1):3–5.
45. Nunnally JCJ, Bernstein I. Psychometric Theory, 2nd edition. New York (NY): McGraw-Hill; 1995.
46. Clark LA, Watson D. Constructing validity: Basic is-sues in objective scale development. Psychol Assess 1995;7(3):309–19.
47. Kaiser H. An index of factor simplicity. Psychometrika 1974;39(1):31–36.
48. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika 1965;32(1):179–85.
49. Hayton JC, Allen DG, Scarpello V. Factor retention decisions in exploratory factor analysis: A tuto-
rial on parallel analysis. Org Res Meth 2007;7(2):
191–205.
50. Velicer WF, Eaton CA, Fava JL. Construct explication
through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. In: Goffin RD, Helmes E, eds. Problems and Solutions in Human As-sessment: Honoring Douglas N. Jackson at Seventy. Norwell (MA): Kluwer Academic; 2000.
51. Comrey AL, Lee HB. A First Course in Factor Analysis. Hillsdale (NJ): Lawrence Erlbaum; 1992.
52. Tucker LR. A Method for Synthesis of Factor Analysis Studies (Personnel Research Section Report No. 984). Washington (DC): Department of the Army; 1951.
53. Lorenzo-Seva U, ten Berge JMF. Tucker’s congruence coefficient as a meaningful index of factor similarity. Methodology 2006;2(2):57–64.
54. Dunn TJ, Baguley T, Brunsden V. From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. Br J Psychol 2014;105(3):399–412.
55. Sijtsma, K. On the use, the misuse, and the very lim-ited usefulness of Cronbach’s Alpha. Psychometrika 2009;74(1):107–20.
56. Pett MA, Lackey NR, Sullivan JJ. Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. Thousand Oaks (CA): Sage; 2003.
57. Velicer WF. Determining the number of components from the matrix of partial correlations. Psychometrika 1976;41(3):321327.
58. Asparouhov T, Muthén BO. Exploratory structural equation modeling. Struct Equat Modeling 2009;16(4): 397–438.
59. Jagdeo A, Cox BJ, Stein MB, et al. Negative attitudes toward help seeking for mental illness in 2 population-based surveys from the United States and Canada. Canadian J Psych 2009;54(11):757–66.
60. Bardi A, Schwartz SH. Values and behavior: Strength and structure of relations. Pers Soc Psychol Bull 2003;29(10):1207–20.
2. Thompson Jr EH, Bennett KM. Measurement of masculinity ideologies: A (critical) review. Psychol Men Masculin 2015;16(2):135–40.
3. Oliffe JL. Three key notes from my address at the men, health and wellbeing conference. Int J Men’s Health 2015;14(3):204–13.
4. Bem SL. The measurement of psychological androgyny. J Consult Clin Psychol 1974;42(2):155–62.
5. Spence JT, Helmreich RL. Masculinity and Feminin-ity: Their Psychological Dimensions, Correlates, and Antecedents. Austin (TX): University of Texas Press; 1978.
6. Spence JT, Helmreich RL, Holahan CK. Negative and positive components of psychological masculinity and femininity and their relationships to self-reports of neurotic and acting out behaviors. J Pers Soc Psychol 1979;37(10):1673–82.
7. Orlofsky JL. Relationship between sex role attitudes and personality traits and the Sex Role Behavior Scale-1: A new measure of masculine and feminine role behaviors and interests. J Pers Soc Psychol 1981;40(5):927–40.
8. Orlofsky JL, O’Heron CA. Development of a short-form Sex Role Behavior Scale. J Pers Assess 1987; 51(2):267–77.
9. Levant RF, Hirsch L, Celentano E, et al. The male role: An investigation of norms and stereotypes. J Mental Health Counsel 1992:14(3):325–37.
10. Mahalik JR, Locke BD, Ludlow LH, et al. Develop-ment of the Conformity to Masculine Norms Inven-tory. Psychol Men Masculin 2003;4(1):3–25.
11. Thompson Jr EH, Pleck JH. The structure of male role norms. Am Behav Scient 1986;29(5):531–43.
12. Eisler RM. The relationship between masculine gender role stress and men’s health risk: The validation of a construct. In: Levant RF, Pollack WS, eds. A New Psy-chology of Men. New York (NY): Basic Books; 1995.
13. O’Neil JM. Summarizing 25 years of research on men’s gender role conflict using the Gender Role Conflict Scale: New research paradigms and clinical implications. Counsel Psychol 2008;36(3): 358–45.
14. McCreary DR, Newcomb MD, Sadava SW. Dimen-sions of the male gender role: Confirmatory analysis in men and women. Sex Roles 1998;39(1–2):81–95.
15. McCreary DR, Newcomb MD, Sadava SW. The male role, alcohol use, and alcohol problems: A structural modeling examination in adult women and men. J Counsel Psychol 1999;46(1):109–24.
16. Sloan C, Connor M, Gough B. How does masculinity impact on health? A quantitative study of masculin-ity and health behavior in a sample of UK men and women. Psychol Men Masculin 2015;16(2):206–17.
17. McCreary DR. Masculinity. In Zeigler-Hill V, Shackelford TK, eds. Encyclopedia of Personality and Individual Differences. New York (NY): Springer; 2016.
18. Pleck JH. The gender role strain paradigm: An update. In Levant RF, Pollack WS, eds. A New Psychology of Men. New York (NY): Basic Books; 1995.
19. Hammer JH, Good GE. Positive psychology: An em-pirical examination of beneficial aspects of endorse-ments of masculine norms. Psychol Men Masculin 2010;11(4):303–18.
20. Kiselica MS, Benton-Wright S, Englar-Carlson M. Accentuating positive masculinity: A new foundation for the psychology of boys, men, and psychology. In: Wong YJ, Wester SR, eds. APA Handbook of Men and Masculinities. Washington (DC): American Psycho-logical Association; 2015.
21. Oliffe JL, Rice S, Kelly MT, et al. A mixed-methods study of the health-related masculine values among young Canadian men. Psychol Men Masculin 2019;20(3):310–23.
22. Kaplan D, Rosenmann A, Schuhendler, S. What about nontraditional masculinities? Toward a quantitative model of therapeutic new masculinity ideology. Men Masculinities 2017;20(4):393–426.
23. Fabrigar LR, Wegener DT. Exploratory Factor Analy-sis. New York (NY): Oxford University Press; 2012.
24. Swami V, Barron D. Translation and validation of body image instruments: Challenges, good practice guidelines, and reporting recommendations for test
adaptation. Body Image 2019; in press.
25. Worthington R, Whittaker T. Scale development re-
search: A content analysis and recommendations for best practice. Counsel Psychol 2006;34(6):806–38.
26. Institute for Health Metrics and Evaluation. GDB Compare. Seattle (WA): IHME; https://vizhub.health data.org/gbd-compare/.
27. Courtenay WH. Dying to Be Men: Psychosocial, Environmental, and Biobehavioral Directions in Promoting the Health of Men and Boys. New York (NY): Routledge; 2011.
28. Bottorff JL, Haines-Saah R, Kelly MT, et al. Gender, smoking and tobacco reduction and cessation: A scop-ing review. Int J Equity Health 2014;13:114.
29. Addis ME, Mahalik JR. Men, masculinity, and the contexts of help seeking. Am Psychol 2003;58(1):5–14.
30. Grandner MA, Jackson NJ, Pak VM, et al. Sleep disturbance is associated with cardiovascular and metabolic disorders. J Sleep Res 2012;21(4):427–33.
31. Seidler ZE, Dawes AJ, Rice SM, et al. The role of masculinity in men’s help-seeking for depression: A systematic review. Clin Psychol Rev 2016;49(9):106–18.
32. Cavanagh A, Wilson CJ, Kavanagh DJ, et al. Differ-ences in the expression of symptoms in men versus women with depression: A systematic review and meta-analysis. Harvard Rev Psych 2017;25(1):29–38.
33. Wong YJ, Ho M-HR, Wang S-Y, et al. Meta-analyses of the relationship between conformity to masculine norms and mental health-related outcomes. J Counsel
Psychol 2017;64(1):80–93.
34. World Health Organization. Depression and Other
Common Mental Disorders: Global Health Estimates. Geneva: World Health Organization; 2017.
35. Office for National Statistics. Ethnicity and Na-tional Identity in England and Wales. London: ONS; https://www.ons.gov.uk/peoplepopulationand community/culturalidentity/ethnicity/articles/ ethnicityandnationalidentityinenglandandwales/2012-12-11
36. Parent MC, Moradi B. Confirmatory factor analysis of the Conformity to Masculine Norms Inventory and development of the Conformity to Masculine Norms Inventory-46. Psychol Men Masculin 2009;10(3):175–89.
37. Fischer E, Farina A. Attitudes toward seeking psy-chological professional help: A shortened form and considerations for research. J Col Student Dev 1995;36(4):368–73.
38. Whittlesey V. Diversity Activities for Psychology. Boston (MA): Allyn and Bacon; 2001.
39. Rice SM, Fallon BJ, Aucote HM, et al. Development and preliminary validation of the Male Depression Risk Scale: Furthering the assessment of depression in men. J Affect Disord 2013;151(3):950–58.
40. Rice SM, Ogrodniczuk JS, Kealy D, et al. Validity of the Male Depression Risk Scale in a representative Canadian sample: Sensitivity and specificity in iden-tifying men with recent suicide attempt. J Mental Health 2019;28(2):132–40.
41. Spitzer RL, Kroenke K, Williams JB, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arc Internal Med 2006;166(10):1092–97.
42. Buysse DJ, Reynolds III CF, Monk TH, et al. The Pittsburgh Sleep Quality Index: A new instrument for psychiatric practice and research. Psych Res 1989;28(2):193–13.
43. Statistics Canada. Canadian Tobacco Use Monitor-ing Survey (CTUMS): 2005 Cycle 1 Questionnaire. Ottawa, 2005; http://www23.statcan.gc.ca/imdb-bmdi/ instrument/4440_Q1_V11-eng.pdf.
44. Buhrmester M, Kwang T, Gosling SD. Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspect Psychol Sci 2011;6(1):3–5.
45. Nunnally JCJ, Bernstein I. Psychometric Theory, 2nd edition. New York (NY): McGraw-Hill; 1995.
46. Clark LA, Watson D. Constructing validity: Basic is-sues in objective scale development. Psychol Assess 1995;7(3):309–19.
47. Kaiser H. An index of factor simplicity. Psychometrika 1974;39(1):31–36.
48. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika 1965;32(1):179–85.
49. Hayton JC, Allen DG, Scarpello V. Factor retention decisions in exploratory factor analysis: A tuto-
rial on parallel analysis. Org Res Meth 2007;7(2):
191–205.
50. Velicer WF, Eaton CA, Fava JL. Construct explication
through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. In: Goffin RD, Helmes E, eds. Problems and Solutions in Human As-sessment: Honoring Douglas N. Jackson at Seventy. Norwell (MA): Kluwer Academic; 2000.
51. Comrey AL, Lee HB. A First Course in Factor Analysis. Hillsdale (NJ): Lawrence Erlbaum; 1992.
52. Tucker LR. A Method for Synthesis of Factor Analysis Studies (Personnel Research Section Report No. 984). Washington (DC): Department of the Army; 1951.
53. Lorenzo-Seva U, ten Berge JMF. Tucker’s congruence coefficient as a meaningful index of factor similarity. Methodology 2006;2(2):57–64.
54. Dunn TJ, Baguley T, Brunsden V. From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. Br J Psychol 2014;105(3):399–412.
55. Sijtsma, K. On the use, the misuse, and the very lim-ited usefulness of Cronbach’s Alpha. Psychometrika 2009;74(1):107–20.
56. Pett MA, Lackey NR, Sullivan JJ. Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. Thousand Oaks (CA): Sage; 2003.
57. Velicer WF. Determining the number of components from the matrix of partial correlations. Psychometrika 1976;41(3):321327.
58. Asparouhov T, Muthén BO. Exploratory structural equation modeling. Struct Equat Modeling 2009;16(4): 397–438.
59. Jagdeo A, Cox BJ, Stein MB, et al. Negative attitudes toward help seeking for mental illness in 2 population-based surveys from the United States and Canada. Canadian J Psych 2009;54(11):757–66.
60. Bardi A, Schwartz SH. Values and behavior: Strength and structure of relations. Pers Soc Psychol Bull 2003;29(10):1207–20.