Developing and Psychometric Validating an Instrument Based on the Protection Motivation Theory to Measure the Factors Influencing Preventive Behaviors of Malaria

Sayed Mohammad Reza Hosseini, Shamsaddin Niknami^*, Iraj Zareban and Ali Reza Hidarnia

^*Correspondence: Shamsaddin Niknami, Department of Health Education and Promotion, Faculty of Medical Sciences, Tarbiat Modares University, Iran, Email:

Author info »

Introduction

Despite global efforts, malaria is considered as one of the most important health challenges in the world, especially in developing countries where control of Malaria is one of the Millennium Development Goals [1]. Based on the WHO assessment, 212 million new cases of malaria and 429 deaths were reported worldwide in 2015 [2]. In 2015, USD 2.9 billion invested in Malaria. Malaria-endemic countries contributed 32% of the total funding the fight against malaria. In Iran, more than 60% of the population lives in malaria-endemic areas [3]. About 95% of malaria cases are reported in southeastern parts of the country, including Sistan and Baluchistan, Hormozgan, and some tropical regions of Kerman province [4,5]. The most important causes of this condition are epidemiological and entomological features of the disease, technical failure, emergence of drug resistance in parasites, mosquitoes resistance to insecticides, environmental problems, difficulty in biological control, organizational deficiencies, inadequate access to effective health services, urbanization and displacement of human populations, social, cultural, political, economic and ecological factors of the countries involved and the economic burden of the disease which have limited the possibility of disease control [6]. According to evidence, malaria control programs with the community participation at compared to programs that designed and implemented by the government more effectively, and the knowledge, attitudes, and practices of the people living in malariaprone areas effectiveness role in planning for the control and prevention of malaria [7]. The first step in designing an educational program is to choose a model [8]. The theories and models help program planners to identify beyond what are the main undesirable risk factors (e.g. Gender socioeconomic status), to answer why, what and how can change the people behavior [9]. One of these theories and models is the protection motivation theory (PMT) which was developed by Rogers in 1975 to understand effect of fear appeal on the health-related attitudes and behaviors. In the main model, fear appeal is considered as an important factor in choosing behavior by a person. Protection motivation theory describes individual motivational factors to do or not to do health behaviors and believes that individual responses to threats depend on the cognitive assessments [10]. PMT is based on two default cognitive pathways: threat appraisal pathway and the coping appraisal pathway. Threat appraisal pathway consists of four constructs in which both components of potential reward of the maladaptive response (internal and external rewards for participating in maladaptive behaviors), and the perceived severity and vulnerability construct assess the outcome of the maladaptive behavior. The coping appraisal pathway, consists of assessing individual perception of his/her ability to prevent threatening risk, including efficacy variables (response-efficacy and self-efficacy) and response cost (financial and non-financial costs such as time, effort, and so on) of the protective maneuver. The balance between the two appraisal pathways determines the protection motivation or intention to start, as well as continuing or controlling an adaptive response. In order to recall the protection motivation, the perceived severity and sensitivity, and perceived response efficacy and self-efficiency should overcome the rewards of maladaptive response (lack of self-protection), and the costs of consistent response (self- protection), respectively. In PMT, the purpose of intention and behavior should prevent a potentially harmful consequence such as diseases (Malaria). The Motivating intention may result in protective action among individuals [11,12]. Thus, designing a new questionnaire regarding society behavior and culture is necessary for the studies implementing the PMT to explain the different health behaviors [13]. A large number of researchers believe that instrument is the first step during the research process [14], and the lack of a desirable measurement instrument is regarded as one of the main problems. In general, robust studies are possible when psychometric instruments are available [15]. Therefore, the present study aims to develop and validate an instrument psychometrically based on the protection motivation theory for the preventing malaria in Sarbaz.

Methods

Participants and sampling

Data were collected between March-December 2016. Participants were male students for religious schools. Schools were selected from a highly endemic rural area in Sarbaz and Pishin, Zahedan, Iran. Participants were selected based on stratified cluster sampling method. First, from the four townships Rask, Sarbaz Kalat, Pishin, and Parud, two Rask and Pishin townships were randomly selected with the same sampling method that refers to the zone where the malaria infects rate of residents and livestock. Second, from 12 religious’ schools in Rask and Pishin area, two schools in each area selected were randomly sampled. Third, eight classes selected were randomly schools sampled. All the students in the selected classes were recruited in this study. The inclusion criteria were the males, aged ≥ 15 years, and those studying in religious schools. The exclusion criteria were the lack of informed consent and unwilling to participate. Based on at least five samples for each item [16,17], for analysis factor, the initial sample consisted of 260 students due to the 46- item questionnaire. Among the total 260 participants, 26 (10.0%) were excluded due to missing data, final sample of 234 (90.0%).

Instrument

The instrument of the present study was a questionnaire based on the protection motivation theory. The scientific text, articles, and dissertations in the field of Malaria were reviewed for questionnaire items. Then, an initial questionnaire was prepared through interviews with experts in the field. The questionnaire included the constructs the protection motivation theory includes of perceived vulnerability (5 items), perceived severity (6 items), perceived rewards (6 items), self-efficacy (6 items), response efficiency (6 items), response cost (6 items), fear (5 items), protection motivation (6 items). A five-point Likert scale with strongly agree, agree, neutral, disagree, and strongly disagree was used to assess the behavior intention.

Content validity

The content validity of the questionnaire was determined by the qualitative method including the interviews with seven experts in health education, two experts in parasitology, and an expert in entomology. Based on the research objectives, the experts expressed their opinions about the questionnaire, using the right words, putting the items in the right place, grammar, correct scoring, and other cases [18]. The content validity ratio (CVR) [19], and content validity index (CVI) [20], were used to determine content validity. In order to determine CVR according to the expert’s opinion, the questions with content validity greater than 0.62 were considered important and the rest were excluded. Then, the CVI was calculated by using expert comments on the relevance, simplicity, and clarity of each item. The CVI higher than 0.79 was accepted [18].

Face validity

The face validity was analyzed in term of quantity and quality. The questionnaire was distributed among 15 students in order to determine face validity by qualitative method. They were asked to comment on the degree difficulty in understanding the concepts, relevance, and misperceptions. In addition, the quantitative impact method was applied to determine the importance of items and exclude inappropriate items. The acceptable impact score was above 1.5 [18].

Construct validity

Confirmatory factor analysis is considered as a method in which predictive factors set a model to the data [16,17]. Exploratory factor analysis strongly emphasizes on different estimation methods, maximum probability, and a different set of standards to evaluate the adequacy of solutions. In general, confirmatory factor analysis is useful to improve the instrument in the next steps. When the construct of the proposed factor is adequately in accordance with the data, and the construct is parsimoniously appropriate for other models, the confirmatory factor analysis would be effective. Here, the ratio of 1 to 4 or 1 to 5 for each item is recommended to determine the number of the sample [21]. According to Gorsuch et al. each item should have at least five samples, and at least 200 samples were selected [22]. The cutoff point of 0.3 was considered for applied loads [23]. Several indicators should be considered to determine the model fitting, among which Chi Square is regarded as one of the most important ones. Since the Chi square in the samples with high volume is significant, it fails to indicate model fitting and χ2/df is used as part of the Chi-square to the model degree of freedom. The values of 1-5 are appropriate for each item [24]. In other indicators such as Goodness of Fit Index (GFI), Normed Fit Index (NFI), Relative Fit Index (RFI), Incremental Fit Index (IFI), Comparative fit index (CFI), adjusted goodness of fit index (AGFI) range from 0 to 1; The closer the values to each other, indicating the fit of the model [25]. Further, the Root Mean Square Error of Approximation RMSEA shows a medium fit of the model when it is in the range of 0.08-0.08, and indicates fit of the model when it is below 0.08 [26].

Reliability

Internal consistency and test-retest were used to assess the reliability. Reliability is considered as one of the most important features for instruments measuring hidden variables or constructs. The results cannot be trusted without a reliable instrument, and the results are significantly different from the first step if the study is repeated. Although reliability is not a sufficient condition for trusting a measurement instrument, it is considered as a necessary condition [24]. The internal consistency was measured by Cronbach's alpha coefficient among 30 samples and the values equal to or higher than 0.65 were considered as acceptable. The Cronbach's alpha coefficient is widely used in various sciences, especially humanities and medical sciences, in order to assess the different instruments reliability of attitude assessment, educational tests, and knowledge assessment [27]. The questionnaire was completed by a two- week interval and the test- retest method in order to determine the retest co-efficient, and the values equal to or higher than 0.4 were accepted [28].

Data analysis

Standard Deviation (SD) and frequency (%) were calculated for the variables. The face validity quality and impact quantity methods were used for measuring the apparent fitness of instrument (difficulty levels, inappropriateness, and ambiguity) and eliminating inappropriate items and identifying the importance of each item, respectively. The CVR and CVI were used to evaluate the content validity. In order to assess the validity of the questionnaire structure, confirmatory factor analysis was applied a long with a criteria of good model fit (GFI>0.9, CFI>0.9, NFI>0.9, RFI> 0.9, IFI>0.9, AGFI>0.9, RMSEA<0.05, and Chi-square/df<2). The internal consistency of the questionnaire constructs was evaluated by the Cronbach’s alpha. Further, the reliability was assessed by test – retest which the coefficients of the group correlation (ICC) were used for each scale. The analysis was conducted by SPSS 19.0 and Lisrel 8.80. The value of P <0.05 was considered as statistically significant.

Ethical considerations

At baseline, the study protocol (IR.TMU. REC.1394.250) was approved by the Committee on Ethics in Tarbiat Modares University, Tehran, Iran. The participants were provided with comprehensive information on the research objectives and methods, advantages and disadvantages. In addition, confidentiality and privacy were assured and consent forms were collected before data collection. Further, they were assured of their freedom to withdraw at any stage of the project. The questionnaires were anonymous and the data were destroyed after the research.

Results

Based on the results, the majority of 234 participants were in the 15-35 age group (M=18.13 ± 3.81), single (83.8%), and with secondary education (42.3%). 97.9% of the families earned less than 300 dollars per month. The qualitative results of content validity indicate that comments and proposals of some experts were collected and some items were modified (Table 1). The quantitative results of content validity for content validity index show the scores higher than 0.62 for all questions which are accepted. Content validity ratio for four questions was less than 0.79, which were excluded from the questionnaire. Regarding the face validity in quantitative method, all items were above 1.5, so all items were retained. For the face validity, some items were modified based on the responses of participants. Some questions related to perceived sensitivity, severity, and costs were excluded due to the Cronbach's alpha coefficient less than 0.65. The alpha coefficient is acceptable when is in the range of 0.71.7- 0.86.6. Spearman correlation coefficient was obtained in the range of 0.76.7-0.98.6, which is acceptable. Table 2 indicates the cluster ICC test for questionnaire. As shown in Table 3, confirmatory factor analysis is above 0.3 for factor loadings of all items which is acceptable. The final fit indicators were calculated after modifying model for GFI, AGFI, NFI, RFI, IFI, and CFI. Table 4 shows the fit of model with relative Chi-square χ²/df and RMSEA equal to 1.36 and less than 0.08, respectively. In general, indicators of goodness of fit were acceptable and construct validity was confirmed by confirmatory factor analysis.

Table 1: Socio-demographic profile of the participant.

Table 2: Cronbach’s alpha coefficient, Spearman correlation coefficient and intraclass correlation coefficient for constructs of protection motivation theory on prevention of malaria.

Table 3: The results of confirmatory factor analysis of the components of the motivational protection theory questionnaire on the prevention of malaria.

Table 4: Model fit indices to confirm the suitability of the theory of protection of motivation in the prevention of malaria.

Discussion

The present study aimed to develop and validate psychometrically the instrument based on the theory of protection motivation in order to measure the prophylactic behaviors of malaria. PMT is widely applied as a theoretical framework for prediction and intervention of health behaviors which guide the intervention programs for the changing targeted behaviors [29]. The research scale was measured by interviews with experts, group discussions, and student seminars in religious schools in Sarbaz [26]. The content validity ratio and index were above 0.75 and 0.81, respectively, indicating high validity. Determining the validity of constructs using factor analysis was considered as the most important step in determining validity of questionnaire. Factor analysis for identifying clusters of questions related to scale or instrument was in the range of 0.8-0.9, which is moderate [26]. Based on the results of construct validity by using confirmatory factor analysis, 42 items related to PMT and the behavioral checklist was approved. The optimum values of indices for the goodness of model fit indicated the construct validity and matching the nine constructs with data, which is consistent with applying PMT to other health issues [30]. Since the reliability of instrument increases the study power to identify the significant differences and connections [31], Cronbach's alpha coefficient was calculated to estimate the internal consistency and the domains of instruments. The reliability of questionnaire was confirmed by the alpha close to one. The appropriate Cronbach's alpha coefficient in each domain indicates that the items represent an appropriate content for that dimension [32]. The value of Cronbach's alpha coefficient for instrument constructs was acceptable. Thus, the questionnaire had a strong reliability. The results are consistent with those of study conducted by Thrul et al. [33], and Sakhavidi et al [34], Helmes obtained Cronbach's alpha coefficient ranged from 0.64-0.87 for questionnaire based on PMT. In addition, test-retest results along with the cluster correlation coefficient (ICC) in the range of 0.70.8-0.79.5 represented the reliability and confidence for instrument. Based on the results, the scale based on PMT and behavioral checklist in different times and places had a high correlation. Therefore, the present study had face validity, content validity, construct validity, internal consistency and instrument stability, and can be used in future studies. The results present good evidence for the factor power and instrument reliability of measuring the PMT in the case of preventive behaviors of malaria in the population.

Conclusion

The psychometric validation indicated that the instrument based on PMT has a high reliability and validity. The confirmatory factor analysis indicates the fit of model. Therefore, the present questionnaire is considered as a suitable source for developing similar studies at the national level.

Limitations

First, the participants included boys with secondary and elementary education, which limited the generalization to all individuals in the region. Second, all of the inhabitants in Sarbaz spoke another language (Baluchi). Therefore, other studies related to the validity and reliability of the questionnaire is recommended for other parts and groups with different languages and culture in Iran.

Acknowledgments

The authors thank Tarbiat Modarres University for the financial support. We greatly appreciate the collaborations of the Health Deputy of Iranshahr University of Medical Sciences, all the personnel of Health Center in Sarbaz and Rask, the principals in religious science schools in Sarbaz and Rask, and all participants involved in the research.

Authors Contribution

Sayed Mohammad Reza Hosseini wrote the research idea, study plan, data collection, manuscript, statistical analysis, and finalized paper. Shamseddin Niknami contributed to the preparation of the study design and questionnaire, and performed statistical analysis and writing the manuscript. Iraj Zarban played a role in developing the research idea, interpreting the results, reviewing the manuscript, and preparing the final paper. Alireza Heidarnia collaborated in developing the research idea, designing the study, analyzing statistically, and preparing the final version.

Conflict of Interest

The authors declare no conflict of interest.

Funding

The study was supported by a grant from Tarbiat Modares University, Tehran, and received no grant from any agency in the public, commercial, or non-profit sector.

References

1. https://www.who.int/topics/millennium_development_goals/diseases/en/
2. https://www.who.int/malaria/publications/world-malaria-report-2016/report/en/
3. Edrissian G. Malaria in Iran: Past and present situation. Iran J Parasitol 2006; 1:1–14.
4. Haghdoost AA, Alexander N, Cox J. Modelling of malaria temporal variations in Iran. Trop Med Int Health 2008; 13:1501-1508.
5. Khalili MB, Anvari-Tafti MH, Sadeh M. Epidemiological pattern of malarial disease in the province of Yazd, Iran (Since 1986-2006). World J Med Sci 2009; 4:41-45.
6. Hanafi-Bojd AA, Vatandoost H, Oshaghi MA, et al. Knowledge, attitudes and practices regarding malaria control in an endemic area of southern Iran. Southeast Asian J Trop Med Public Health 2011; 42:491–501.
7. Basseri H, Holakouie Naieni K, Raeisi A, et al. Comparison of knowledge, Attitude and Practice (KAP) regarding malaria transmission and protection between Afghan refugees and Iranian residents in Iran shahr, 2005-2006. Iran J Epidemol 2008; 3:7-13.
8. Mohammadi N, Rafiee, Far SH, et al. Health education curriculum (Introduction to applied concepts) 1st Edn. Tehran: Ministry of health and medical education 1: 2004.
9. http://www.sbccimplementationkits.org/demandrmnch/wp-content/uploads/2014/02/Theory-at-a-Glance-A-Guide-For-Health-Promotion-Practice.pdf
10. Rogers RW. A protection motivation theory of fear appeals and attitude change. J Psychol 1975; 91:93-114.
11. Rogers RW, Deckner CW. Effects of fear appeals and physiological arousal upon emotion, attitudes, and cigarette smoking. J Pers Soc Psychol 1975; 32:222–230.
12. Rogers RW, Prentice-Dunn S. Cognitive and physiological processes in fear appeals and attitude change: A revised theory of protection motivation. Handbook of Health Behavior Research: I–Personal and Social Determinants. New York 1983:113–132.
13. Cole ML. Investigating the cognitive processes that mediate protection motivation theory: A parallel-process latent growth modeling analysis. Ph. D. Thesis, Wayne State University 2008.
14. Schutz RW. Methodological issues and measurement problems in sport psychology. International perspectives on sport and exercise psychology. Serpa 1994; 35-57.
15. Eldredge LK, Markham CM, Kok G, et al. Planning health promotion programs: An intervention mapping approach. 4th Edn John Wiley Sons 2016; 1-704.
16. https://psycnet.apa.org/record/1995-97110-004
17. Tinsley HEA, Brown SD. Handbook of applied multivariate statistics and mathematical modeling. 1st Edn Elsevier, 2000.
18. Hajizadeh E MA. Statistical methods and analysis in health and biosciences. A research methodological approach using SPSS practical approach. 1st Edn Tehran, 2011.
19. Lawshe CH. A Quantitative approach to content validity. Pers Psychol 1975; 28:563–575.
20. Waltz CF, Bausell BR. Nursing research: Design statistics and computer analysis. 1981.
21. Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assessment 1995; 7:286-229.
22. https://link.springer.com/chapter/10.1007/978-1-4613-0893-5_6
23. Kazemnejad A, Heidari MR, Norouzadeh R. Statistical methods for healthcare research and application of SPSS in data analysis. In: Munro BH. Statistical methods for healthcare research. 5th Edn Jameenegar Press, 2010; 390.
24. Ghasemi V. Structural models in social science research, Jahad Daneshghahi Press, Isfahan, 2010; 67-112.
25. Kline RB. Principles and practice of structural equation modeling, 3rd Edition, Guilford Press: NewYork, 2010.
26. MacCallum RC, Widaman KF, Zhang S, et al. Sample size in factor analysis. Psychol Methods 1999; 4:84-99.
27. Cronbach LJ, Shavelson RJ. My current thoughts on coefficient alpha and successor procedures. Edu Psychol Measur 2004; 64:391-418.
28. Baumgartner TA, Chung H. Confidence limits for intra class reliability coefficients. Meas Physl Edu Exere Sci 2001; 5:179-188.
29. Milne S, Sheeran P, Orbell S. Prediction and intervention in health-related behavior: A meta-analytic review of protection motivation theory. J Appl Soc Psychol 2000; 30:106–43.
30. Rahaei Z, Ghofranipour F, Morowatisharifabad MA, et al. Psychometric properties of a protection motivation theory questionnaire used for cancer early detection J Sch Public Health Inst Public Health Sci 2015; 12:69-79.
31. DeVellis RF. Scale development: Theory and applications. 2nd Edn Thousand Oaks, CA: Sage Publications; 2003.
32. Tabachnick BG, Fidell LS. Using Multivariate Statistics. 3rd Edn Harpercollins: New York, 1995.
33. Thrul J, Stemmler M, Buhler A, et al. Adolescents’ protection motivation and smoking behavior. Oxford J 2013; 28:683-691.
34. Zare Sakhvidi MJ, Zare M, Mostaghaci M, et al. Psychosocial predictors for cancer prevention behaviors in workplace using protection motivation theory. Adv Prevent Med 2015; 2015:467-498.