Translation and cultural validation of the Revised Illness Perception Questionnaire for Healthcare Professionals for Brazilian Portuguese

ABSTRACT

Background: Multiple sclerosis progression and disability can be rated differently by healthcare professionals. Therefore, how physicians perceive the disease can impact treatment decisions. There are no previous studies on this matter. Objective: To translate and transculturally validate the Revised Illness Perception Questionnaire for Healthcare Professionals (IPQ-R HP), for use in Brazilian Portuguese. Methods: The process used to validate the IPQ-R HP was based on the steps presented in the guide proposed by Dorcas Beaton. The final version of the IPQ-R HP had 38 questions, divided into seven different dimensions to assess the patient's disease. Also, two clinical cases that were representative of real-life patients with multiple sclerosis (MS) were assembled to consider the two main profiles of the disease. We applied the questionnaire to neurologists at the Federal University of São Paulo (UNIFESP) to assess their perception of MS. These doctors also answered a brief survey to establish the profile of the interviewees. For statistical analysis, we used Bayesian CFA models and kappa statistics. Conclusions: The kappa statistics showed a general agreement of 0.4. For the Bayesian CFAs with seven-factor correlation solution, we had a poor fit for case 1 with a 95% confidence interval ranging from -52.893 to 273.797 and a PPP of 0.107. Regarding case 2, the model did not converge even after 50,000 iterations, which indicated that the specified model (i.e. seven-factor correlation solution) for case 2 was inadmissible. Thus, the IPQ-R HP questionnaire in Brazilian Portuguese has not been validated.

RESUMO

Antecedentes: A progressão da esclerose múltipla e a incapacidade podem ser avaliadas de formas diferentes por médicos. Portanto, a forma como estes percebem a doença pode afetar as decisões de tratamento. Não há estudos anteriores sobre o assunto. Procuramos traduzir e validar o Revised Illness Perception Questionnaire-Revised Healthcare Professionals (IPQ-R HP). Objetivos: Validação transcultural da versão IPQ-R HP para português. Métodos: O processo de validação do IPQ-R HP foi baseado nas etapas apresentadas no guia proposto por Dorcas Beaton. A versão final do IPQ-R HP continha 38 questões, divididas em sete dimensões diferentes para avaliar a doença do paciente. Além disso, dois casos clínicos representativos de esclerose múltipla (EM) foram criados para contemplar os dois perfis principais da doença. Aplicamos o questionário a neurologistas da UNIFESP para avaliar sua percepção sobre a EM, além de uma pesquisa para estabelecer o perfil dos entrevistados. Para a análise estatística, usamos modelos CFA Bayesianos e estatísticas kappa. Conclusões: A estatística kappa mostrou concordância geral de 0,4. Para os CFAs bayesianos com solução de sete fatores correlacionados, tivemos um ajuste ruim para o caso 1 com um intervalo de confiança de 95% variando de -52,893 a 273,797 e o PPP de 0,107. Em relação ao Caso 2, o modelo não convergiu mesmo após 50000 iterações, indicando que o modelo especificado (ou seja, solução de sete fatores correlacionados) para o caso 2 é inadmissível. Assim, o questionário IPQ-R HP em português não é validado.

Keywords:

Multiple Sclerosis - Surveys and Questionnaires - Disease - Perception - Validation Study

Palavras-chave:

Esclerose Múltipla - Inquéritos e Questionários - Doença - Percepção - Estudo de Validação

INTRODUCTION

Multiple sclerosis (MS) is a demyelinating, progressive, degenerative autoimmune disease of the central nervous system (CNS), with prevalence ranging from about 15 to 20/100,000 inhabitants, in Brazil[1]. The progressive nature of the disease, which gradually limits the patient's functionality, makes early treatment with effective drugs necessary[2]. Early and adequate treatment allows a better prognosis for the patient and better use of the resources destined for the disease[3].

Diseases that can present in various ways with diverse symptoms can possibly be interpreted differently by each professional during patient care. For example, Arat et al. demonstrated that doctors who treat patients with systemic lupus erythematosus (SLE) have quite different perceptions of the disease[4],[5]. Those authors developed the Illness Perception Questionnaire-Revised for Healthcare Professionals (IPQ-R HP) to highlight differences in perception of SLE. Thus, they were able to describe how specialists have different views on the consequences, limitations and clinical evolution of the disease and on the ways in which their understandings could affect this disease[5],[6].

Given the clinical characteristics of MS, its symptoms, clinical progression and accumulation of disability over time, an understanding of how neurologists perceive MS can be an important tool for improving therapeutic decisions[7]. Moreover, the peculiarities of the Brazilian National Health System (Sistema Único de Saúde, SUS) and its high-cost drug program make Brazil an unusual country in terms of public health. Every patient in Brazil who is diagnosed with MS receives disease-modifying treatment free of charge. This is a high-cost treatment with a major impact on public spending within government healthcare policies. Therefore, there is a need for accurate treatment, and this makes the physician's perception of the disease a fundamental issue.

To start filling the knowledge gap on how a physician's perception of illness and his/her expertise may interfere with patient care, and to gather information on how multiple sclerosis is perceived by neurologists, we aimed to translate and validate the IPQ-R HP questionnaire for use in Brazilian Portuguese.

Objectives

To translate and culturally validate the Revised Illness Perception Questionnaire for Healthcare Professionals (IPQ-R HP).

METHODS

Validation of the questionnaire

The IPQ-R HP questionnaire developed by Arat et al. consists of 38 questions in a Likert-scale format, through which participants explain their level of agreement with the topics addressed in terms of five possibilities, ranging from "totally disagree" to "totally agree". The questions are distributed in seven different dimensions of the disease, also called domains of the questionnaire: 1) the consequences of the disease for the patient; 2) the perception of time concerning the duration of the disease, whether it has a prognosis of rapid improvement or not; 3) the patient's ability to control his/her illness; 4) the effectiveness of some type of treatment that is carried out in the context of controlling the disease; 5) the individual's understanding of his/her illness; 6) the perception of a possible cyclical nature for the disease, when analyzed over a longer period; and 7) the emotional experience that the patient has regarding his/her illness[6].

The validation process was based on the guide proposed by Beaton et al.[8]. The first step was the translation in Portuguese. The first translation was made by a neurologist who was a specialist in demyelinating diseases. The second translator was made by a sworn translator without knowledge of medicine. The third translation was made by author responsible for this study.

The second step was to produce a synthesis of all the translations, in which the aim was to eliminate discrepancies between the versions and create a unified text.

The third step was to produce a "back-translation", i.e. a translation of the unified version of the text in Portuguese, back to the language in which the questionnaire had been created. This process was necessary in order to evaluate how reliable the text obtained through merging the translations was, in comparison with the original. For this, the translation was done by the same sworn translator.

The fourth stage encompassed all the translators, healthcare professionals, and professionals working with linguistics. The function of this committee was to analyze the differences and to prepare a version of the questionnaire that was considered pre-final. In comparing the two versions, it could be seen that among the 38 items, only two of them presented more significant differences, while most were the same as the original, and a few items differed only in word order. We submitted the documents that reported on the validation process, together with the translated questionnaire, to the original developers of the questionnaire so that they could verify the steps were followed and approve the final content of the translated version.

In addition to the questionnaire, two clinical cases of multiple sclerosis were developed based on real clinical cases of relapsing-remitting and primary progressive MS. These were given to the participants together with the questionnaire. They were created to represent patients under follow-up at the Neuroimmunology Clinic of the Department of Neurology and Neurosurgery, Escola Paulista de Medicina, Federal University of São Paulo (UNIFESP). Thus, each study participant completed the IPQ-R HP twice, i.e. for each clinical case.

All stages of this project, which all involved doctors whose participation was voluntary, were approved by the Research Ethics Committee of UNIFESP, under the number CAAE 86002717.1.0000.5505.

Population

The final Brazilian Portuguese version was applied between April 2018 and November 2019 ([Table 1]). Prospectively and consecutively, neurologists were asked to answer a questionnaire based on the clinical cases presented to them, after signing a consent form.

Table 1 Revised Illness Perception Questionnaire for Healthcare Professionals: Brazilian Portuguese version.

	Discorda fortemente	Discorda	Nem concorda nem discorda	Concorda	Concorda fortemente
1. A doença do meu paciente é grave
2. A doença de meu paciente traz consequências graves a sua vida
3. A doença de meu paciente não causa grandes efeitos sobre sua vida
4. A doença de meu paciente afeta fortemente a forma como os outros o/a veem
5. A doença de meu paciente lhe traz importantes consequências financeiras
6. A doença de meu paciente causa dificuldades àqueles próximos a ele/a
7. A doença de meu paciente durará pouco tempo
8. A doença de meu paciente será permanente e não temporária
9. A doença de meu paciente vai durar por um longo período
10. A doença de meu paciente será de breve duração
11. É esperado que a doença de meu paciente dure pelo resto de sua vida
12. A doença de meu paciente irá melhorar ao longo do tempo
13. Meu paciente pode realizar muitas ações para controlar seus sintomas
14. O que meu paciente faz pode determinar se doença irá melhorar ou piorar.
15. A evolução da doença de meu paciente depende dele/a
16. Nada do que meu paciente faça afetará sua doença
17. Meu paciente tem o poder de influenciar sua doença
18. As ações de meu paciente não produzirão nenhum efeito no desfecho de sua doença
19. Há muito pouco que se possa fazer para melhorar a doença de meu paciente
20. O tratamento de meu paciente será efetivo em curar sua doença
21. Os efeitos negativos da doença de meu paciente podem ser prevenidos (evitados) com um tratamento
22. O tratamento de meu paciente pode controlar sua doença
23. Não há nada que possa ajudar a condição de meu paciente
24. A doença de meu paciente é um mistério para ele/a
25. Os sintomas da doença de meu paciente são intrigantes para ele/a
26. Meu paciente não entende sua doença
27. A doença de meu paciente não faz sentido para ele/a
28. Meu paciente tem uma imagem ou compreensão clara de sua doença
29. Os sintomas da doença de meu paciente variam demasiadamente com o passar dos dias
30. Os sintomas de meu paciente vêm e vão em ciclos
31. A doença de meu paciente é muito imprevisível
32. Meu paciente passa por ciclos nos quais a sua doença melhora e piora
33. Meu paciente fica deprimido quando pensa em sua doença
34. Meu paciente fica aborrecido quando pensa em sua doença
35. A doença de meu paciente o deixa bravo
36. A doença de meu paciente realmente o/a preocupa
37. Ter esta doença deixa meu paciente ansioso
38. A doença de meu paciente faz com que ele/a tenha medo

Initially, we applied a physical version of the questionnaire in face-to-face interviews. However, it was observed that such a model was not practical for the subjects to answer during the working day. Therefore, an online version of the questionnaire was developed.

The inclusion criteria were that the subjects needed to be neurologists working in the neurology discipline of the Department of Neurology and Neurosurgery, Escola Paulista de Medicina, UNIFESP; or neurology residents with at least two years of full training and a minimum of five weeks of internship at a demyelinating disease clinic, during their residency. The following physicians were excluded: physicians who were temporarily in an internship of less than 6 months at the institution; and pediatrics neurologists without previous neurology training. Neurologists who claimed to care for more than 10 patients with MS per month were deemed to be MS specialists.

In addition, epidemiological data were collected from the study volunteers, including the following: age, sex, length of time since graduation, postgraduate qualifications, work sector and number of patients with MS treated by the physician.

Statistical analysis

We described categorical data as absolute values (n) and relative frequencies (percentage); and continuous variables in terms of mean and standard deviation. Kappa statistics[9] were used to quantify the level of agreement among raters, considering relapsing-remitting and primary progressive MS cases separately.

A variety of criticisms of kappa statistics have emerged based on different statistical elements. Klein[10] revised Gwet’s framework[11],[12], with discussion of different alternative agreement coefficients (ACs), and this revision was implemented in Stata version 14 (StataCorp, 2009) under the command kappaetc [10]. The percentage of agreement and different variations of kappa were calculated as the following formulations: Brennan and Prediger[13], Cohen[9], Gwet’s AC[11] and Krippendorff’s Alpha[14]). For all the coefficients, unweighted analysis was conducted; therefore, the identity matrix was considered as described by Klein[10], and 95% confidence intervals [CI] were calculated. According to the benchmark scale proposed by Landis & Koch[15], the coefficient interpretations are as follows: below 0.00 indicates poor agreement; 0.00 to 0.20 slight; 0.21 to 0.40 fair; 0.41 to 0.60 moderate; 0.61 to 0.80 substantial; and 0.81 to 1.00 almost perfect. Kappa coefficients for both general neurologists and MS neurologists were calculated.

General model testing

We performed a confirmatory factor analysis (CFA) for evidence of factorial construct validity using the 63 respondents for two cases (1 and 2). In this, the 38 items were considered as ordered-categorical variables. A Bayesian estimator was used rather than the traditional frequentist approach. In Mplus[16], the frequentist default estimator is the weighted least square using a diagonal weight matrix with standard errors and mean and variance-adjusted (WLSMV) when the indicators observed (e.g. items) are dichotomous or ordered-categorical.

According to Brown[17], the Bayesian method has several potential advantages over the traditional frequentist approach. For our study, the Bayesian method may have been better for a small sample, given the number of items under evaluation. For the Bayesian CFA, we used loadings with mean at zero and variance at 0.01, as prior factors. A prior factor is a previously held belief concerning likely parameter values, before collecting related data.

To evaluate the goodness-of-fit for the Bayesian CFA models tested, the following fit indices were used: posterior predictive p-value (PPP) and associated 95% credibility interval[18]. While a low PPP (p < 0.05) and positive 95% lower limit point to a poor model fit, a well-fitting model is expected to show a PPP of around 0.5 and a symmetrical 95% credibility interval centering around zero. We used a minimum number of total iterations of 50,000, including the discards. Thus, depending on when convergence occurs, more than 25,000 iterations could be used for subsequent distribution.

RESULTS

The final Brazilian version of the IPQ-R HP can be seen in [Table 1]. Seventy neurologists were recruited and agreed to participate but only 63 returned the questionnaire completely answered ([Figure 1]).

Figure 1 Diagram showing study recruitment and enrollment.

The neurologists’ profile and characteristics are described in [Table 2]. Briefly, the participants were predominantly male (in the proportions of 2:1) and young (average of 12 years since graduation). Around 24% had not completed their residency and only 20.6% had finished their PhD. The vast majority were working in both the private and the public sector and only 22% of the participants were caring for more than 10 patients with multiple sclerosis per month.

Table 2 Main demographic characteristics of 63 neurologists working at UNIFESP who returned valid answered questionnaires.

Sex	Male: 66.7%
	Female: 33.3%
Age	Average: 35.7 years (range: 27 to 63)
Length of time since graduation	Average: 12 years (range: 4 to 39 years)
Residency	Ongoing: 23.8%
	Concluded: 76.2%
Postgraduate studies (1.6% abstention rate)	Without/Ongoing: 49.2%
	Master’s: 28.6%
	Doctorate: 20.6%
Work sector (6.3% abstention rate)	Private healthcare: 6.3% Public healthcare: 9.6%
	Both: 77.8%
Number of patients with MS treated (6.5% abstention rate)	< 10/month: 33.3%
	> 10/month: 22.2%
	< 10/year: 26.9%
	> 10/year: 11.1%

MS: multiple sclerosis.

The kappa statistics showed a general agreement of 0.4 for both cases 1 and 2. The kappa values for the neurologists belonging to the MS specialists' group were 0.48 and 0.5, for cases 1 and 2, respectively. For the Bayesian CFAs with seven- factor correlation solution, we had poorly fitting models as follows: case 1 showed a 95% confidence interval for the difference between the observed and the replicated chi-square values ranging from -52.893 to 273.797, with a PPP of 0.107. In the case of Case 2, the model did not converge even after 50,000 iterations, thus indicating that the model specified (i.e. seven-factor correlation solution) for case 2 was inadmissible.

DISCUSSION

The perception of illness can be defined as the way in which an individual perceives and mentally plans to live with an illness[19]. Beliefs and attitudes are predictors of treatment adherence in relation to different chronic diseases, such as SLE, asthma and Crohn's disease[19]. However, assessing the effect of the healthcare professional's conceptualizations on treatment and clinical evolution is an arduous task. For example, neurologists and psychiatrists have different views on epilepsy and this can lead to discrepancies that affect the treatment of patients who are followed up by both specialists[20].

In multiple sclerosis, the perception of disease is associated with and influences the interpretation of symptoms and treatment effects[21],[22]. In our study, we aimed to translate the disease perception questionnaire for healthcare professionals (IPQ-R HP) into Brazilian Portuguese and validate it[6]. Furthermore, we tried to assess the perceptions that neurologists working in tertiary teaching hospital would have of a complex neurological disease, such as multiple sclerosis.

Overall, we observed a kappa of 0.4, i.e. a percentage agreement of 40%. The level of agreement was slightly higher among specialists, ranging from 48 to 50%, than among non-specialists. This suggesting that the level of agreement was acceptable but not enough to consider the questionnaire validated. Thus, we need to consider that the sample size and the complexity of the questionnaire were limiting factors.

On the other hand, the characteristics of our population may have been one explanation for our results. It was a young population, with an average of 12 years of neurology practice and little MS training. The small difference observed between general neurologists and specialists in multiple sclerosis suggests that the study population could have included a larger number of neurologists with experience in the care of patients with MS, which thus might have enabled validation of the questionnaire. However, our result reinforces the idea that motivated this study in the first place: neurologists have different perceptions about multiple sclerosis, and this may affect the treatment and care provided for patients.

In terms of CFA, our sample size was small, even though we used a Bayesian approach. The five categories of responses and the large number of items (38) in relation to such a small sample size generated difficulties in estimating item thresholds and factor loadings. This explains the non-convergence for case 2. Differently from case 2, case 1 returned poor fit indices, meaning that the model at least converged.

Some ideas for the future specification of the questionnaire might include reducing the number of items, especially the items with similar wording within the same domain of evaluation. Moreover, the number of response categories per item could be reduced from 5 to 3, in addition to reducing the number of items. This would consequently reduce the complexity of the questionnaire and generate a more parsimonious model.

It was seen that some items did not show the presence of the full range of response categories (for example, items 6 and 8, in case 2). This indicates that the neurologists chose not to use the full spectrum of response options that were available to them. Generally, the most common response categories were at the endpoints (1 or 5). Thus, intermediate responses were not being endorsed and, consequently, were not informative about the factors under investigation.

Moreover, the original questionnaire was developed to assess SLE and it is possible that some of its domains were inappropriate for capturing some matters of relevance for rating with regard to multiple sclerosis[5].

Our results are modest but point to an interesting path. The illness perception questionnaire needs to be improved, and a better, shorter version should be tested in the future in order to investigate whether the slight difference observed among neurologists will prevail.

In conclusion, the IPQ-R HP questionnaire (Revised Illness Perception Questionnaire for Healthcare Professionals) in its Brazilian Portuguese version showed acceptable agreement, but not adequate for its validation.

Conflict of interest:

Fernando De Nigris Vasconcellos received a scientific scholarship from São Paulo Research Foundation (FAPESP), grant number 2017/13838-3; Felipe Gregório Lima has nothing to disclose; Enedina Maria Lobato de Oliveira has received speaker fees and consultant honoraria from Merck and Sanofi, and travel expenses to scientific meetings sponsored by Merck.

ACKNOWLEDGMENT

The authors would like to thank the University of Leuven, Belgium, for providing the original questionnaire, granting permission for its translation and enabling the study.

Authors’ contributions:

EMLO: had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis; FNV: had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis; FGL: had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

• REFERENCES

• 1 Pereira ABCNG, Lacativa MCS, Pereira FFC, Alvarenga RMP. Prevalence of multiple sclerosis in Brazil: a systematic review. Mult Scler Relat Disord. 2015 Nov 1;4(6):572-9. https://doi.org/10.1016/j.msard.2015.08.004
• 2 Biotti D, Ciron J. First-line therapy in relapsing remitting multiple sclerosis. Rev Neurol (Paris). 2018 Jun;174(6):419-28. https://doi.org/10.1016/j.neurol.2018.03.012
• 3 Bargiela D, Bianchi MT, Westover MB, Chibnik LB, Healy BC, De Jager PL, et al. Selection of first-line therapy in multiple sclerosis using risk-benefit decision analysis. Neurology. 2017 Feb 14;88(7):677-84. https://doi.org/10.1212/WNL.20200451202004513612
• 4 Arat S, Moons P, Vandenberghe J, Lenaerts JL, de Vlam K, Westhovens R. Diverging illness perceptions between physicians about patients with systemic lupus erythematosus and systemic sclerosis: a vignette-based study. Rheumatol Int. 2017 Jun;37(6):915-22. https://doi.org/10.1007/s00296-017-3667-8
• 5 Arat S, Lenaerts JL, De Langhe E, Verschueren P, Moons P, Vandenberghe J, et al. Illness representations of systemic lupus erythematosus and systemic sclerosis: a comparison of patients, their rheumatologists and their general practitioners. Lupus Sci Med. 2017 Nov 14;4(1):e000232. https://doi.org/10.1136/lupus-2017-000232
• 6 Arat S, Van den Zegel A, Van Rillaer M, Moons P, Vandenberghe J, De Langhe E, et al. Development and preliminary evaluation of the validity and reliability of a revised illness perception questionnaire for healthcare professionals. BMC Nurs. 2016 Jun 1;15:34. https://doi.org/10.1186/s12912-016-0156-4
• 7 Vasconcelos CCF, Thuler LCS, Rodrigues BC, Calmon AB, Alvarenga RMP. Multiple sclerosis in Brazil: a systematic review. Clin Neurol Neurosurg. 2016 Dec;151:24-30. https://doi.org/10.1016/j.clineuro.2016.07.011
• 8 Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000 Dec 15;25(24):3186-91. https://doi.org/10.1097/00007632-200012150-00014
• 9 Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37-46. https://doi.org/10.1177/001316446002000104
• 10 Klein D. Implementing a general framework for assessing interrater agreement in Stata. Stata J. 2018 Dec 1;18(4):871-901. https://doi.org/10.1177/1536867X1801800408
• 11 Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008 May;61(Pt 1):29-48. https://doi.org/10.1348/000711006X126600
• 12 Gwet LK. Handbook of inter-rater reliability: the definitive guide to measuring the extent of agreement among raters. 4th ed. Advanced Analytics, LLC; 2014. 426 p.
• 13 Brennan RL, Prediger DJ. Coefficient kappa: some uses, misuses, and alternatives. Educ Psychol Meas. 1981 Oct 1;41(3): 687-99. https://doi.org/10.1177/001316448104100307
• 14 Krippendorff K. Estimating the reliability, systematic error and random error of interval data. Educ Psychol Meas. 1970 Apr 1;30(1):61-70. https://doi.org/10.1177/001316447003000105
• 15 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977 Mar;33(1):159-74.
• 16 Muthén KL, Muthén BO. Mplus the comprehensive modelling program for applied researchers: user’s guide. 5th ed. Muthén & Muthén; 2012.
• 17 Brown AT. Confirmatory factor analysis for applied research. 2nd ed. Guilford Publications; 2015. 462 p.
• 18 Muthén B, Asparouhov T. Bayesian structural equation modeling: a more flexible representation of substantive theory. Psychol Methods. 2012 Sep;17(3):313-35. https://doi.org/10.1037/a0026802
• 19 Shahin W, Kennedy GA, Stupans I. The impact of personal and cultural beliefs on medication adherence of patients with chronic illnesses: a systematic review. Patient Prefer Adherence. 2019 Jul 1;13:1019-35. https://doi.org/10.2147/PPA.S212046
• 20 Whitehead K, Reuber M. Illness perceptions of neurologists and psychiatrists in relation to epilepsy and nonepileptic attack disorder. Seizure. 2012 Mar 1;21(2):P104-9. https://doi.org/10.1016/j.seizure.2011.09.012
• 21 Wilski M, Kocur P, Górny M, Koper M, Nadolska A, Chmielewski B, et al. Perception of multiple sclerosis impact and treatment efficacy beliefs: mediating effect of patient's Illness and self-appraisals. J Pain Symptom Manage. 2019 Sep 1;58(3):437-44. https://doi.org/10.1016/j.jpainsymman.2019.06.013
• 22 Wilski M, Kocur P, Brola W, Tasiemski T. Psychological factors associated with self-management in multiple sclerosis. Acta Neurol Scand. 2020 Jul;142(1):50-7. https://doi.org/10.1111/ane.13236

Publication History

Received: 21 September 2020

Accepted: 05 November 2020

Article published online:
04 July 2023