Initial Psychometric Evaluation of the Portuguese Version of the Canine Orthopedic Index

Abstract

Objective The aim of this study was to perform the initial psychometric evaluation of the Portuguese version of the Canine Orthopaedic Index (COI).

Study Design In a prospective, randomized study, a group of 50 police working dogs with bilateral hip osteoarthritis was assessed. A copy of the translated Portuguese version of the COI was completed by canine handlers, native Portuguese speakers. This result was compared with that of 50 sound dogs. Construct validity was determined by comparing sound dogs and dogs with osteoarthritis scores with the Mann–Whitney test. Additionally, the Kaiser–Meyer–Olin measure of sampling adequacy, eigenvalue, and scree-plot analysis were performed. Internal consistency was tested with Cronbach's α.

Results A significant difference was observed between osteoarthritis and sound dogs in all dimensions of the COI (p < 0.01 for all), indicating construct validity. Two factors accounted for 87.8% of the total variance. Cronbach's α was 0.98, with a high interitem correlation, accounting for strong internal consistency.

Conclusion We presented criterion and construct validity of the Portuguese version of the COI. It is a step in providing clinicians with a validated and accessible tool to evaluate orthopaedic patients. Further studies are required.

Keywords

dog - osteoarthritis hip - Canine Orthopedic Index - clinical metrology instruments - observer-related outcome measure

Introduction

Having valid and reliable quantitative outcome measures is crucial for the clinician, improving the quality of diagnosis and the development, testing, and monitoring of treatment interventions.[1] As one component of the evaluation of small animal orthopaedic patients, several orthopaedic clinical metrology instruments or observer-related outcome measures have been developed in a patient-centred approach, allowing an observer to describe how a dog has benefited from a treatment.[2] [3] A clinical metrology instrument is a questionnaire comprising several questions or items, scored according to an observer's experiences or observations. The individual scores are then used to calculate an overall instrument score.[4] Additionally, they can complement other measures, such as objective measures, adding information aiming to evaluate the dog's general demeanour and activity in an everyday environment, and daily activities.[4] [5] [6] Several instruments have been developed and used in dogs as a part of the initial evaluation and assessment response to treatment.[7] [8] Some, such as the Canine Brief Pain Inventory or the Helsinki Chronic Pain Index, focus on assessing pain. Similar to the Liverpool Osteoarthritis in Dogs, others evaluate the impact of osteoarthritis (OA) pain on a dog's daily activities.[9] [10] Sporting and working dogs are at an increased risk of orthopaedic disease compared with pet dogs, as their musculoskeletal structures are under increased stress due to the activities they are involved in.[11] [12] [13] [14] For that reason, it makes sense to have a questionnaire that better encompasses the impact that orthopaedic disease has. A recent report evaluating six observer-related outcome measures determined that the Canine Brief Pain Inventory, the Liverpool Osteoarthritis in Dogs, and Canine Orthopedic Index (COI) can be recommended for use in dogs with OA.[15]

The original version of the COI has been developed using the standard methodology for developing a health assessment questionnaire to reliably measure an owner's assessment of outcome in a dog with orthopaedic disease. Questions were generated from the results of focus groups of owners and veterinarians.[6] The items were subjected to factor analysis, reliability, and validity testing and shown to have excellent reliability and validity, differentiating animals with OA and sound animals.[16] [17] The final version comprises four factors—stiffness, gait, function, and quality of life—with 16 items (questions).[16] [17] Four questions sum up to deliver the stiffness score, five questions provide the gait score, four questions add up to the function score, and three questions compose the quality-of-life score.[1] It has also been used to evaluate response to treatment in patients with OA, comparing a nonsteroidal anti-inflammatory drug to a placebo.[1]

If a clinical metrology instrument is translated, several properties must be reassessed in the target population after translation into a local language.[18] After an instrument is translated, it must be evaluated if it measures what it is supposed to measure in the same way for all individuals.[19] After translation, the psychometric properties must be assessed, which can be performed through different approaches. To determine if an instrument is measuring what it is proposed to evaluate, instrument validity is assessed.[4] [20] Face validity is judged by a group of experts that assess if the scale set looks reasonable for the purpose set. If the attribute we aim to evaluate cannot be directly observed, construct validity is evaluated.[21] Commonly, construct validity is assessed through factor analysis and internal consistency through Cronbach's α.[2] [4] [20] The instrument's reliability must be evaluated to determine if the questionnaire delivers consistent results.[21]

This study aimed to perform initial psychometric testing of a Portuguese version of the COI. As Portuguese is spoken by 261 million people as a primary language, there would be an enormous potential in using the Portuguese COI for studies on canine orthopaedic conditions around the world.[22] We hypothesized that the Portuguese version of the COI would show a comparable validity and reliability to the English COI.

Materials and Methods

The study protocol was approved by the Ethical Review Committee of the University of Évora (Órgão Responsável pelo Bem-estar dos Animais da Universidade de Évora, approval no. GD/37187/2021/P1) and complies with relevant institutional, national, and ARRIVE (Animal Research: Reporting of In Vivo Experiments) guidelines for the care and use of animals. Written informed consent was obtained from the institution responsible for the animals. Permission to translate the COI into Portuguese was obtained from the copyright holder, the American College of Veterinary Surgeons. The original version is freely available online (https://www.vet.upenn.edu/docs/default-source/VCIC/canine-orthopedicorthopaedic-index-weekly-reformat.pdf?sfvrsn=6). A bilingual researcher translated the original English version into Portuguese, and then a bilingual reviewer backward translated it into the original language.[18] [21] Both translators were familiar with orthopaedic disease in dogs. The translations were reviewed until a consensus around the Portuguese version was reached. The Portuguese version of the COI is presented in [Appendix Table A1] (available in online version only). Each question has five different response options corresponding to a 1 to 5 (Likert) score scale.[6] The Portuguese COI was structured in the same way. A convenience sample of 100 police working dogs was selected; 50 were sound dogs and 50 were dogs with bilateral hip OA. All dogs were submitted to a physical, orthopaedic, and neurological examination, haematology, and serum biochemistry to rule out other possible diseases. For the diagnosis of bilateral hip OA, several criteria had to be met: a consistent history of difficulty rising, jumping, and maintaining obedience positions, stiffness, and decreased overall performance; a physical examination showing pain during joint mobilization, stiffness, and reduced range of motion; and radiographic findings based on the Orthopaedic Foundation for Animals grading scheme (sound dogs must have grades of excellent or good, and OA dogs grades of mild, moderate, or severe).[23] Additional inclusion criteria were a body weight ≥ 20 kg, age > 2 years, and a period > 6 weeks without receiving any medication or nutritional supplements. All inclusion criteria had to be met to include the animal in the study. A dog was not included in the study if any other disease was suspected. For all dogs, the canine handlers completed a single online copy of the Portuguese COI, prepared with Google Docs. The choice for a digital version was made mainly to provide handlers with the possibility of an anonymous completion of the Portuguese COI, thus reducing possible bias. All handlers were native Portuguese speakers.

Construct validity was evaluated by different means: results of sound and OA dogs were compared with the Mann–Whitney test to test the hypothesis that the Portuguese COI can differentiate sound dogs from dogs clinically affected by bilateral hip OA. A p < 0.05 was set. Factor analysis was performed to explore the relationship between the different items of the instrument.[24] The Kaiser–Meyer–Olin measure of sampling adequacy was performed to test if the data were adequate for factor analysis, with adequacy considered >0.6. Eigenvalue and scree-plot analysis were used to assess extracted values, and item loading on the extracted components was based on a varimax-rotated model of factor analysis, with a communality cut-off value set of 0.4. The Spearman's correlation rank was used to assess the correlation between items, with 0.40–0.59 being considered moderate, 0.60–0.79 strong, and 0.80–1.0 very strong correlation. Reliability was assessed by testing the internal consistency with Cronbach's α, evaluating the consistency of observed results. A value of at least 0.8 was considered reliable.[4] [18] [21] All results were analysed with IBM SPSS Statistics version 20.

Results

In this sample of 100 dogs, both sexes were represented (55% males and 45% females), with a mean age of 7.4 ± 3.2 years and a body weight of 24.1 ± 7.2 kg. Four breeds were represented: German Shepherd dogs (n = 34), Belgian Malinois Shepherd dogs (n = 30), Labrador Retriever (n = 20), and Dutch Shepherd dogs (n = 16). Each handler was responsible for a single dog.

A significant difference was observed between dogs affected by hip OA and sound dogs with all dimensions (p < 0.01), with sound dogs showing lower scores. Kaiser–Meyer–Olkin factor analysis was 0.95. As all values were above 0.8, factor analysis was conducted. The varimax-rotated model of factor analysis identified two factors with an eigenvalue > 1, accounting for 87.8% of the variance (78.9 and 8.9%, respectively). These two factors corresponded to the first two questions of the COI. The remaining factors have eigenvalues < 0.6. A scree plot ([Fig. 1]) confirmed the retention of the two factors. Based on the varimax-rotated solution, loading for these two items was performed. All items loaded heavily on the first component, with communality ranging between 0.61 and 0.97. Cronbach's α was 0.98. A strong interitem correlation was observed and is presented in [Appendix Table 2] (available in online version only).

Fig. 1 Scree plot of factor analysis of the Portuguese version of the Canine Orthopaedic Index. Two factors had eigenvalues > 1, with a discernible “shoulder” observed. The two factors with eigenvalues > 1 accounted for 87.8% of the variance (78.9 and 8.9%, respectively).

Discussion

The validation of a new instrument or the translation into a new language goes through several steps. Factor analysis resulted in the extraction of two components, different from the four previously described.[16] This is not an unusual finding when different populations are considered. The population considered in this study is quite singular, constituted by a very homogenous set of animals in terms of breeds, diagnosis, housing conditions, feeding, and activity levels. An additional contributing factor for these differences may be the observer completing the instrument. In dogs, these instruments are completed by an observer, usually the owner, based on the assumption that they can identify degrees and changes of their pets' subjective states. Still, the observer's ability to detect these changes can influence the results of the instrument.[25] [26] As in this study the instrument was completed by experienced dog handlers, it is reasonable to consider that they have a good ability to detect changes in the dog's demeanour and overall activity. But this fact has to be kept in mind in future evaluations of this translated version of the COI, as pet owners are unlikely to have the same experience as dog handlers, which may affect the overall results.

Factor analysis was also conducted as an alternative measure of construct validity, showing extraction of two factors with eigenvalues greater than 1 and through scree-plot analysis, the first two items of the COI. It is possible that the Portuguese version of the COI has different domains. However, this can also be related with the homogeneous nature of sample. Since the present study is an initial evaluation, we plan to explore this possibility in a follow-up study, comparing the results obtained with the Portuguese COI to an objective measure and evaluating response to treatment.

Item loading of the components showed a high correlation with the factors, also supported by the good interitem correlations and Cronbach's α.[21] Comparing the sound and OA dogs results, the COI scores of the two populations were significantly different. While the purpose of the COI is to make intrapatient comparisons rather than intergroup comparisons, this is an important part of instrument validation, showing the ability to differentiate sound dogs from dogs with disease. Interestingly, but not surprisingly, a strong to very strong correlation between the different dimensions of the COI was observed. This sample was composed exclusively of OA patients, where pain is the most relevant clinical sign, and pain is a complex, multidimensional experience, with an impact on sensory, evaluative, and affective components.[20] [27] [28] For that reason, it could be expected to see an effect on all dimensions of the COI.

This study presents some limitations, namely the relatively homogenous nature of the sample. With that in mind, this Portuguese COI should be applied and tested in a larger population, with patients with more heterogeneous characteristics, to determine if it behaves similarly across breeds, for example. Likewise, as this sample was composed exclusively of patients with hip OA, it would be interesting to evaluate if its broader use is reliable for OA in general, as has been done for other instruments, and orthopaedic diseases as a whole.[4] It would also be necessary to compare the results obtained to an objective measure, as force plait gait analysis or weight-bearing distribution evaluation, considered the gold standard for limb function, further contributing to the validation of the COI. This ongoing process should be continued in future studies, evaluating other psychometric properties as construct validity by hypothesis testing, criterion validity, reproducibility, and stability.[29] The ability of the Portuguese COI to determine response to treatment and over time also has to be determined, similarly to what has been done to the English version and adaptations to other languages. This ability of an instrument is known as responsiveness.[1] [17] Still, we presented enough data showing that the Portuguese version of the COI addresses the clinical manifestations of orthopaedic disease, namely OA affecting the hip joint, which can differentiate sound dogs from diseased dogs.

Conclusion

We evaluated and determined criterion and construct validity of the Portuguese COI, validating its use in the Portuguese language. Further studies are required to determine the results in samples with different characteristics, compare the results with an objective measure, and evaluate response to treatment.

Conflict of Interest

None declared.

Acknowledgment

The authors would like to thank Manuel Pereira for the assistance in the analysis of the data.

Authors' Contribution

J.C.A. designed the protocol, conducted treatments, and prepared the manuscript.

• References

• 1 Brown DC. The Canine Orthopedic Index. Step 3: responsiveness testing. Vet Surg 2014; 43 (03) 247-254
• 2 Stadig S, Lascelles BDX, Nyman G, Bergh A. Evaluation and comparison of pain questionnaires for clinical screening of osteoarthritis in cats. Vet Rec 2019; 185 (24) 757-757
• 3 Gruen ME, Griffith EH, Thomson AE, Simpson W, Lascelles BDX. Criterion validation testing of clinical metrology instruments for measuring degenerative joint disease associated mobility impairment in cats. PLoS One 2015; 10 (07) e0131839
• 4 Walton MB, Cowderoy E, Lascelles D, Innes JF. Evaluation of construct and criterion validity for the ‘Liverpool Osteoarthritis in Dogs’ (LOAD) clinical metrology instrument and comparison to two other instruments. PLoS One 2013; 8 (03) e58125
• 5 Brown DC, Bell M, Rhodes L. Power of treatment success definitions when the Canine Brief Pain Inventory is used to evaluate carprofen treatment for the control of pain and inflammation in dogs with osteoarthritis. Am J Vet Res 2013; 74 (12) 1467-1473
• 6 Brown DC. The Canine Orthopedic Index. Step 1: devising the items. Vet Surg 2014; 43 (03) 232-240
• 7 Alves JC, Santos A, Jorge P, Lavrador C, Carreira LM. Clinical and diagnostic imaging findings in police working dogs referred for hip osteoarthritis. BMC Vet Res 2020; 16 (01) 425
• 8 Alves JC, Santos A, Jorge P, Lavrador C, Carreira LM. Intra-articular injections with either triamcinolone hexacetonide, stanozolol, hylan G-F 20, or a platelet concentrate improve clinical signs in police working dogs with bilateral hip osteoarthritis [retracted in: Front Vet Sci 2023;9:1130041]. Front Vet Sci 2021; 7: 609889
• 9 Walton B, Cox T, Innes J. ‘How do I know my animal got better?’ – measuring outcomes in small animal orthopaedics. In Pract 2018; 40 (02) 42-50
• 10 Meeson RL, Todhunter RJ, Blunn G, Nuki G, Pitsillides AA. Spontaneous dog osteoarthritis - a One Medicine vision. Nat Rev Rheumatol 2019; 15 (05) 273-287
• 11 Sellon DC, Martucci K, Wenz JR, Marcellin-Little DJ, Powers M, Cullen KL. A survey of risk factors for digit injuries among dogs training and competing in agility events. J Am Vet Med Assoc 2018; 252 (01) 75-83
• 12 Baltzer WI, Owen R, Bridges J. Survey of handlers of 158 police dogs in New Zealand: functional assessment and Canine Orthopedic Index. Front Vet Sci 2019; 6 (April): 85
• 13 Pechette Markley A, Shoben AB, Kieves NR. Internet-based survey of the frequency and types of orthopedic conditions and injuries experienced by dogs competing in agility. J Am Vet Med Assoc 2021; 259 (09) 1001-1008
• 14 Alves JC, Dos Santos AM, Fernandes ÂD. Evaluation of the effect of mesotherapy in the management of back pain in police working dog. Vet Anaesth Analg 2018; 45 (01) 123-128
• 15 Radke H, Joeris A, Chen M. Evidence-based evaluation of owner-reported outcome measures for canine orthopedic care - a COSMIN evaluation of 6 instruments. Vet Surg 2022; 51 (02) 244-253
• 16 Brown DC. The Canine Orthopedic Index. Step 2: psychometric testing. Vet Surg 2014; 43 (03) 241-246
• 17 Andersson A, Bergström A. Adaptation of the Canine Orthopaedic Index to evaluate chronic elbow osteoarthritis in Swedish dogs. Acta Vet Scand 2019; 61 (01) 29
• 18 Essner A, Zetterberg L, Hellström K, Gustås P, Högberg H, Sjöström R. Psychometric evaluation of the canine brief pain inventory in a Swedish sample of dogs with pain related to osteoarthritis. Acta Vet Scand 2017; 59 (01) 44
• 19 Krogsgaard MR, Brodersen J, Christensen KB. et al. How to translate and locally adapt a PROM. Assessment of cross-cultural differential item functioning. Scand J Med Sci Sports 2021; 31 (05) 999-1008
• 20 Reid J, Nolan AM, Scott EM. Measuring pain in dogs and cats using structured behavioural observation. Vet J 2018; 236: 72-79
• 21 Ragetly GR, Massey L, Brown DC. Initial psychometric testing and validation of the French version of the Canine Brief Pain Inventory. Vet Anaesth Analg 2019; 46 (05) 667-672
• 22 Camões I. Uma língua para o Mundo. Published 2021. https://www.instituto-camoes.pt/images/eplp/Diptico_dlp16.pdf Accessed on November 20th, 2021
• 23 Puckler K, Tellhelm B, Kirberger R. The hip joint and pelvis. In: Kirberger R, McEvoy F. eds. BSAVA Manual of Canine and Feline Musculoskeletal Imaging. Wiley; 2016: 212-231
• 24 Tavakol M, Wetzel A. Factor analysis: a means for theory and instrument development in support of construct validity. Int J Med Educ 2020; 11: 245-247
• 25 Albuquerque N, Guo K, Wilkinson A, Savalli C, Otta E, Mills D. Dogs recognize dog and human emotions. Biol Lett 2016; 12 (01) 20150883
• 26 Wiseman-Orr ML, Nolan AM, Reid J, Scott EM. Development of a questionnaire to measure the effects of chronic pain on health-related quality of life in dogs. Am J Vet Res 2004; 65 (08) 1077-1084
• 27 van Weeren PR. General anatomy and physiology of joints. In: Joint Disease in the Horse. Elsevier; 2015. :1–24
• 28 Piel MJ, Kroin JS, van Wijnen AJ, Kc R, Im HJ. Pain assessment in animal models of osteoarthritis. Gene 2014; 537 (02) 184-188
• 29 Della Rocca G, Di Salvo A, Medori C, Della Valle MF, Cimino Brown D. Initial psychometric testing and validation of the Italian version of the Canine Brief Pain Inventory in dogs with pain related to osteoarthritis. Front Vet Sci 2021; 8: 736458

Publication History

Received: 22 November 2021

Accepted: 14 March 2023

Article published online:
09 May 2023