Development of a content-valid standardized orthopedic assessment tool (SOAT)
- PMID: 17203268
- DOI: 10.1007/s10459-006-9050-2
Development of a content-valid standardized orthopedic assessment tool (SOAT)
Abstract
Introduction: Content validation of an instrument that measures student performance in OSCE-type practical examinations is a critical step in a tool's overall validity and reliability [Hopkins (1998), Educational and Psychological Measurement and Evaluation (8th ed.). Toronto: Allyn & Bacon].
Purpose: The purpose of the paper is to outline the process employed to create a content-valid Standardized Orthopedic Assessment Tool (SOAT). Orthopedic assessment skills are employed by athletic therapists, physiotherapists and physicians. All follow very similar diagnostic algorithms and that system was used to develop the original SOAT [Cyriax (1982). Textbook of Orthopaedic Medicine, (vol. 1) Bailliere Tindall].
Methods: To validate the tool, the study followed procedures outlined by Violato, Salami, and Muiznieks (2002), Journal of Manipulative Physiological Therapeutics, 25, 111-115, and Butterwick, Paskevich, Vallevand and Lafave (2006), Journal of Allied Health: a modified Ebel procedure. An expert group of athletic therapists from across Canada were chosen to create the content valid tool. Representation from all accredited athletic therapy programs in Canada was sought. Experts participated in three stages of content validation: Stage one consisted of individuals grading tasks on difficulty (hard, medium, easy) and importance (essential, important, not important) for 8 body regions (cervical spine, lumbar spine, shoulder, elbow, wrist/hand/thumb, hip, knee and lower leg/foot/ankle) and three diagnoses for each body region (24 total). If 80% consensus was not achieved in the first stage, then in stage two, face to face discussion is meant to clarify positions and achieved consensus, if possible.
Results: The history component, the observation component, scanning exams, clearing joints above and below the lesion site and active range of motion, passive range of motion and isometric resisted testing all yielded 80% consensus in the first two stages of development. A third stage was added to this process because a new model of measurement was conceptualized during the second stage due to a lack of consensus on two orthopedic assessment categories: special testing and palpation. The new model consisted of a ;;clinical reasoning" metric that tied each section of an orthopedic assessment (history; observation; scanning and clearing; examination, special testing; palpation; conclusion) together and would permit the expert rater to evaluate the student performance based on the student's rationale for tests chosen rather than the traditionally rigid checklists. At least 80% consensus was achieved with the new ;;clinical reasoning" metric and the originally contentious special testing and palpation categories.
Conclusions: The new SOAT that underwent content validation may be a hybrid between the traditional OSCE-type checklists and global scales that provide a more realistic and thus more valid depiction of orthopedic assessment clinical competence. Now that content validation has been established, the next steps include inter-rater reliability testing.
Similar articles
-
Content validity of the Rodeo-SCAT.Int J Sports Med. 2013 Feb;34(2):170-5. doi: 10.1055/s-0032-1311651. Epub 2012 Sep 7. Int J Sports Med. 2013. PMID: 22960990
-
Initial reliability of the Standardized Orthopedic Assessment Tool (SOAT).J Athl Train. 2008 Sep-Oct;43(5):483-8. doi: 10.4085/1062-6050-43.5.483. J Athl Train. 2008. PMID: 18833311 Free PMC article.
-
Development of a tool to support holistic generic assessment of clinical procedure skills.Med Educ. 2008 Jun;42(6):619-27. doi: 10.1111/j.1365-2923.2008.03023.x. Epub 2008 Apr 23. Med Educ. 2008. PMID: 18435710 Review.
-
Design and validation of 2 objective structured clinical examination stations to assess core undergraduate examination skills of the hand and knee.J Rheumatol. 2007 Feb;34(2):421-4. J Rheumatol. 2007. PMID: 17304661
-
Development of content-valid technical skill assessment instruments for athletic taping skills.J Allied Health. 2006 Fall;35(3):147-55. J Allied Health. 2006. PMID: 17036669
Cited by
-
A set of pedagogical recommendations for improving the integrated approach to childhood overweight and obesity: A Delphi study.PLoS One. 2020 Apr 27;15(4):e0231245. doi: 10.1371/journal.pone.0231245. eCollection 2020. PLoS One. 2020. PMID: 32339183 Free PMC article.
-
Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments.MedEdPORTAL. 2019 Apr 12;15:10817. doi: 10.15766/mep_2374-8265.10817. MedEdPORTAL. 2019. PMID: 31139736 Free PMC article.
-
Concurrent Validation of the Banff Patella Instability Instrument to the Norwich Patellar Instability Score and the Kujala Score in Patients With Patellofemoral Instability.Orthop J Sports Med. 2016 May 11;4(5):2325967116646085. doi: 10.1177/2325967116646085. eCollection 2016 May. Orthop J Sports Med. 2016. PMID: 27231700 Free PMC article.
-
Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology.BMC Med Res Methodol. 2016 May 20;16:56. doi: 10.1186/s12874-016-0165-8. BMC Med Res Methodol. 2016. PMID: 27206853 Free PMC article.
-
Validation of the Continuum of Care Conceptual Model for Athletic Therapy.J Sports Med (Hindawi Publ Corp). 2015;2015:391459. doi: 10.1155/2015/391459. Epub 2015 Jul 26. J Sports Med (Hindawi Publ Corp). 2015. PMID: 26464897 Free PMC article.
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Medical