Addressing challenges in inter-rater reliability in Traditional Chinese Medicine

Diagnostic agreement between practitioners is an ongoing challenge in the evaluation of Traditional Chinese Medicine (TCM) and other healthcare systems that rely on constitutional types. The topic is the subject of a Special Focus Issue on Challenges in Inter-Rater Reliability in Traditional Chinese Medicine that is published as the November 2019 issue of JACM, The Journal of Alternative and Complementary Medicine.

The Special Issue was led by Guest Editors Rosa Schnyer, DAOM, LAc, IFMCP, Clinical Assistant Professor in the School of Nursing at the University of Texas and adjunct faculty at the Oregon College of Oriental Medicine and Bastyr University and Claudia Citkovitz, Ph.D., MS, LAc, Department of Rehabilitation Medicine, NYU Medical School, and Director of the Acupuncture Program at Lutheran Medical Center. The special issue includes an exploratory editorial from the Guest Editors, 4 invited commentaries, a systematic review, and 7 original articles.

In their editorial, “Inter-Rater Reliability in Traditional Chinese Medicine: Challenging Paradigmatic Assumptions,” Schnyer and Citkovitz note the issue’s timeliness. The WHO’s newly updated International Classification of Diseases (ICD-11) includes standardized terminology for Traditional Chinese Medicine (TCM) pattern diagnosis. This change reflects increasing prevalence of TCM’s use in the United States and other countries throughout the world and “stronger evidence of acupuncture’s effect on pain over and above placebo and increasing integration of TCM into mainstream healthcare systems.” Yet, as their editorial shares, the issue of reliability is fraught with multiple paradigmatic challenges.

The systematic review, from a 6-member team led by Eric Jacobson, Ph.D., MPH, Department of Global Health and Social Medicine, Harvard Medical School, “Experimental Studies of Inter-Rater Agreement in Traditional Chinese Medicine: A Systematic Review,” found 21 studies that met the criterion set by the 6-member team. They found that “with few exceptions, the levels of agreement were low to moderate.” Noting “significant deficits” in methods, they urge future studies to have “better experimental controls and more thorough reporting of outcomes” together with increased utilization of “methods of complex systems analysis.”

A trio of articles led by Australian researcher Michael Popplewell, Ph.D., Faculty of Health, School of Life Sciences, University of Technology, Sydney, Australia, suggests that new statistical approaches may be more suited to the work and concludes with “A Novel Approach to Describing Traditional Chinese Medical Patterns: The ‘Traditional Chinese Medical Diagnostic Descriptor’.” After applying the methods previously recommended, they conclude that “simple agreement was significantly greater” than that obtained with the typical TCM format.

Three of the commentaries were invited by Schnyer and Citkovitz to respond to subsets of the original submissions: Steve Birch, Ph.D. from Kristiania University College, Oslo, Norway, on the Popplewell set; Ayurvedic medicine clinician and researcher Jennifer Rioux, Ph.D., AD, AYT, C-IAYT, CCA-III on the Jacobson and two other studies examining reliability; and Scott Mist, Ph.D., MAOM, MS, MA from the Department of Anesthesiology and Perioperative Medicine, Oregon Health & Science University on two studies using big data approaches.

Source: Read Full Article