 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
| |
|
|
| |
Activator Methods International Ltd.
2950 N. Seventh Street, Suite 200
Phoenix, AZ 85014
ofc 602-224-0220
fax 602-224-0230
toll free 1-800-598-0224
|
| |
|
 |
|
|
Arlan w. Fuhr, D.C., President
Activator Methods, Inc.
3714 E. Indian School Road, Phoenix AZ 85018 USA
(602) 224-0220; AWFuhr@aol.com
Filename: Growing Pains 02/05/17
The concept was relatively straightforward, the execution was meticulous, and the authors' cautions about the limitations in interpreting their findings were clear and appropriate. So, what went wrong?
Meridel Gatterman, D.C. and her team of faculty and private practice clinicians set out to evaluate 10 chiropractic procedures in relation to 15 specific health problems, including acute and chronic disorders of the low back, legs and sacrum (I). It was a next logical step beyond the Mercy Conference guidelines (2), wherein chiropractic methods had been rated in isolation, that is, without regard to the particular clinical conditions to which they might be applied.
The team recruited eight chiropractors (see Table) with a "broad knowledge of chiropractic technique procedures" (I), who were not affiliated with any proprietary technique, to conduct two sets of ratings. (The authors of the study did not participate as raters.) This evaluation panel first judged the quality of the available scientific literature bearing on each of the 150 treatment- by-conditions (10 treatments x 15 disorders), and secondly rated each treatment-by-condition for effectiveness. Each rating was made on an 11-point scale (0-10). In all, a maximum of 2,400 ratings were possible (10 treatments x 15 disorders x 2 kinds of ratings x 8 raters).
Table: Chiropractor-raters of quality of literature and effectiveness
| Thomas F. Bergmann |
Peter Gale |
Margaret Karg |
Leslie Wise |
| Jackie Buettner |
Mitchell Haas |
George McClelland |
Ron Williams |
Unfortunately, something happened on the way to consensus. After reviewing some 172 research articles assembled by search of several databases (CRAC, MANTIS, MEDLINE, and CINAHL), Gatterman and co-authors reduced the list to 139 papers; these included not only randomized controlled clinical trials, but also cohort studies and clinical case series. Confronted by this limited base of evidence, the evaluation panel balked. Although there were only 5 abstentions in the ratings of the literature, the rating process for effectiveness produced 327 abstentions. In other words, in more than 27% (327/1200) of the possible judgments of effectiveness, raters felt they had too little information to make a sound judgment about the usefulness of the procedure for a particular clinical condition.
Indeed, the panel of raters had good reason to be reluctant to draw conclusions in many cases. With only 139 papers to fill 150 cells (10 treatments x 15 disorders), most cells had "no literature in them, and all cells have inadequate literature in them" (1). The resulting matrix resembled a block of Swiss cheese with more holes than cheese.
Nonetheless, the authors computed the mean averages and standard deviations for the ratings within each cell, adjusting the sample size per cell based upon the several hundred abstentions. What resulted were two sets of ratings, one for the quality of the literature and the other for the presumed effectiveness of the 10 chiropractic techniques. The average ratings for each technique by each condition were displayed in tables. However, Gatterman and co-authors cautioned that comparisons among the rankings for treatment procedures and extrapolation to the real world of clinical practice were problematic, owing to several factors, including the paucity of evidence in the literature, the potential non-representativeness of patients studied to those seen in practice, and the fact that in actual practice, specific methods may be used in various combinations, thereby influencing their effectiveness.
The Gatterman team also identified a strong correlation between the strength of the ratings for the quality of the literature and the magnitude of the ratings for effectiveness. In other words, a particular treatment method for a particular clinical condition was rated higher if there was more and better quality evidence to support it. This is not too surprising, but the authors felt compelled to remind readers of the JMPT that "Lack of evidence in the literature is not evidence of lack of effectiveness." Another way of saying this is that just because something hasn't been studied, that doesn't mean that it doesn't work. To draw such conclusion would be to commit what philosophers refer to as an "appeal to ignorance": a logical fallacy in which the absence of evidence is offered as evidence.
The weaknesses and limitations of this project are not a reflection on the quality of work conducted by the authors or the panelists. What "went wrong," if you will, is a reflection of the still meager state of our science: too little hard data to draw firm, evidence-based conclusions concerning the many musculoskeletal problems that fill our offices. And perhaps this should not be too surprising either, given that the history of hard core research in chiropractic is barely two decades old (e.g., 3-6). Gatterman and her team suggest that the greatest value of their project may lie in the identification of specific areas of research we have yet to conduct. Their project also serves as prelude to the next clinical guidelines consensus conference, which will surely have to concern itself with the greater specificity that Gatterman and co-workers attempted.
Unfortunately, this story doesn’t end with the publication of this important and sobering paper. Within days of its publication, the internet was buzzing with various interpretations of the Gatterman project. Apparently, many readers misinterpreted this consensus project as equivalent to research per se, and thought the last word on chiropractic technique had been written. As though to confirm the lack of sophistication in reading scientific literature, one wag, ignoring Gatterman et al.'s admonition that the lack of evidence is not equivalent to lack of effectiveness, went so far as to suggest that the continued use of those chiropractic procedures which received lower ratings amounted to "malpractice " Oh, Lordy!
I was reminded of what transpired at the Consensus Conference on Validation of Chiropractic Methods at Seattle in March 1990 (7), one of the earliest profession-wide attempts to make sense of the wide variety of clinical procedures. People were frightened; some Technique developers arrived with their attorneys! A few imbeciles suggested that all brand- name techniques should be thrown out, since none had adequate experimental support. Fortunately, cooler heads prevailed, and sentiment coalesced around the notion that all techniques should be investigated, and we should retain whatever could be demonstrated to work. Then as now, we seemed to be our own worst enemies.
This time around, after a pause to calm myself, I realized that what we're witnessing are growing pains. Chiropractors have been drawn, kicking and screaming, into this new era of research and accountability, and many of us are ill-prepared by our formal education to deal with the details and nuances of the scientific process. Although our evidence base has been growing (e.g., 8), a mere generation of chiropractors have had access to the profession's premier scholarly periodical, the JMPT (founded in 1978), and most have had little formal training in the interpretation of scholarly works. In my own case, learning about research has been a sometimes painful and stumbling process gained during years of collaboration with trained scientists and clinical investigators. Ironically, these non-DC-mentors have often been kinder and more patient than some of my chiropractic peers. The PhDs have enjoyed the luxury of making their inevitable errors in the relative privacy of their graduate School classes, whereas we in chiropractic seem prone to learn our lessons the hard way: in public.
Let me close here by thanking and congratulating Meridel and her group for taking this next significant step on our path toward a better, more accountable and more effective chiropractic. Our DC-scholars receive little appreciation for doing the hard work that will point the way to a better day for chiropractors and the patients we serve.
References:
1. Gatterman MI, Cooperstein R, Lantz C, Perle SM, Schneider MJ. Rating specific chiropractic technique
procedures for common low back conditions. Journal of Manipulative & Physiological Therapeutics 2001 (Sept); 24(7): 449-56
2. Haldeman S, Chapman-Smith D, Petersen DM (Eds.): Guidelines for chiropractic quality assurance and practice parameters: Proceedings of the Mercy Center Consensus Conference. Gaithersburg MD: Aspen, 1993
3. Gitelman R. The history of chiropractic research and the challenge of today. Journal of the Australian Chiropractors' Association 1984 (Dec); 14(4): 142-6
4. Keating JC. Toward a philosophy of the science of chiropractic: a primer for clinicians. Stockton CA: Stockton Foundation for Chiropractic Research, 1992; Chapter 4: "A brief history of developments in the science of chiropractic"
5. Keating JC, Green BN, Johnson CD. "Research" and "science" in the first half of the chiropractic century. Journal of Manipulative & Physiological Therapeutics 1995 (July/Aug); 18(6): 357- 78
6. Waagen GN, Haldeman S, Cook G, Lopez D, DeBoer KF. Short-term trial of chiropractic adjustments for the relief of chronic low back pain. Manual Medicine 1986; 2(3):63-7
7. Chiropractic Technique 1990 (Aug); 2(3): entire issue
8. Keating JC, Caldwell S, Nguyen H, Saljooghi S, Smith B. A descriptive analysis of the Journal of Manipulative & Physiological Therapeutics, 1989-1996. Journal of Manipulative & Physiological Therapeutics 1998 (act); 21(8): 539-52
|
|
Print
|
|
|
|
|
