Viewpoint
Oct 2004

Evidence-Based Surgery: a Growing Need for a Limited Enterprise

Angelique M. Reitsma, MD, MA
Virtual Mentor. 2004;6(10):465-469. doi: 10.1001/virtualmentor.2004.6.10.oped1-0410.

 

The call for rigorous scientific testing of surgical therapies is not new,1,2 but has seen a steady increase in volume. About a decade ago a surgery textbook observed that:

When large numbers of innovative treatments are being continuously introduced into clinical practice, rigorous testing is mandatory for the protection of individual patients and the just use of limited resources. This holds true with even greater force in light of the evidence that many innovations show no advantage over existing treatments when they are subjected to properly controlled study.3

The latter observation came painfully true for cardiac ventriculotomy and arthroscopic knee surgery.4,5

Concurrently, the push to innovate is greater than ever. Surgery professional meetings encourage the presentation of "new approaches" specifically selected by their program committees, and most surgery journals have a section especially reserved for innovations and new techniques. New is hot, old is not. Academic and private surgeons alike feel this unrelenting push to innovate, not only for the sake of their patients, but also to safeguard their status, their reputation and, ultimately, their income. Yesterday's procedures are old news, and surgeons may be reluctant to continue using old-fashioned techniques, regardless of the fact that these may have been proven safe and effective unlike their innovative counterparts. While ideally these "new and improved" procedures should have been scientifically tested for safety and efficacy, this is not often the case. Though most surgeons' awareness of the importance of evidence-based therapy is deepening, as a profession, surgeons fail to keep up with the growing demand for evidence-based surgery (EBS).6,7,8,9,10,11,12,13,14 The gap between surgeons' professional idealism and clinical reality is widening.

It needs to be stated up front that surgeons are not alone in using therapies that are essentially untried and unproven. Other physicians are at liberty to employ medical therapies at their discretion, may opt for off-label use of drugs, and can choose dosages, medication combinations, and strategies as they see fit. But unlike innovative surgeries, (conventional) drugs have generally been tested before entering the market and have been determined to be safe and effective (for specific conditions) in FDA-monitored trials and studies. Surgeons have, in essence, carte blanche to introduce untested new procedures. In the absence of any overseeing entity,15,16,17,18,19 it currently is the sole responsibility of the surgical profession to ensure the use of evidence-based techniques.20 Individual surgeons have brought this point to light, and have sought ways to stimulate others to innovate responsibly,19,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36 and the American College of Surgeons has made it its goal to improve the use of EBS.37

Regardless, it remains a formidable challenge for surgeons to embrace EBS. Within evidence-based therapy, there is a hierarchy of evidence according to strength.38,39 The best evidence is ranked Level 1: (meta-analysis of) prospective randomized clinical trials and Level 2: cohort studies or outcomes research. Evidence of lesser strength is ranked Level 3 (case-control studies), 4 (case series) or 5 (expert opinion and bench research). Modern-day surgical literature still consists mainly of reported Level 4 and 5 evidence, sometimes Level 3, and rarely Level 2 or 1.34 The reported declining engagement of surgeons in research adds to this troublesome situation.40 Moreover, as Rangler et al. reported recently, surgeons are less likely to receive federal funding for research, they often receive smaller awards when they do, fewer surgeons participate in the National Institutes of Health (NIH) reviewing process, and, most surprisingly, surgeons were in the minority in the Surgery Study Sections.41

Why is the occurrence of evidence-based therapy so limited in surgery? Several explanations that are more or less related have been given for this apparent discrepancy between surgery and the rest of medicine. First, the very definition of surgical innovation and, related to that, clinical surgical research, is vague. The boundaries between clinical care and clinical experimentation are not always clear in the surgical situation. It is difficult for surgeons to determine when routine modifications in operations become extensive enough to warrant formal study and a patient's informed consent for participation in research. This elusive definition of surgical research is the very core of this problem.34,35,36,37,38,39,40,41,42

Second, and related to this, is the notion of surgical exceptionalism.43 This is the view that the somewhat exceptional ethical or regulatory status of surgery and surgical research is justified by the exceptional differences between surgeries and pharmaceutical interventions.44,45,46 Comparing 2 pills appears more straightforward than comparing 2 surgical procedures. Whether this comes down to substantial differences between the 2 forms of therapy or is due, perhaps, to psychological differences between physicians and surgeons is not wholly understood.

Third, and indirectly flowing from the above notion; to produce Level 1 evidence, one has to conduct randomized controlled clinical trials (RCTs) or meta-analyze existing RCTs. Implementing RCTs can be problematic for the surgical situation, and incorporating sham surgery as a placebo into such trials even more so. Sham surgery is ethically contentious at best,47 and although it has proven effective in certain scenarios5 and its ethical permissibility has been defended effectively,48 it has not gained wide acceptance yet. It deserves attention that while the RCT is widely regarded as the gold standard of evidence-based therapy, it is not always necessary, feasible, or appropriate in the surgical situation.11,49,50,51,52

Finally, and perhaps the most worrisome reason why EBS has not taken off, is the alleged insufficiency of knowledge and research skills among surgeons.18,53 In his 2004 American College of Surgeons presidential address, surgeon R. Scott Jones paints a dismal picture of the current research climate in surgery.40 Most alarmingly, Jones stipulates that the mainstream surgery community actually devalues research, and is neither well-informed about nor respectful of the scientific method. Jones goes on to state that it is enormously difficult to incorporate research into a surgeon's professional life, not in the least because other demands are increasingly taking away time and energy from potential research efforts. Understandably, Jones urges his professional community to address this problem and dedicate more time and attention towards EBS and to the education of future surgeons about this matter. Indeed, it appears necessary and timely for surgeons to realize they are lagging behind in the quest for evidence-based therapy, and to increase their efforts to embrace, produce, and implement EBS into their professional lives and in every day clinical practice, for the protection of themselves, their patients, and the public interest.54

Read More

References

  1. Lombardo PA. The history of ethics in innovative surgery: a few stories, many questions. Commissioned paper for the Committee on Development of Ethical Guidelines for Innovative Surgery. March 2004. Charlottesville: University of Virginia Press (In press).

  2. McCullough LB. Standard of care, innovation and research in surgery: a problem in research ethics or professional medical ethics? Commissioned paper for the Committee on Development of Ethical Guidelines for Innovative Surgery. March 2004. Charlottesville: University of Virginia Press. (In press).

  3. Roy DL, Black P, McPeek B, "Ethical Principles in Research" In: Troidl H, Spitzer WO, McPeek B, Mulder DS, McKneally MF, Wechsler AS, et al, eds. Principles and Practice of Research: Strategies for Surgical Investigation. New York: Springer-Verlag, 1991:92.

    Google Scholar
  4. Starling RC, McCarthy PM, Buda T, et al. Results of partial left ventriculectomy for dilated cardiomyopathy: hemodynamic, clinical and echocardiographic observations. J Am Coll Cardiol. 2000;36(7):2098-2103.
    View Article PubMed Google Scholar
  5. Moseley JB, O'Malley K, Petersen NJ, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med. 2002;347(2):81-88.
    View Article PubMed Google Scholar
  6. Law S, Wong J. Use of controlled randomized trials to evaluate new technologies and new operative procedures in surgery. J Gastrointest Surg. 1998;2(6):494-495.
    View Article PubMed Google Scholar
  7. McLeod RS. Issues in surgical randomized controlled trials. World J Surg. 1999;23(12):1210-1214.
    View Article PubMed Google Scholar
  8. Carey TS. Randomized controlled trials in surgery: an essential component of scientific progress. Spine. 1999;24(23):2553-2555.
    PubMed Google Scholar
  9. Moss RL. The CONSORT Statement: progress in clinical research in pediatric surgery. J Ped Surg. 2001;36(12):1739-1742.
    View Article PubMed Google Scholar
  10. Maddem GJ. Evidence-based medicine in practice—surgery. Med J Aust. 2001;174(10):528-529.
    View Article PubMed Google Scholar
  11. Meakins JL. Innovation in surgery: the rules of evidence. Am J Surg. 2002;183(4):399-405.
    View Article PubMed Google Scholar
  12. Arya J, Wolford H, Harken AH. Evidence-based science. Arch Surg. 2002;137(11):1301-1303.
    View Article PubMed Google Scholar
  13. Cook RC, Alscher KT, Hsiang YN. A debate on the value and necessity of clinical trials in surgery. Am J Surg. 2003;185(4):305-310.
    View Article PubMed Google Scholar

  14. Jimenez RE, Guitierrez AR, Benitez IM. Methodological requirements for assessing surgical procedures in current medical literature. World J Surg. 2003;27:229-233.

    Google Scholar
  15. Spodick DH. Numerators without denominators. There is no FDA for the surgeon. JAMA. 1975;232(1):35-36.
    View Article PubMed Google Scholar

  16. Frader JE, Caniano DA. Research and innovation in surgery. In: McCullough LB, Jones JW, Brody BA, eds. Surgical Ethics. New York: Oxford University Press;1998:216-241.

    Google Scholar

  17. Siegler M. Ethical issues in innovative surgery: should we attempt a cadaveric hand transplantation in a human subject? Transplant Proc. 1998;30:2779-2782.

    PubMed Google Scholar

  18. Reitsma AM, Moreno JD. Ethical regulations for innovative surgery: the last frontier? J Am Coll Surg. 2002;194:792-801.

    View Article PubMed Google Scholar

  19. Strasberg SM, Ludbrook PA. Who oversees innovative practice? Is there a structure that meets the monitoring needs of new techniques? J Am Coll Surg. 2003;196:938-948.

    View Article PubMed Google Scholar

  20. Jones JW. The surgeon's autonomy: defining limits in therapeutic decision-making. Commissioned paper for the Committee on Development of Ethical Guidelines for Innovative Surgery. March 2004. Charlottesville: University of Virginia Press. (In press).

  21. Love JW. Drugs and operations. some important differences. JAMA. 1975;232(1):37-38.
    View Article PubMed Google Scholar
  22. Bunker JP, Hinkley D, McDermott WV. Surgical innovation and its evaluation. Science. 1978;200(4344):937-941.
    View Article PubMed Google Scholar
  23. Shinebourne EA. Ethics of innovative cardiac surgery. Br Heart J. 1984;52(6):597-601.
    PubMed Google Scholar

  24. Qualms about innovative surgery. Lancet. 1985;1:149.

  25. Gross M. Innovations in surgery. J Bone Joint Surg Br. 1993;75(3):351-354.
    View Article PubMed Google Scholar
  26. Ward CM. Surgical research, experimentation and innovation. Br J Plast Surg. 1994;47(2):90-94.
    PubMed Google Scholar
  27. Waring GO, III. A cautionary tale of innovation in refractive surgery. Arch Ophthalmol. 1999;117(8):1069-1073.
    View Article PubMed Google Scholar
  28. McKneally MF. Ethical problems in surgery: innovation leading to unforeseen complications. World J Surg. 1999;23(8):786-788.
    View Article PubMed Google Scholar
  29. Jones JW. Ethics of rapid surgical technological advancement. Ann Thorac Surg. 2000;69(3):676-677.
    PubMed Google Scholar

  30. Josefson D. University must tell patients that they were research "guinea pigs." BMJ. 2000;321:1487A.

    PubMed Google Scholar
  31. Moore FD. Ethical problems special to surgery: surgical teaching, surgical innovation, and the surgeon in managed care. Arch Surg. 2000;135(1):14-16.
    View Article PubMed Google Scholar

  32. Bunch WH, Dvonch VM. Moral decisions regarding innovation. The case method. Clin Orthop. 2000;378:44-49.

    PubMed Google Scholar
  33. Frader JE, Flanagan-Klygis E. Innovation and research in pediatric surgery. Semin Pediatr Surg. 2001;10(4):198-203.
    View Article PubMed Google Scholar
  34. Margo CE. When is surgery research? Towards an operational definition of human research. J Med Ethics. 2001;27(1):40-43.
    View Article PubMed Google Scholar
  35. Cronin DC, II, Millis JM, Siegler M. Transplantation of liver grafts from living donors into adults--too much, too soon. N Engl J Med. 2001;344(21):1633-1637.
    View Article PubMed Google Scholar
  36. Casler JD. Clinical use of new technologies without scientific studies. Arch Otolaryngol Head Neck Surg. 2003;129(6):674-677.
    View Article PubMed Google Scholar
  37. Jones RS, Richards K. Office of Evidence-Based Surgery Charts Course for Improved System of Care. Bull Amer Coll Surg. 2003;88(4):11-21.
    PubMed Google Scholar

  38. Centre for Evidence Based Medicine. Levels of evidence and grades of recommendation. Available at http://www.cebm.net/levels_of_evidence.asp. Accessed September 27, 2004.

  39. Guyatt G, Haynes RB, Jaeschke RZ, et al. User's guide to the medical literature XXV. Evidence-based medicine: principles for applying the user's guides for patient care. JAMA. 2000;284(10):1290-1296.
    View Article Google Scholar
  40. Jones RS. Requiem and renewal. Presidential address. Ann Surg. 2004;240(3):395-404.
    PubMed Google Scholar
  41. Rangel S, Efron B, Moss RL. Recent trends in National Institutes of Health funding of surgical research. Ann Surg. 2002;236(3):277-287.
    PubMed Google Scholar
  42. Reitsma AM, Moreno JD. Surgical research, an elusive entity. AJOB. 2003;3(4):49-50.
    View Article PubMed Google Scholar

  43. London AJ. Cutting surgical research at the joints: individuating and assessing surgical procedures. Commissioned paper for the Committee on Development of Ethical Guidelines for Innovative Surgery. March 2004. Charlottesville: University of Virginia Press. (In press).

  44. Agich GJ. Ethics and innovation in medicine. J Med Ethics. 2001;27(5):295-296.
    View Article PubMed Google Scholar
  45. McKneally MF. A bypass for the Institutional Review Board: reflections on the Cleveland Clinic study of the Batista operation. J Thorac Cardiovasc Surg. 2001;121(5):837-839.
    View Article PubMed Google Scholar

  46. McKneally MF. Surgical innovation. Panel presentation at American Society for Bioethics and Humanities. November 2003; Montreal, Canada.

  47. London AJ, Kadane JB. Placebos that harm: sham surgery controls in clinical trials. Stat Methods Med Res. 2002;11(5):413-427.
    View Article PubMed Google Scholar
  48. Miller FG. Sham surgery: an ethical analysis. AJOB. 2003;3(4):41-48.
    View Article PubMed Google Scholar
  49. McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D. Randomized controlled trials in surgery: problems and possible solutions. BMJ. 2002;324(7351):1448-1451.
    Google Scholar
  50. Reeves B. Health-technology assessment in surgery. Lancet. 1999;353(Suppl 1):S1 3-5.
    View Article PubMed Google Scholar
  51. Baum M. Reflections on randomized controlled trials in surgery. Lancet. 1999;353(Suppl 1):SI6-8.
    View Article PubMed Google Scholar
  52. Lilford RJ, Braunholz D, Harris J, Gill T. Trials in surgery. Br J Surg. 2004;91(1):6-16.
    View Article PubMed Google Scholar
  53. Rutan RL, Deitch EA, Waymack JP. Academic surgeons' knowledge of food and drug administration regulations for clinical trials. Arch Surg. 1997;132(1):94-98.
    View Article PubMed Google Scholar

  54. Reitsma AM, Moreno JD, Bosk CL, et al. Ethical guidelines for innovative surgery: recommendations for national policy. Position statement. Submitted for publication. 2004. J Am Coll of Surg.

Citation

Virtual Mentor. 2004;6(10):465-469.

DOI

10.1001/virtualmentor.2004.6.10.oped1-0410.

The viewpoints expressed in this article are those of the author(s) and do not necessarily reflect the views and policies of the AMA.