Clinicians trying to make decisions based on the current best evidence are faced with an overwhelming obstacle, Too Much Information (TMI)! With over 25,000 new randomized controlled trials published each year how can a provider critically appraise these studies, determine their internal and external validity, put these results into context with what is already known, and then make an informed decision? Some of these data are biased and have uncertain value in patient care. Consequently, high quality critically appraised systematic reviews provide the best opportunity for evidence-based healthcare.
In 2008 an Institute of Medicine (IOM) report Knowing What Works in Healthcare was presented to the U. S. Secretary of Health. As a result, Congress under the auspices of theMedicare Improvement for Patients and Providers Act of 2008 contracted with the IOM and the Agency for Healthcare Research and Quality (AHRQ) to determine the best methods for developing clinical practice guidelines and to help guide organizations developing guidelines to improve objectivity, scientifically validity, and consistency. This report, entitled Clinical Practice Guidelines We Can Trust, was published in 2011. The committee who wrote the report also developed a new definition of a CPG as follows:
Clinical practice guidelines are statements that include recommendations intended to optimize patient care that are informed by a systematic review of the evidence and an assessment of the benefits and harms of alternative care options.
The committee proposed eight standards that should be assessed when evaluating whether a CPG is trustworthy. A shortened and edited version of these standards is listed below.
Standards for Developing Trustworthy Clinical Practice Guidelines
Transparency
- The processes by which a CPG is developed and funded should be detailed explicitly and publicly accessible.
Conflicts of Interest (COI)
- Prior to selection of the guideline development group (GDG), individuals being considered for membership should declare all interests and activities potentially resulting in COI with development group activity, by written disclosure to those convening the GDG.
- Whenever possible GDG members should not have COI.
- In some circumstances, a GDG may not be able to perform its work without members who have COIs.
- The chair or co-chair should not have a COI.
Group Composition
- The GDG should be multidisciplinary and balanced, comprising a variety of methodological experts and clinicians, as well as patient populations expected to be affected by the CPG.
Clinical Practice Guideline-Systematic Review Intersection
- Clinical practice guideline developers should use systematic reviews that meet standards set by the Institute of Medicine’s Committee on Standards for Systematic Reviews of Comparative Effectiveness Research.
Evidence Foundations and Rating the Strength of Recommendations
For each recommendation, an explanation of the reasoning underlying the recommendation, including
- A clear description of potential benefits and harms;
- A summary of relevant available evidence, description of the quality, quantity, and consistency of the aggregate available evidence
- A rating of the level of confidence in the evidence underpinning the recommendation
- A rating of the strength of the recommendation
- A description and explanation of any differences of opinion regarding the recommendation
Articulation of Recommendations
- Recommendations should be articulated in a standardized form detailing precisely what the recommendation is, and under what circumstances it should be applied.
- Strong recommendations should be worded so that compliance with the recommendation(s) can be evaluated.
External Review
- External reviewers should comprise a full spectrum of relevant stakeholders, including scientific and clinical experts, organizations, patients, and representatives of the public.
Updates
- The CPG publication date, date of pertinent systematic evidence review, and proposed date for future CPG review should be documented in the CPG.
- Literature should be monitored regularly following CPG publication to identify the emergence of new, potentially relevant evidence and evaluate the continued validity of the CPG.
- CPGs should be updated when new evidence suggests the need for modification of clinically important recommendations.
Many of the current CPGs used by clinicians in the United States do not meet these standards. It will take time for guideline developers to adopt these criteria and some may never adhere to all eight standards. If a CPG does not fully adhere to these recommendations, is the CPG invalid? Not necessarily. However, if the GDG did not perform a systematic review of the literature (Standard 4) or provide foundations for rating the strength of the evidence (Standard 5), you should be wary. The current ACCF/AHA guidelines and the VA/DoD CPGs perform well using this new quality rating scheme. However, we should carefully evaluate upcoming guidelines such as the American Diabetes Association (ADA) Diabetes Guidelines, the Joint National Committee (JNC VIII) Hypertension Guidelines, and the Adult Treatment Panel IV Lipid Guidelines before automatically adopting their recommendations. Will they be trustworthy? And finally, remember, some of the highest quality CPGs are published by organizations from other countries. Evidence-based medicine did not originate in the United States!
[collapsed]
Conflicts of Interest & Guidelines
I think conflicts of interest undermine the quality of guidelines to a similar, if not greater degree, than lack of evidence grading or systematic review. The IOM recommends against financial conflicts of interest. Yet, for example the American Cancer Society, in their new guideline development process, does not heed this advice — See: JAMA 2011 (Dec 14);306(22):2495-2499.
There is also the issue of intellectual conflict of interest, something a colleague and I discuss in an article to be published in the International Journal of Person Centered Medicine. The intellectual COIs consist of 1) a focus on content over process by including content experts and not methodologists on guideline committees; 2) the issue of confirmation bias among experts (or any of us, for that matter); and 3) the inability to acknowledge controversies.
Skeptical about guidelines
Serving on the Chest antithrombotic consensus conference for 10 years substantially increased my skepticism. Consider:
Even some of the authors of the guidelines on a particular condition do not follow the guidelines that they write – or even agree with them.
Finding clinical experts with no COI is practically impossible in some fields and one is at risk of identifying those who have COI but report that they have none.
Focusing too much on methods and a list of study design characteristics can result in subtle but fatal flaws going over-looked such that the results of an otherwise well-designed trial reaches a seriously flawed conclusion.
Some guidelines are written to “raise the floor” rather than provide optimal care. For example, in the 2004 Chest guidelines, the advantages of warfarin vs ASA post-MI are discussed but ASA is recommended for most settings in the U.S. because, in the author’s view, “meticulous INR control” as reported in the trials is not readily achievable in most sites. (In one of the key studies – ASPECT 2 – the INR time in range was only 50%).
Maybe we should treat guidelines like court decisions and have another group write a dissenting opinion and explain their points of dissention.
Too many competing interest
In my opinion, every specialty in medicine has there are own guideline or attempting to develop one. This has diluted the whole idea of guidelines, particularly in certain disease states such as heart disease, diabetes and so on. It is often annoying when you have specialists debating about their speciaty guidelines rather than providing optimal patient care.
I support the idea of guidelines but do we need so many for the same disease state.