09 - Strengths and Limitations of evidence
The strengths and limitations of the body of evidence are clearly described.
User’s Manual Description:
Statements highlighting the strengths and limitations of the evidence should be provided. This ought to include explicit descriptions - using informal or formal tools/methods - to assess and describe the risk of bias for individual studies and/or for specific outcomes and/or explicit commentary of the body of evidence aggregated across all studies. This may be presented in different ways, for example: using tables commenting on different quality domains; the application of a formal instrument or strategy (e.g., Jadad scale, GRADE method); or descriptions in the text.
Where to Look:
Examine the paragraphs/chapters describing the guideline development process for information on how the methodological quality of the studies (e.g., risk of bias) were described. Evidence tables are often used to summarize quality features. Some guidelines make a clear distinction between description and interpretation of evidence, for instance, in a results section and a discussion section, respectively.
How to Rate:
Item content includes the following CRITERIA:
- descriptions of how the body of evidence was evaluated for bias and how it was interpreted by members of the guideline development group
- aspects upon which to frame descriptions include:
- study design(s) included in body of evidence
- study methodology limitations (sampling, blinding, allocation concealment, analytical methods)
- appropriateness/relevance of primary and secondary outcomes considered
- consistency of results across studies
- direction of results across studies
- magnitude of benefit versus magnitude of harm
- applicability to practice context
Additional CONSIDERATIONS:
- Is the item well written? Are the descriptions clear and concise?
- Is the item content easy to find in the guideline?
- Are the descriptions appropriate, neutral, and unbiased? Are the descriptions complete?
