Introduction
The evidence-based process for development of the recommendations in the first edition of the European guidelines was established at the outset of the project in 2006 by an editorial board with extensive experience in development of best practice guidelines, in evaluation of strategies for colorectal cancer (CRC) screening and in programme management. In 2007 the editorial board drafted an initial comprehensive outline of the Guidelines and recruited a multidisciplinary group of experts in colorectal cancer screening and diagnosis across the European Union to collaborate in revising the outline and drafting the chapters, including guiding principles and recommendations. Additional scientific support was provided by a Literature Group consisting of epidemiologists with special expertise in the field of CRC and in performing systematic literature reviews.
The expert Literature Group provided technical and scientific support to the authors and editors in searching the relevant literature, assessing the methodological quality of retrieved studies, defining a grading system of the level of evidence and strength of the recommendations, and preparing evidence tables and summary documents for over 500 references identified through systematic reviews of the literature according to the priorities and procedures agreed with the editorial board and the authors.
The Literature Group was coordinated by N. Segnan at the Unit of Cancer Epidemiology, Department of Oncology of the Piedmont Centre for Cancer Prevention (CPO Piemonte) and S. Giovanni University Hospital, Turin, Italy, and was lead by S. Minozzi at the same institution. Other members of the Literature Group were based at the CPO in Turin and at the Oxford University Cancer Screening Research Unit, Cancer Epidemiology Unit, Oxford, United Kingdom. Additional scientific and technical support was provided by the International Agency for Research on Cancer, Quality Assurance Group, Section of Early Detection and Prevention, Lyon, France.
Definition of clinical questions
In multidisciplinary workshops conducted in 2007 and 2008 the chapter authors met with the editorial board and the Literature Group. At these meetings, the table of contents of the Guidelines was repeatedly revised and the methodology of evidence-based guideline development, including the process of identifying and evaluating the relevant evidence for each chapter based on the topics in the revised outline was agreed with the authors. Subgroups of authors responsible for each chapter also worked individually with members of the Literature Group to develop clinically relevant questions based on the revised chapter outlines, and the results for each chapter were subsequently discussed with the entire group of authors and editors and the Literature Group in plenary workshop sessions in order to ensure a common methodological approach and to reach a consensus on questions of key importance requiring the support of the Literature Group in order to identify and assess the relevant evidence. This collaborative, multidisciplinary approach remained a guiding principle throughout the entire process up to completion of drafting and editing of the Guideline chapters.
The clinical questions initially formulated by the authors of each chapter and subsequently agreed with the editorial board and the other authors were developed according to the PICOS method [4]
[8]
[9] modified slightly to take into account the aim of screening to lower the burden of the disease in the population:
P: patients/population characteristics
I: experimental intervention on which the question is focused
C: comparison intervention/control/reference group
O: outcome measure relevant for the clinical question
S: study design on which to base the evidence search
The extensive list of initial clinical questions was reduced to a feasible number, by prioritising questions of key importance for each chapter. In total, 113 clinical questions were prioritised. The PICOS components of each prioritised question were subsequently used by the Literature Group to define specific key words that were then employed in comprehensive bibliographic searches. The results of these activities were reported back to the authors and editors in subsequent workshops and electronically. This enabled the editors and authors to provide continuous professional and scientific support to the process of identifying and analysing the relevant evidence.
Bibliographic review
The Literature Group performed bibliographic searches on Medline, and in many cases also on Embase and The Cochrane Library using MeSH terms and free text words. Most searches were limited to the years 2000 to 2008 or were conducted without date restrictions if the authors or editors who were experts in the field knew that there were relevant articles published before 2000. Published articles suggested by the authors and not retrieved by a systematic search were also considered. Only scientific publications in English, Italian, French and Spanish were included. Priority was given to recently published, systematic reviews or clinical guidelines. If systematic reviews of high methodological quality were retrieved, the search for primary studies was limited to those published after the last search date of the most recently published systematic review (i. e. if the systematic review had searched primary studies until February 2006, primary studies published after February 2006 were sought). If no systematic reviews were found, a search for primary studies published since 2000 was performed.
In selected cases references not identified by the above process were included in the evidence base, i. e. when authors of the chapters found relevant articles published after 2008 during the period when chapter manuscripts were drafted and revised prior to publication. The criteria for relevance were: articles concerning new and emerging technologies where research is growing rapidly, high quality and updated systematic reviews, and large trials that make a significant contribution to the robustness of the results or allow upgrading of the level of evidence.
Inclusion criteria
The inclusion criteria applied by the Literature Group were based on the highest level of available evidence, taking into account study design. For primary studies, for each kind of question (e. g., effectiveness, diagnostic accuracy, acceptability and compliance) a hierarchy of the study designs and inclusion/exclusion criteria was developed by the epidemiologists in the Literature Group. For example, for effectiveness studies randomised controlled trials (RCT) were initially searched for. If RCTs were retrieved, no other types of study design were considered. If no, or only a few and/or small RCTs were retrieved, quasi-experimental studies were considered. If no quasi-experimental studies were found, prospective or retrospective cohort and case-control studies were considered. If studies with none of the above designs were retrieved, cross-sectional studies and case series were included. For diagnostic accuracy questions, cross-sectional studies with verification by reference standard were considered as the best source of evidence.
Quality assessment
The methodological quality of the publications retrieved by the Literature Group was assessed using the following criteria obtained from published and validated check lists.
Systematic reviews – quorum checklist
A validated checklist for evaluating the manner in which systematic reviews have been conducted was not available when the methods for the present EU Guidelines were established. Therefore the QUOROM checklist that assesses the quality of reporting was used as a proxy to assess the quality of conduct of systematic reviews. This approach reflects the view that the quality of reporting can be used as a criterion for the quality of the process of preparing a systematic review [7].
Randomised Controlled Trials
Randomised controlled trials were assessed using the following criteria suggested in the Cochrane Handbook [5] and by the Cochrane Effective Practice and Organisation of Care Review Group [2]:
-
Unit of allocation (i. e. who or what was allocated to study groups: individuals or clusters);
-
Unit of analysis (i. e. results analysed as events at the level of individuals or clusters);
-
If unit of allocation and unit of analysis differ, was cluster analysis performed?
-
Protection against selection bias (adequate sequence generation and allocation concealment);
-
Protection against performance bias (blinding of providers);
-
Protection against contamination (blinding of participants);
-
Protection against attrition bias (intention to treat analysis, few lost at follow up balanced between groups); and
-
Protection against detection bias (blinding of participants and outcome assessors).
Observational studies: cohort studies and case control studies
Observational studies were evaluated using the following criteria of the Newcastle-Ottawa Scale (for recent overview see: [14]
-
Case control studies:
-
Adequate definition of the cases;
-
Representativeness of the cases;
-
Selection source of controls;
-
Definition of controls;
-
Comparability of cases and controls on the basis of the design or analysis;
-
Method of exposure assessment;
-
Same method of ascertainment for cases and controls;
-
Non-Response rate.
-
Cohort studies:
-
Representativeness of the exposed cohort;
-
Selection source of the non-exposed cohort;
-
Method of exposure assessment;
-
Demonstration that outcome of interest was not present at start of study;
-
Comparability of cohorts on the basis of the design or analysis;
-
Method outcome assessment;
-
Adequacy of follow up of cohorts.
Interrupted time series studies
Studies based on interrupted time series were assessed using the following criteria suggested by the Cochrane Effective Practice and Organisation of Care Review Group (EPOC 2002):
-
Clearly defined point in time when the intervention occurred.
-
A: Intervention occurred at a clearly defined point in time;
-
B: NOT CLEAR because not reported in the paper;
-
C: Intervention did not occur at a clearly defined point in time.
-
At least three data points before and three after the intervention.
-
A: Three or more data points before and three or more data points recorded after the intervention;
-
B: NOT CLEAR because not reported in the paper;
-
C: Less than three data points recorded before, and less than three data points recorded after intervention.
-
Protection against secular changes (the intervention is independent of other changes).
-
A: Intervention occurred independently of other changes over time;
-
B: NOT CLEAR because not reported in the paper;
-
C: Intervention was not independent of other changes over time.
-
Protection against detection bias (intervention unlikely to affect data collection).
-
A: Intervention unlikely to affect data collection (for example, sources and methods of data collection were the same before and after the intervention);
-
B: NOT CLEAR because not reported in the paper;
-
C: Intervention likely to affect data collection (for example, any change in source or method of data collection before vs. after the intervention).
-
Blinded assessment of primary outcome(s).
-
A: Explicit statement of authors that the primary outcome variables were assessed blindly OR the outcome variables are objective e. g. length of hospital stay, drug levels as assessed by a standardised test;
-
B: NOT CLEAR if not specified;
-
C: Outcomes were not assessed blindly.
-
Completeness of data set.
-
A: Data set covers 80 – 100 % of total number of participants or episodes of care in the study;
-
B: NOT CLEAR if not specified;
-
C: Data set covers less than 80 % of the total number of participants or episodes of care in the study.
Diagnostic accuracy studies
The criteria used to evaluate diagnostic accuracy studies were obtained from the QUADAS checklist [15]:
-
Study design: diagnostic cross-sectional studies with prospective or retrospective recruitment; case control;
-
Spectrum of patients representative of the individuals who will receive the test in practice;
-
Patients selection criteria clearly described;
-
Verification by reference standard of all or a randomised sample of subjects (absence of verification bias);
-
Execution of the index and comparator tests adequately described;
-
Execution of the reference standard adequately described;
-
Independent and blind interpretation of index test and reference standard results;
-
Un-interpretable/intermediate test results reported;
-
Withdrawals from the study explained.
Clinical guidelines
The quality of clinical guidelines evaluated by the Literature Group was assessed using the following most relevant criteria derived from the COGS checklist [11]:
-
Description of the clinical specialisation of the members of the panel of guideline authors;
-
Search strategy described (databases, years covered, any language restriction);
-
Inclusion criteria of primary studies stated;
-
Method used to analyse and synthesise the evidence and to reach the consensus among the panellists to elaborate the recommendation described;
-
Presence of a grading of level of evidence and/or of the strength of the recommendation; and
-
Presence of a complete reference list.
Evidence tables and summary documents
The Literature Group prepared the following documents based on the publications retrieved for each clinical question or group of clinical questions. The documents were subsequently used by the authors in drafting respective chapters:
-
An evidence table for each retrieved study with the main characteristics of the study (study design, objective of the study, comparisons, participant’s characteristics, outcome measures, results, methodological quality, level of evidence);
-
A summary document with a synthesis of the number, types and characteristics of the retrieved studies, their overall methodological quality, a description of the main methodological flaws, the study results and the conclusions and the overall level of evidence.
The evidence tables and summary documents for each chapter are documented in [6]. Evidence tables were not prepared for: additional publications cited in the background sections of the chapters; pathological and clinical classifications; technical instructions; narrative reviews; editorials and personal communications; and articles published before 2000 and cited by the authors after the systematic search of the literature.
Some articles published between 2000 and 2008 and not retrieved by the systematic search were considered to be relevant by the authors. Those references have therefore been included in the body of evidence in agreement with the editorial board. For these articles, additional evidence tables were prepared after December 2009, but the respective results were not included in the respective summary documents.
Grading system
The key recommendations presented in each chapter of the Guidelines are listed at the front of the respective chapter together with a grading of the evidence on which each recommendation is based, and the strength of the recommendation. Only the highest level of evidence supporting a recommendation is reported. The following grading scales are used:
Level of the evidence
-
I: multiple randomised controlled trials (RCTs) of reasonable sample size, or systematic reviews (SRs) of RCTs
-
II: one RCT of reasonable sample size, or 3 or less RCTs with small sample size
-
III: prospective or retrospective cohort studies or SRs of cohort studies; diagnostic cross section-al accuracy studies
-
IV: retrospective case-control studies or SRs of case-control studies, time-series analyses
-
V: case series; before/after studies without control group, cross sectional surveys
-
VI: expert opinion
Strength of the recommendations
The strength of recommendations was graded according to the following scale:
-
A: intervention strongly recommended for all patients or targeted individuals
-
B: intervention recommended
-
C: intervention to be considered but with uncertainty about its impact
-
D: intervention not recommended
-
E: intervention strongly not recommended
The strength of each key recommendation was determined by the authors of each chapter in agreement with the Guidelines editorial board.
Following the list of key recommendations at the beginning of each chapter, the rationale and the evidence on which the recommendations are based is summarised in the body of the chapter, including the respective levels of evidence.
In a number of chapters, in addition to the key recommendations, fundamental statements (Guiding Principles) defining the aims and scope of the recommendations presented in the chapter are provided at the front of the text. Most of the Guiding Principles are considered to be self-evident. All reflect the consensus of the authors and editors on essential principles of best practice in screening and diagnosis of colorectal cancer. In addition to these principles, additional advisory statements are made in the body of the chapters that are not specifically graded. These statements also represent the consensus of the authors and editors on best practice.
Correspondence between level of evidence and strength of recommendation
This present grading of the strength of recommendations did not require a rigid correspondence with the levels of evidence. For example grade A was given to interventions for which there was evidence level I (multiple RCTs or SR of RCTs) but also to interventions that could not be assessed by RCTs, (e. g. psychological aspects, the importance of an accurate information to the patients, etc). Grade B was given to interventions with lower evidence level (II or III) but also for interventions with evidence level I but with uncertainty about their impact in the population or about practical implementation (e. g. lack of resources for implementation, social barriers, supposed lack of acceptability by the target population). Grade C level was given to interventions for which evidence was not available or was of low grade (i. e. IV, V) or that may not have been considered of high importance for other reasons (i. e. psychological or social aspects). Grades D and E were assigned to interventions for which there was evidence of no benefit for participants, or for which the harm outweighed the benefits. [Table 1]
Table 1
Correspondence between level of evidence and strength of recommendations
|
Strength of recommendation
|
|
Levels of evidence
|
|
A
|
B
|
C
|
D
|
E
|
|
I
|
C
|
C
|
|
C
|
C
|
|
II
|
Nc
|
C
|
|
C
|
Nc
|
|
III
|
Nc
|
C
|
C
|
C
|
Nc
|
|
IV
|
Nc
|
Nc
|
C
|
Nc
|
Nc
|
|
V
|
Nc
|
Nc
|
C
|
Nc
|
Nc
|
|
VI
|
Nc
|
Nc
|
C
|
Nc
|
Nc
|
C: Coherence between the level of evidence and the strength of recommendations
Nc: No coherence between the level of evidence and the strength of recommendations
Method of obtaining consensus between the chapter authors and editors and the internal peer review
Each subgroup of authors responsible for a chapter received all the evidence tables and summary documents relating to the respective clinical questions. The authors drafted each chapter by describing the relevant issues, summarising the evidence, and including recommendations and conclusions. The authors also proposed a grading for the strength of the evidence and the strength of the respective recommendations, based on the results of the literature search and on their clinical experience, as well as any additional pertinent scientific literature that was taken into account with agreement from the editorial board. The draft chapters and the proposed strength of each recommendation were discussed with the editorial board and the authors of all chapters to reach consensus.
External peer review
Chapter drafts were subsequently sent to international experts in their respective fields for external peer review. They were also made available for web consultation with restricted access by experts involved in screening programmes. Comments and criticisms were considered and a final version of the chapters was elaborated. Preliminary and nearly final versions of the Guidelines chapters were prepared and discussed at pan-European network meetings of screening experts, clinicians, advocates, healthcare planners and regulators from all of the EU member states and two EU applicant countries in 2008 and 2009.
Final editing
During 2010, final changes resulting from the network discussion in November 2009 were taken into account by the authors of respective chapters. The consistency of the recommendations between the individual chapters was reviewed by the editorial board and corrections were made where necessary.
