ACCF/AHA clinical practice guideline methodology summit report: A report of the American college of cardiology foundation/American heart association task force on practice guidelines

TY - JOUR

T1 - ACCF/AHA clinical practice guideline methodology summit report

T2 - A report of the American college of cardiology foundation/American heart association task force on practice guidelines

AU - Jacobs, Alice K.

AU - Kushner, Frederick G.

AU - Ettinger, Steven M.

AU - Guyton, Robert A.

AU - Anderson, Jeffrey L.

AU - Ohman, E. Magnus

AU - Albert, Nancy M.

AU - Antman, Elliott M.

AU - Arnett, Donna K.

AU - Bertolet, Marnie

AU - Bhatt, Deepak L.

AU - Brindis, Ralph G.

AU - Creager, Mark A.

AU - Demets, David L.

AU - Dickersin, Kay

AU - Fonarow, Gregg C.

AU - Gibbons, Raymond J.

AU - Halperin, Jonathan L.

AU - Hochman, Judith S.

AU - Koster, Marguerite A.

AU - Normand, Sharon Lise T.

AU - Ortiz, Eduardo

AU - Peterson, Eric D.

AU - Roach, William H.

AU - Sacco, Ralph L.

AU - Smith, Sidney C.

AU - Stevenson, William G.

AU - Tomaselli, Gordon F.

AU - Yancy, Clyde W.

AU - Zoghbi, William A.

AU - Harold, John G.

AU - He, Yulei

AU - Mangu, Pamela B.

AU - Qaseem, Amir

AU - Sayre, Michael R.

AU - Somerfield, Mark R.

N1 - Funding Information: 1 How can the Task Force and GWC identify the appropriate “patient representative”? What are the criteria to be met for selection of “patient representatives” who serve? Will they be volunteers or contracted for their work? As defined by the Cochrane Consumer Network, consumers (“patient representatives” herein) include all users or receivers of health care, including patients, members of the public, caregivers, family members, and members of consumer advocacy groups. Several organizations have identified or are in the process of identifying individuals to serve on GWC. The 4 largest of these are the Cochrane Collaboration through Cochrane Consumer Network and a U.S. subsidiary, Consumers United for Evidence Based Medicine, Agency for Healthcare Research and Quality (AHRQ), Consumers Union, and Guidelines International Network. Consumers Union is currently developing a list of “qualified” patient representatives to serve on evidence-based medical guideline committees under the auspices of a grant from the American Institutes for Research/AHRQ. They solicited interested parties from their subscriber base of 300,000 and received 2,000 expressions of interest. The AHA has a long history of inviting lay volunteers to serve on its boards and committees, in mission and fundraising activities, and in advocacy, and regularly recommends consumer representatives to private and governmental panels and committees. The American Society of Clinical Oncology has included consumers on their GWCs. Patient representatives with defined constituencies can be particularly valuable because of their institutional involvement. On the basis of the comments from Consumers Union and the American Society of Clinical Oncology and discussions among its members, Workgroup 1 recommends the following principles concerning the identification of appropriate patient representative candidates: A A job description, including desired and necessary attributes and expectations, should be available to the Task Force and GWC chairs and to organizations that may recommend candidates. B The Task Force and GWC should seek nominations for patient representatives from trusted organizations with knowledge of the individual, such as the AHA, ACCF, Consumers Union, Cochrane Collaboration, and AHRQ. C Patient representative candidates and/or their sponsoring organizations should submit their resumes, curriculum vitae, or personal statements. D Patient representatives should be advised of the time commitment and compensated for travel expenses identical to physician and other Task Force and GWC members. E Patient representative candidates, similar to all members, should complete the relationships with industry and other entities (RWI) disclosure. F The Task Force and/or GWC chair or designee should interview potential consumer candidates and identify other conflicts where they exist. G The development of a questionnaire similar to those used by major charitable organizations to elicit possible bias should be considered. H Prospective members should be required to confirm that they will give precedence to ultimate patient health, well-being, quality care, and value in their input, deliberation, and voting. I Patient representatives, as is true for other members of the GWC, must be sensitive to their role as impartial members and not permit financial and nonfinancial conflicts, including personal, intellectual, or organizational relationships to influence their judgment. J Patient representatives on GWCs should be engaged in the formulation of key clinical questions, topic selection, patient choices, values, preferences, and issues surrounding quality of life. They should be encouraged to provide input on the selection of diagnostic tests and treatment modalities. K Patient representatives should be acknowledged as members of the GWC at publication, and a complete summary of their RWI should be provided. They should be subject to the same RWI and recusal rules as other members. 2 Specifically, how is the patient representative to be integrated into the CPG process and/or peer review and/or external stakeholder review? What expectations are there for patient representative involvement? As a member of the Task Force, patient representatives would assist in directing and overseeing CPG development and establishing policy with a focus on issues such as shared decision making, patient preference, value, translation, and implementation. Patient representatives may be best suited to participate in the initial work of the GWC, during the formulation of key clinical questions, topic selection, and outline development, and, particularly, to comment on the translation and communication of CPGs to the general public. They may not be as helpful or comfortable with SR or assignment of COR and LOE to specific recommendations. Patient representatives with certain technical skills such as science journalists and writers may aid in a patient-provider communication and summary section. Patient representatives should be expected to contribute key questions to be answered by the GWC. They should provide input about patient choices, values, preferences, and issues surrounding quality of life in selecting diagnostic modalities, therapies, medications, and follow-up. 3 What form of training will be required for patient representative participants? To integrate patient representatives effectively into the CPG creation process, substantial preparatory training is needed before they begin their service on the Task Force or GWC. One study observed that without preparatory training, patient representatives feel that they are “participating observers of technical language to which they could hardly offer input.” Patient representatives contributed infrequently to the discussions, had difficulty with the technical language, only contributed during discussions of patient education, and in general felt that their contributions were not subsequently acted on ( 5 ). Involvement of patient representatives requires their understanding of the evidence. Difficulty with medical terminology or other jargon is an important barrier to active or meaningful involvement. Well-informed and experienced patient representatives are more likely to have meaningful exchanges with the GWC than those less informed or less familiar with medical terminology ( 6 ). The capacity for active participation in GWCs presupposes foundations of access, knowledge, information, understanding, confidence, agency, engagement, and advocacy ( 7 ). Although resource intensive, it is feasible to train patient representatives to understand the technical elements of CPG development. It is not expected that these members of GWCs understand most of the science related to specific medical issues. They may also have limitations in their ability to understand the details of SR or health economics. Nonetheless, focused instruction in the CPG development process will allow them to fulfill their role. Training is required for understanding elements of recommendation classifications and LOE, including treatment risks and benefits, comparative efficacy, biostatistics, and clinical trial design (i.e., the value of a single center case report or retrospective observational data versus prospective blinded RCT along with meta-analysis). The Guideline International Network Patient and Public Involvement Working Group has been created to support the development, implementation, and evaluation of guideline-oriented patient and public involvement programs. They have found that patient representative training should cover the fundamentals of CPG development and approaches for reporting back to patient constituencies. Their participants concluded that training and support may facilitate understanding of the technical aspects of CPG development, address financial and organizational barriers to participation, and enhance mutual understanding. Guideline International Network Patient and Public Involvement Working Group collaboration priorities include the development of recruitment methods, training and support strategies, information material and tools, and glossaries of technical terms used in CPGs ( 8 ). The U.S. Cochrane Center and Consumers United for Evidence-Based Medicine have created a web-based course, “Understanding Evidence-based Healthcare: A Foundation for Action,” through a grant from AHRQ ( http://us.cochrane.org/understanding-evidence-based-healthcare-foundation-action ) ( 9 ). This course is divided into 6 modules: INTRO (what is evidence-based health care and why is it important), ASK (importance of research questions in evidence-based health care), ALIGN (research design, bias, and LOE), ACQUIRE (assessing harms and benefits), APPRAISE (understanding healthcare statistics), and APPLY (critical appraisal and making better decisions for evidence-based care; determining causality) ( 9 ). The Cochrane Collaboration also has created the Cochrane Consumer Network to engage patient representatives in the development of SR, raise awareness among patient representatives, that is, serve as a clearinghouse of patient representatives for advisory groups, commission plain language summaries, and recruit coauthors for reviews. New ideas to promote patient representative involvement, such as videos, workshops, learning materials, evaluations, and use of social networks have been recently implemented. 4 Should the Task Force support the standard to have CPGs publicly reviewed? How will that affect the final product? How much of a burden will that put on volunteers? The IOM has recommended that public agencies, patients, and representatives of the public should be external reviewers. The IOM has also recommended that before publication, a draft of the CPG should be made available to the general public for comment and reasonable notice of impending publication should be provided to interested public stakeholders. The IOM believes that for transparency, fairness, completeness, and credibility, these recommendations are reasonable. Currently, while the ACCF/AHA CPGs undergo an extensive peer review process that includes scientific and clinical content experts in addition to partnering, collaborating, and other relevant professional societies, they are not open to public review and comment. Fundamental to the CPG development process is the ability to develop the CPG without bias from commercial interests. The Workgroup carefully considered the value provided by opening the draft CPG to public opinion. It was recognized that a period of open public comment would introduce a window for potential interference in the process by industry and other external stakeholders that cannot be adequately controlled or managed. However, review of the CPG by public representatives and stakeholders such as governmental agencies, for example, the AHRQ and the U.S. Food and Drug Administration (FDA), as well as scientific and professional experts, is reasonable and recommended. 5 How will conflicts of interest (organizational, intellectual, and practice based) be adjudicated by the GWC and Task Force? What is an “intellectual conflict” or “intellectual bias”? The ACCF/AHA CPGs are in substantial compliance with 7 of the 8 standards for developing trustworthy CPGs proposed by the IOM, ( Workgroup 1 Comparison Table ); the CPG-SR intersection is the subject of the remainder of this report. However, although the ACCF and AHA have a rigorous policy for defining, disclosing, and managing RWI ( 4 ), the policy is currently undergoing evaluation and updating for potential conflicts related to intellectual bias and practice (or employment) bias. Intellectual or clinical practice bias (the latter term is operative when an individual‘s income is enhanced by performing a certain test or procedure relevant to the guideline topic) may result in a conflict of interest. An intellectual or clinical practice bias is a predisposition of an individual based on a reaction to a past or current event, treatment, relationship, or other circumstance, or an opinion, belief, or position so strongly held that it might prevent the individual from exercising objective judgment about a matter relevant to the work of the GWC. In the infrequent scenario where, in the opinion of the Task Force or GWC chair, a true bias (with its negative connotation) exists, that individual will not be invited to participate as a member of the GWC. In contrast, and similar to the ACCF/AHA choice of the term RWI rather than COI related to industry, the ACCF and AHA have chosen the terms intellectual perspective and clinical practice perspective to denote intellectual and practice-related opinions and expertise based on evidence and/or experience. Both science and patient representative GWC members with intellectual perspective and clinical practice perspective must be identified and included and their perspective disclosed and managed. It is understood that they have an open-minded approach to evidence and opinion that distinguishes them from individuals with actual intellectual or practice bias . Identification of Potential Intellectual or Clinical Practice Perspectives of Candidates for Appointment to a GWC Unlike potential conflicts arising from RWI, determination of the existence of an intellectual or clinical practice perspective may be subjective and less readily apparent to the Task Force. The key to mitigate effectively any conflict is early identification and management of the conflict by the Task Force initially and then by the GWC chair once the work begins. The Task Force chair should be responsible for implementing applicable intellectual and clinical practice perspective procedures. Following are ways to identify and adjudicate the perspectives: 1 Nominations to GWCs should be sought from trusted organizations with a deep knowledge of the individuals nominated. For example, both the ACCF and the AHA, as well as many of their collaborating organizations, have a thorough understanding of CPG development, and each has a long relationship with numerous consumer and scientific volunteers who would have the credentials to serve on a GWC. From that relationship, each organization has knowledge of individual volunteers' life experiences, professional work, and personal and professional views. It has been the practice of the Task Force to maintain a balance of members with and without RWI so that a minimum of 50% of the members have no relevant RWI. In addition, the expertise of the GWC members is reviewed carefully to ensure that there is a balance of perspectives. The ACCF and AHA could request that other organizations meet these same requirements. In addition, it is the practice of the ACCF and AHA to provide additional scrutiny of nominations by officers and senior staff before the Task Force review. Seeking nominations from similar organizations would reduce the likelihood of nominees with unknown perspectives. 2 Any candidate for GWC membership may be requested to submit a detailed curriculum vitae, which the Task Force and the GWC chair and staff can review for possible intellectual or clinical practice perspectives (or bias). 3 The GWC chair, at his or her discretion, should be empowered to interview proposed GWC members who are not well known or who have possible or unknown intellectual or clinical practice perspectives. 4 Candidates for GWC membership may be asked to complete a questionnaire through which they disclose circumstances that might give rise to a potential or actual perspective or bias related to the GWC assignment. The questionnaire could include inquiries found in questionnaires typically used by charitable health organizations with respect to director and officer perspectives or bias. As some individuals may have difficulty recognizing or disclosing their own intellectual or practice perspectives that may lead to a conflict, questionnaire inquiries should be broadly worded to obtain information from which the GWC chair, Task Force, or staff could identify actual or potential intellectual or practice perspectives. 5 At each meeting of the GWC, remind the members of the policy whereby the group was formulated to incorporate a balance of RWI and varying intellectual and clinical practice perspectives that we continue to manage throughout the guideline writing effort. 6 Prospective GWC members should be educated about being transparent with respect to their individual position and be required to confirm that they will give priority to ultimate patient health and well-being, quality care, and value in their input and deliberation and when voting on issues before the GWC. Inclusion of Essential GWC Members With Known Intellectual and Clinical Practice Perspectives To develop the most effective CPGs, a GWC may require the participation of an individual who is outspoken in support of or against a particular procedure, medication, or other matter relevant to the GWC's work in the absence of supportive data and therefore has a known intellectual or clinical practice perspective. In such cases, the Task Force should make certain that the GWC chair is informed of the individual's perspective and that the GWC membership is balanced in viewpoint. All GWCs should be composed of members determined to be able to apply fair judgment with respect to all issues and particularly those in contention, so that they will be able to make objective assessments of all the information relevant to the GWC task. In addition, the GWC chair should make certain that the individual follows the policy and procedures noted in the ACCF/AHA Methodology Manual and Policies ( 4 ) as it applies to recusal from discussion at the request of the chair. Importance of Intellectual, in Comparison to Financial Conflict and RWI, to External Organizations and Government Regulators Although conflicts arising from RWI have received most of the attention by external groups, the Task Force should effectively address all potential conflicts, whether arising from RWI or intellectual or clinical practice perspective. The Task Force should be prepared to demonstrate to external groups that its policies for managing any conflict are practical and effective. Maintaining a Majority of GWC Members Without RWI and a Balance of GWC Members With Intellectual and Clinical Practice Perspectives Throughout the Duration of CPG Development All GWC members should be assessed by the Task Force and GWC chair for potential relevant RWI at the outset of the process. A majority of members who have no relevant RWI must be assigned. They will be asked to commit to developing no new RWI or other relationships that may represent potential intellectual perspectives (e.g., serve as principal investigator of a new study relevant to the CPG topic) during the course of their service on the GWC. As RWI or intellectual perspectives may arise in connection with any one of the many recommendations a GWC may make in developing a CPG, the GWC chair and staff must be vigilant in identifying any relationships as they arise and adjudicating them in accordance with established policies. 1.2 Funding Information: 3. Standards for Finding and Assessing Individual Studies IOM Standards and Elements ACCF/AHA Current Methodology Workgroup 3 Proposal 3.1 Conduct a comprehensive, systematic search for evidence Two researchers are proficient in the SR process and performing literature searches. Three project managers are proficient in using PubMed. Researchers work closely with the GWC to perform literature searches with key search terms and obtain full-text copies of articles. However, some GWC members conduct their own literature searches. Data supplement tables are created for key studies. The following are general recommendations for conducting SR: • Given existing resources, conducting SR of evidence for the entire scope of topics in an ACCF/AHA CPG may not be feasible. Whenever possible, existing, high-quality SR conducted by external groups (e.g., the Cochrane Collaboration, AHRQ, and journal-based SR) should be used. A search for SRs should be conducted at the beginning of the guideline process. • In the absence of existing SRs, a dedicated ERC, separate from the GWC, should conduct de novo SR on a limited number of clinical questions within a CPG. ( Note : This approach would initially be piloted with a focused approach to a confined topic, perhaps with 1 or 2 ACCF/AHA Focused Updates; see Workgroup 1). • When selecting clinical questions for SR, the focus should be on key levers in the process of care, especially on clinically critical topics for which there may be substantial evidence and/or a pressing need for guidance. • When conducting a comprehensive SR is not feasible due to resource or other constraints, the processes and procedures used to review the evidence should be clearly described. • The GWC and the ERC should define the population and setting under consideration. In addition, the GWC and ERC will prespecify topics of potential recommendations, including potential interventions, comparators, and outcomes for evidence review. • Following the SR pilots on key clinical questions/topics: 1 the feasibility of conducting SR on a greater number of topics within a guideline should be evaluated; 2 to maximize resources, the ACCF/AHA should consider partnering with other organizations that conduct SR as stand-alone products or to support guideline development; and, 3 additional funding to contract with an EPC to conduct SR should be considered. 3.1.1 Work with a librarian or other information specialist trained in performing SR to plan the search strategy No librarian is on staff. Key questions in PICO(TS) format are not used; however, key search terms are used based on the CPG table of contents. GWC members often ask research staff to perform literature searches using a literature search form; however, some GWC members conduct their own literature searches. It is recommended that all evidence searches be conducted by trained staff research specialists. The ERC should include a librarian and/or an in-house information specialist(s) who will work with the ERC and the GWC to plan the search strategy. Training in developing comprehensive search strategies and conducting searches should be provided to staff research specialists (e.g., National Library of Medicine courses or on-site training by an information specialist or librarian). 3.1.2 Design the search strategy to address each key research question GWC members determine the search strategy using the CPG table of contents, a spreadsheet of LBCTs, and literature used in the previous CPG. New CPGs or full revisions have no time limit on searches, whereas searches for focused updates are limited based on the time following the last full revision of the CPG. For each SR topic, the search should be structured based on a clear clinical question described in the PICO(TS) format. For all other clinical topics in the CPG, a PICO(TS)-like format should be used to structure and clearly describe a broad search strategy. 3.1.3 Use an independent librarian or other information specialist to peer review the search strategy GWC members may provide peer examination, input into search strategies, and key words. Contracting with a librarian or information specialist on an as-needed or project basis to review search strategies, especially in the pilot phase, should be considered. 3.1.4 Search bibliographic databases Databases used include MEDLINE, PubMed, clinicaltrials.gov , and the list of trials on the ACCF Cardiosource Web site. The ACCF/AHA CPG methodology is currently compliant. 3.1.5 Search citation indexes PubMed's “search for additional articles” and Google Scholar are both used. Citation indexes (e.g., Scopus and Web of Science) should be searched whenever possible. 3.1.6 Search literature cited by eligible studies See 3.1.5. Unpublished data are only considered if the data were 1) presented at a major scientific meeting, 2) not older than 2 years, 3) not used to support a recommendation, and 4) not used in CPG figures and tables (unless the data have important public health implications; this is reviewed by the Task Force on an-ad hoc basis). Additional references may be submitted to the ERC by the GWC. Reference lists of eligible studies should be reviewed to identify additional relevant studies. Every effort should be made to review all clinically relevant literature. Derivative studies, secondary analyses, substudies, and confirmatory/refuting research reports are by definition relevant but provide a different LOE than the primary results of a properly conducted clinical trial. Additional references may be submitted to the ERC by the GWC. 3.1.7 Update the search at intervals appropriate to the pace of generation of new information for the research question being addressed The time period for searches is stated in the CPG introduction. Articles in press are followed to determine if they will be published within the CPG development period. Cardiovascular scientific meetings that use the English language are routinely monitored. ACCF/AHA staff research specialists currently conduct biannual reviews of emerging literature, abstracts, and presentations. Task Force and GWC members also identify important studies and emerging literature. ACCF/AHA staff research specialists currently conduct biannual reviews of emerging literature, abstracts, and presentations. Task Force and GWC members also identify important studies and emerging literature. Adjustments to the current focused update process might include 1) creation of a calendar of prespecified dates, roughly oriented around premier scientific meetings (ACCF/AHA, ESC, HRS, and Transcatheter Therapeutics); 2) delineation of a clear structure around who other than Task Force members has input into the selection and assessment of new data; and 3) consideration of a mechanism using the website or a questionnaire to solicit suggestions from ACCF/AHA members. Automated processes, such as My NCBI in PubMed, should be considered to rerun original searches for each clinical question or topic on a regular basis. If a focused update is to be the pilot of an SR, then the appropriate PICO(TS) question would need formulation and all relevant evidence (not just new evidence) would need to be reviewed. A “focused” update does not imply a limited SR. It does imply that the update question might be adequately framed in a single PICO(TS) question for SR. 3.1.8 Search subject-specific databases if other databases are unlikely to provide all relevant evidence See 3.1.4. for search subject-specific databases used. Subject-specific databases should be reviewed as suggested by the ERC and the GWC. 3.1.9 Search regional bibliographic databases if other databases are unlikely to provide all relevant evidence EMBASE is currently not used, but the department is obtaining authorization to use EMBASE, and staff will be trained in its use. EMBASE should be added to the list of routine databases to be searched. Other regional bibliographic databases should be added as appropriate to the topic and/or as suggested by ERC and GWC members. 3.2 Take action to address potentially biased reporting of research results 3.2.1 Search grey-literature databases, clinical trial registries, and other sources of unpublished information about studies Grey-literature databases searched include monitoring registries such as the NCDR, FDA Websites, LBCTs presented at scientific sessions, and, when appropriate, ongoing trials covered within guidelines. Unpublished abstracts <2 years old may be addressed/included. Online databases such as MEDLINE/PubMed, Cardiosource Clinical Trials Database , and the Cochrane library are also searched. The ACCF/AHA CPG methodology is currently compliant. Another important source of information might be postapproval clinical registries required by the FDA or CMS for devices or drugs. Collaboration with the FDA or CMS would be important in this regard. The grey literature should be searched, especially for signals of unreported safety issues. Lack of peer review is an issue that needs consideration. 3.2.2 Invite researchers to clarify information related to study eligibility, study characteristics, and risk of bias When necessary, GWC members work jointly with CPG staff to clarify study eligibility, study design, and/or study characteristics in order to refine literature searches. Primary study authors may be contacted about questions pertaining to the study. If a large or important study is potentially downgraded by the absence of information about study eligibility, study characteristics, or control of bias, that study should be flagged and an inquiry sent to authors. The corresponding author would be the primary point person in this regard. Unfortunately, this has the potential for continuation of bias or introduction of additional bias, because the response of the corresponding author might neither be audited nor peer reviewed. 3.2.3 Invite all study sponsors and researchers to submit unpublished data, including unreported outcomes, for possible inclusion in the SR Study sponsors/researchers may be approached about unpublished data or unreported outcome, which may be described in the text or data summary tables. The results of unpublished data considered include data presented at a major national or international scientific meeting and data presented <2 years prior. GWC members may also contact scientific meeting presenters to ask questions, clarify matters, obtain presentation slides, perform detailed reviews, or request general guidance. If important unpublished data are identified, the ERC should request the data, document the request for additional information, and report on the disposition of the request in a publicly accessible appendix to the CPG. 3.2.4 Hand search selected journals and conference abstracts Tables of contents of major medical journals (e.g., Journal of the American Medical Association, New England Journal of Medicine, The Lancet, Journal of the American College of Cardiology, and Circulation ) are received weekly by guideline staff, and full-text articles pertinent to GWC members are sent electronically. Guideline staff maintains electronic copies of articles relevant to published CPG or CPG in development. The ACCF/AHA CPG methodology is currently compliant. 3.2.5 Conduct a Web search Web searches are conducted by GWC and CPG staff. Sites searched include PubMed, Cardiosource clinical trials , clinicaltrials.gov , controlled-trials.com , Google, and Google Scholar. The ACCF/AHA CPG methodology is currently compliant. 3.2.6 Search for studies reported in languages other than English if appropriate PubMed search features allow the inclusion of English-only journals. Each CPG introduction clarifies if the searches were limited to the English language. Articles published in a language other than English but translated into English may be used in the CPG reference list (e.g., articles in the Canadian Medical Association Journal often report trials in French and English). Non-English articles are identified by the primary language reported in parentheses as cited in the reference list. The ACCF/AHA CPG methodology recommends inclusion of studies published simultaneously in English and other languages. However, studies published in a language other than English and that do not include an English version are neither translated nor included in the evidence reviews. It is recommended that the ERC or GWC be permitted to consider studies reported in languages other than English for consideration. 3.3 Screen and select studies 3.3.1 Include or exclude studies based on the protocol's prespecified criteria Studies are included or excluded on the basis of GWC-determined prespecified criteria. The scope of the CPGs and review of LBCTs also influence which studies may be included or excluded, as well as studies pertinent to the CPGs. The ACCF/AHA CPG methodology should be more explicit in identifying the prespecified criteria for inclusion or exclusion of studies. The use of clinical questions in a PICO(TS) format would facilitate this process. 3.3.2 Use observational studies in addition to RCT to evaluate harms of interventions Literature searches focus on published articles only (i.e., RCT followed by observational studies, meta-analyses, and SR). The majority of literature searches focus on RCTs but also include nonrandomized studies (i.e., observational studies, case studies, and opinion documents). The ACCF/AHA CPG methodology is currently compliant. Observational studies and databases of studies not completed or published should be searched for potential harms of intervention. 3.3.3 Use two or more members of the review team, working independently, to screen and select studies CPG sections may be drafted by GWC members who have expertise and in-depth knowledge of the subject or by a group of GWC members. As a result, relevant studies for each section may be screened independently by multiple GWC members. However, the screening and selection of studies is not explicitly done by multiple members. Dual screening and selection of eligible studies is currently not conducted. To incorporate dual reviewing processes would require additional staff. Training a cadre of volunteer members to conduct dual reviews as part of an overall ERC training program should be considered. ACCF/AHA members early in their academic careers may find benefit in becoming involved in this process with no financial remuneration. Academic credit might include letters to department chairs and listing ERC members as coauthors of the CPG along with the GWC. 3.3.4 Train screeners using written documentation; test and retest screeners to improve accuracy and consistency GWC committee members are given an in-depth orientation to the ACCF/AHA Methodology Manual ( 4 ), as well as in-depth orientation about how to populate data summary tables. Written procedures and forms for screening and selecting studies should be developed. Members of the ERC who serve as screeners will need training and verification of accuracy and consistency. 3.3.5 Use one of two strategies to select studies: 1) read all full-text articles identified in the search, or 2) screen titles and abstracts of all articles and then read the full text of articles identified in initial screening Research staff perform literature searches, and a document containing the study reference and abstract is emailed to the appropriate GWC member(s). Full-text copies of relevant articles are also identified by the research analyst and sent to GWC members. The final selection of relevant studies is the purview of the GWC member, so the actual strategy used is not explicit or mandated. Titles and abstracts of all articles should be screened, and then the full text of potentially relevant articles identified in the initial screening should be read. The screening process should be conducted by the individuals identified and trained as members of the ERC, separate from the GWC. 3.3.6 Taking account of the risk of bias, consider including observational studies to address gaps in the evidence from RCT on the benefits of interventions All trials (e.g., RCT and observational trials) are examined by GWC members for possible inclusion in the evidence review. The ACCF/AHA CPG methodology is currently compliant. A search should be performed for observational studies and unpublished studies, looking for benefits and potential harm, which would otherwise be missed if only RCT are reviewed. Audited data, such as registry data submitted for FDA approval, may be considered, with due consideration of the lack of peer review and the importance of an unbiased audit process. 3.4 Document the search 3.4.1 Provide a line-by-line description of the search strategy, including the date of every search for each database, web browser, etc. Research staff maintains an electronic file of searches conducted by either staff or GWC members. A hard copy of the standard literature search form is also kept on file. Text describing literature search criteria and key search terms is included in the CPG introduction, thereby allowing CPG users access to the comprehensive literature search conducted. The ACCF/AHA CPG methodology is currently compliant. 3.4.2 Document the disposition of each report identified including reasons for their exclusion if appropriate Excluded studies or reports are not called out specifically within the CPG. Use of a flowchart should be considered to depict the number of studies that remain after each stage of the selection process. Reasons for exclusion of peer-reviewed RCTs and large observational trials published in major journals should be documented in a Web-accessible appendix to the CPG. Software such as Reference Manager should be potentially used to document reports, with fields for entering reasons for exclusion, such as “did not meet fields of interest.” 3.5 Manage data collection 3.5.1 At a minimum, use two or more researchers, working independently, to extract quantitative or other critical data from each study. For other types of data, one individual could extract the data while the second individual checks for accuracy and completeness. Establish a fair procedure for resolving discrepancies—do not simply give final decision-making power to the senior reviewer Currently 1 staff researcher is assigned to assist GWC members on 1 CPG; however, additional research staff are available to assist and work in a collaborative manner. GWC members also review articles/studies in detail individually and in small or large groups. Currently, there is no mandate for the number of researchers/GWC members required to review each article. Two members of the ERC should extract quantitative or other critical data from each study. A third individual should be involved in resolution of discrepancies. These 3 individuals should be members of the ERC, not the GWC. Their work should lead to coauthorship of the finished guideline. In some CPG, there may be more than one 3-person ERC team for review of evidence in different parts of the CPG, depending on the scope. 3.5.2 Link publications from the same study to avoid including data from the same study more than once Multiple publications of the same study are identified by GWC members, thereby decreasing the possibility of repeating information or data. The ACCF/AHA CPG methodology is currently compliant. 3.5.3 Use standard data extraction forms developed for the specific SR Evidence tables with a uniform set of key characteristics identified by GWC members for each guideline are used to standardize data collection. A standardized data extraction form should be developed that includes some standard elements and additional elements unique to the PICO(TS) question. For broader clinical questions not included in a formal SR, a similar data extraction form should be developed . 3.5.4 Pilot-test the data extraction forms and process The format of data supplement tables undergoes multiple revisions before their adoption and use, but no formal pilot-testing is done. The data extraction form should be piloted in a few studies and then revised. 3.6 Critically appraise each study 3.6.1 Systematically assess the risk of bias, using predefined criteria GWC expertise is used to assess the risk of bias. The ACCF/AHA system for assessing bias is currently subjective and qualitative. Documentation of the assessment of the risk of bias is essential. The system used to assess bias is controversial. Multiple tools exist, including one in evaluation by ACCF/AHA. The Cochrane Risk of Bias tool is a qualitative, descriptive tool used to evaluate individual studies across 6 domains of bias: selection, performance, detection, attrition, reporting, and other. In each domain, bias is judged as low, high, or unclear, with brief supporting comments for each judgment. There is also a summary judgment for each study, recommended to be outcome specific. For each study, the same 2 ERC group members performing data extraction could simultaneously perform a quality assessment, with a third individual to adjudicate differences in data extraction or quality assessment. A separate risk of bias (separate from the RCT tool) tool is needed for observational studies and registries. 3.6.2 Assess relevance of the study's populations, interventions, and outcome measures The ACCF/AHA system for assessing bias is currently subjective and qualitative. GWC members are relied on to implicitly assess the relevance of the study population, intervention, and outcome measures. The relevance of each study's populations, interventions, comparators, outcomes, timing, and setting to the key clinical question/topic of interest should be assessed. 3.6.3 Assess the fidelity of the implementation of interventions The GWC reviews the study interventions implicitly to ensure/assess the fidelity of the implementation of interventions. For each study, an explicit assessment of the extent to which the intervention was delivered as planned and the quality of delivery of the intervention should be included. ACCF indicates American College of Cardiology Foundation; AHA, American Heart Association; AHRQ, Agency for Healthcare Research and Quality; CMS, Centers for Medicare and Medicaid Services; CPG, clinical practice guideline; EPC, evidence-based practice center; ERC, evidence review committee; ESC, European Society of Cardiology; FDA, U.S. Food and Drug Administration; GWC, guideline writing committee; HRS, Heart Rhythm Society; IOM, Institute of Medicine; LBCT, late-breaking clinical trial; LOE, level of evidence; NCBI, National Center for Biotechnology Information; NCDR, National Cardiovascular Data Registry; PICO(TS), mnemonic: population, intervention, comparator, outcomes, timing, and setting; RCT, randomized controlled trial; SR, systematic review; and Task Force, ACCF/AHA Task Force on Practice Guidelines.

PY - 2013/1/15

Y1 - 2013/1/15

UR - http://www.scopus.com/inward/record.url?scp=84871874030&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84871874030&partnerID=8YFLogxK

U2 - 10.1016/j.jacc.2012.09.025

DO - 10.1016/j.jacc.2012.09.025

M3 - Article

C2 - 23238451

AN - SCOPUS:84871874030

SN - 0735-1097

VL - 61

SP - 213

EP - 265

JO - Journal of the American College of Cardiology

JF - Journal of the American College of Cardiology

IS - 2

ER -