While systematic literature reviews (SLR) of indirect treatment comparisons (ITC) historically have not been commonly published in the medical literature, they represent an increasingly frequent research activity undertaken in various capacities relevant to the practice of health technology assessment (HTA). ITC-SLRs can provide valuable information for multiple audiences:
- For academic researchers involved in comparative effectiveness research, establishing the timeliness, methodologic rigor and validity of available ITCs for a disease and its related treatments can shed light on whether to: (a) invest efforts to develop a grant proposal for new funding to support an improved ITC; (b) dedicate time to the performance of new ITCs (as we know, studies such as NMAs take time!); and (c) prepare new work for the development or updating of new clinical practice guidelines, or other such applications. An objective, well performed SLR of published ITCs can be vital in helping researchers make choices about such activities, while laying groundwork for future research.
- For HTA agencies that receive submissions for many disease areas and emerging therapies at an increasingly rapid pace, developing a strong awareness of ITCs for a disease and its associated treatments can be helpful in several ways. First and foremost, it provides an understanding of the landscape of ITCs in the disease area regarding issues of methodologic approach, the relevance of clinical heterogeneity between studies (and its related implications for modeling decisions in ITCs), and potential considerations that may impact associated economic evaluations. This can provide organizational staff with enriched knowledge of the content area that will enhance the rigor of appraisals for future submissions in the disease space. Affiliated expert committees responsible for recommendations about treatment funding may also gain insights from having a greater awareness of ITCs (and their methodologies and limitations) in the clinical field.
- For drug manufacturers, SLRs of ITCs can be especially insightful in understanding how their product currently ranks against other comparators for a given disease based upon all of the available published ITC evidence. Appraising these clinical findings and correlating them with features of the ITCs such as methodologic quality, research question scope, evidence base considered, choice of analytic approach, subgroup / sensitivity analyses performed and outcomes assessed can provide valuable strategic insights to address concerns and to plan future ITCs to support the clinical benefits of their product.?
Some practical considerations for a structured methodologic approach for the performance of an ITC-SLR include the items below, all of which should be established a priori in a systematic review protocol as would be done when appraising the literature on a given clinical topic of interest. These include the following:
- Establish a question of appropriate scope. The research question should establish the clinical population of interest, the forms of ITCs that will be reviewed, the nature of the interventions of focus, and possibly also the outcomes that will be inspected.
- Design a thorough literature search. A multi-database search constructed of terminology for the condition of interest as well as the concept of ITCs and related terms should be used. Date restrictions may often be of interest to focus upon ITCs that encompass clinically relevant comparators. ?
- Use established methods for screening and data extraction. Methods for these tasks are analogous to those used in a traditional systematic review of clinical trials (e.g. independent reviewers or a reviewer with verification), and should be maintained to ensure high review quality while limiting the risk of missing important trials.
- Conduct a thorough data collection exercise. Sufficient information must be gathered to allow the review team and the SLR's end users to distinguish key differences in the design, implementation and findings of the included ITCs. This can include many considerations outlined below:
- Assess the credibility and relevance of each ITC. The ISPOR 2014 questionnaire designed to help decision makers judge the relevance and credibility of ITCs is a well-suited tool [reference provided below]. It contains 26 questions that can be answered by members of the review team to appraise the set of included ITCs on a range of key criteria, and provides an additional lens through which to examine the included ITCs and their findings.
- Synthesize comparisons of directionality and magnitude of treatment effects for the comparisons of interest. This may include the complete set of all pairwise comparisons of treatments as well as secondary treatment rankings considered (where available, e.g., SUCRA) for researchers and clinicians, while drug manufacturers may choose to focus their investigations on comparisons involving their product. Summary tables, heat maps and/or forest plots may all provide helpful in visualizing differences in the magnitude and directionality of comparisons across ITCs while trying to develop insights that go beyond a simple descriptive synthesis of all the ITCs.
- Review and discuss the compiled evidence. Members of the day-to-day review team, along with methodologic and clinical experts and those procuring the ITC-SLR should discuss and investigate the data collectively. Different types of expertise can help to identify and appraise potential associations between the ITC characteristics and their clinical findings. Multiple discussions may prove helpful in working through the intricacies of all the available information and to draw key interpretations.
- Proceed with plans for implementation / dissemination and future work. Depending upon the parties undertaking the ITC-SLR, this may include different considerations such as: recommendations on methodologic features of future ITCs in a clinical area; plans for grant submissions; education and training of HTA agency staff and guideline teams; plans for publication or conference dissemination; and/or initiation of strategic plans for a de novo ITC that optimizes methodologic approach and which may better represent the comparative effectiveness of competing therapies.
SLRs incorporating critical appraisals of published ITCs are not commonly published in the medical literature at this time. However, they are commonly performed by different audiences with different objectives, and can provide valuable insights for academic researchers, policy makers, HTA agencies and drug manufacturers alike.
Don’t hesitate to reach out and suggest additional methodologic features you feel should be included in a general framework for SLRs of ITCs. I hope to post a structured checklist that others may consider as a starting point for their own work in the future and would be pleased to receive feedback from those who share this common interest!
Jansen J, Trikalinos T, Cappelleri J, Daw J, Andes S, Eldessouki R, Salanti G. Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health. 2014 Mar;17(2):157-73. doi: 10.1016/j.jval.2014.01.004.
