Clinical study protocols are important cornerstones on the journey of collecting the clinical data needed to elucidate the benefit-risk profile of medical treatments/devices. Not only do they lay down the plan for the study at hand, they serve as a repository of knowledge for the strategic intentions and will be used for multiple down-stream activities. They are the basis for trial registration, provide guidance to those running the study, and will be used by participants, ethics committees/institutional review boards, funders, regulators, journal editors, and systematic reviewers to appraise the study. Yet, many protocols do not fully and clearly address important study elements, which impairs understanding and implementation, often leads to protocol amendments, and reduces the efficiency of using protocols in later stage activities (e.g. writing of the clinical study reports and clinical summaries for regulatory submissions). The importance of these documents is often not truly recognized and the impact of getting them wrong costs the industry billions and delays, or even hinders, drug development. This article looks at what should be considered to optimize the designing and writing of a good protocol.
Clinical studies frequently falter because they try to do too much, and their design and intent is poorly communicated in their study protocols. The more complex a protocol becomes, the less likely it is to perform well (1). More complex protocols tend to have more amendments, longer cycle times, and poor recruitment and retention rates (2, 3, 4). A recent Tufts study that gathered data on substantial global protocol amendments found that protocols with global amendments were larger in scope and had longer patient recruitment and overall study durations compared with those without a global amendment (2). Notably, almost half (45%) of the amendments in these studies were considered to be avoidable (2).
Over the last 20 years, there has been a significant increase in the complexity of study designs, resulting in lower quality clinical data and increased trial costs (5). Despite initiatives such as the SPIRIT 2013 checklist and guidelines (6) and adaptive trial design techniques intended to simplify protocol design and reduce unnecessary procedures in trials, companies continue to run unnecessarily complex studies (5). These overly complex study designs and poorly crafted study protocols are a key reason for increased costs. It has been estimated that the cost of activities included in studies not considered essential to the objectives and endpoints is between $4 and $6 billion USD each year (5). In addition, the median cost of implementing a substantial amendment has been reported to be 141 000 USD for a phase 2 protocol and 535 000 for a phase 3 protocol (2). These are substantial costs that could be avoided if teams could be made to understand how to develop and write more effective study protocols.
What makes an effective study protocol?
There are 4 factors that are crucial to make a study protocol truly fit for purpose. It needs to be precise, simple, practical, and clear in terms of the study design and its communication. Instead of trying to do a bit of everything, the study should concentrate on a few precise questions and the study design should focus on answering these. Many studies collect far too much extraneous data, which means that the investigators and sites are busy doing things that won’t actually contribute to the task at hand (5, 6).
The practicality of running the study and the clarity with which the study is communicated is critical to its success. When writing a study protocol, teams need to consider how feasible it will be to do all the things planned. The logistics of the study design need to be practical in the real world. Many protocol amendments are needed because, after that study is up and running, it becomes clear that certain activities just can’t be done the way they were planned, or site staff misunderstand what they are meant to do. Protocols must present the ideas and activities in a clear and consistent way to make sure that everyone involved truly understands the intention of the study, what is meant to happen, and when. The user-friendliness of the protocol plays a huge role in ensuring that the same activities happen in the same way across all sites and will ultimately reflect on the consistency of the data collected across sites.
Appropriately accounting for all these factors contributes to a study protocol that is more likely to avoid misunderstanding by investigators (thereby avoiding protocol violations, difficulties in subject enrolment, etc.), is less expensive and more practical to run, and thus overall is more effective (1, 3, 5).
Common problems that prevent protocol effectiveness
The most common problems with study protocols are:
· too many objectives (i.e. a lack of precision),
· too many inclusion and exclusion criteria and too many secondary variables (a lack of simplicity),
· too many activities planned (a lack of practicality), and
· inconsistency and poor communication of the intentions (lack of clarity).
Study findings reported in 2012 by the Tufts Center for the Study of Drug Development found that a typical protocol has an average of 7 objectives and 13 endpoints (7). If a study is trying to answer 7 different key questions, that is probably 5 too many to truly answer any properly!
Consider this: how many secondary variables does it take to be sure you will be able to answer a question posed by a key objective (e.g. does this dose of the drug work)? Teams need to think about how the data collected in the study will be used and reported later in submission documents to tell the ‘big picture’ story. Often less is more and having too
much tangential information can simply cloud the picture and distract from what could otherwise be a crisp, clean message. Teams also need to keep in mind that drug development and the task of getting a drug approved is a different exercise than that of a scientific exploration of the many facets of a drug. They need to ask themselves what data are critical to demonstrate that the drug works and is safe in the particular indication being sought. If endpoints simply give the same answer in another way, a choice should be made to select the most relevant tests, and the others can be removed. Let’s face it, showing that a drug works (or doesn’t work) in 10 different ways is probably not needed; 5 good, solid ways are certainly sufficient.
Even if the study design is simple and practical, inconsistent and confusing information will not aid usability of the protocol. The different sections of a protocol should fulfil their intended purpose with minimal repetition between sections. For example, the introduction should make clear why this study is needed, the objectives section should clearly state the (limited) aims of the study, the assessments section should identify what is to be measured to answer the stated objectives, and the statistics section should make clear how the data collected from the assessments will be analysed. Study protocols often confuse study objectives with the study endpoints, such that endpoints are given in place of objectives, leaving the readers unclear about the fundamental aims of the study (which then must be extrapolated by the reader, based on the endpoints indicated). Similarly, it is often difficult to find important information in protocols, such as the exact dosing regimen, details about dietary or other restrictions during the study, or the intended timing for all activities (while key activities are usually described in detail, others are often only mentioned in passing, leaving site staff to guess how these are meant to be done).
Another common pitfall seen in protocols is a lack of consistency between key sections such as the study title, study objectives and the planned primary analysis. If these are not aligned, the investigators are left to assume the true intention and it is likely that different investigators will assume different things. This is not a situation conducive to obtaining streamlined and consistent data.
The importance of a good medical writer
An experienced medical writer increases the likelihood that the protocol will be practical and user friendly. Their experience in writing an array of protocols means they can make useful suggestions about how to depict complicated study designs to make them more understandable. In addition, when writing a protocol, a medical writer is envisioning how the data collected will appear in the study report and can help teams to understand, and perhaps choose, which parameters are going to be the most meaningful when it comes to crystalizing the take-home messages. They will help keep the team focussed on these details and flagging up places that could be simplified, details that could be removed, and inconsistencies between sections.
The mistake many companies make is to assume that the content stakeholders (e.g. clinical scientists, study manager, and statisticians) are also effective communicators. Out of convenience, many protocols are often written by teams at CROs who will be
responsible for running the study, generally as part of a full-service contract. This may mean the sponsor has one less contract to organise, but it does not guarantee they will have a trained medical writer in place to craft a strong protocol. While those teams may be very good at the skills needed to run a study (e.g. for recruiting, project management, study monitoring, etc.), they are not necessarily specialised in writing documents.
Conclusion
In today’s regulatory environment, many studies are too complex with protocols that are unclear and lacking important information. By consciously designing studies that are precise, simple, practical, and clearly communicated, we can improve the consistency and quality of the data that will be obtained. With a good writing process, more robust study protocols will be generated, which will have a direct impact on the subsequent drug development journey. With so much at stake – not only considerable costs but also the quality of the data obtained and the efficiency of many downstream activities – companies should do everything in their power to ensure that experienced medical writers are used to help the clinical teams craft protocols that communicate effectively. The success, timelines, and costs of a study should not be jeopardised for the convenience of including the protocol writing in a package deal with a CRO. It may not be the only way to improve the design and quality of clinical studies but having specialized medical writing companies involved in protocol writing can go a long way to improving clinical study protocols, which everyone can reap the benefits from.
References
1. Kaitin KI, editor. Optimizing protocol design to improve clinical study performance, efficiency, and cost. Tufts Center for the Study of Drug Development R&D Management Report. 2013 Apr;8(2) [RS 3313]
2. Getz K, Stergiopoulos S, Short M, et al. The Impact of Protocol Amendments on Clinical Trial Performance and Cost. Ther Innov Regul Sci. 2016;50(4):436-441.
3. Getz K. Predicting successful site performance. Applied Clinical Trials. 1 Nov 2011.
4. Getz K. Protocol amendments: a costly solution. Applied Clinical Trials. 1 May 2011.
5. Getz K. Improving Protocol Design Feasibility to Drive Drug Development Economics and Performance. International Journal of Environmental Research and Public Health. 2014;11:5069-5080.
6. SPIRIT Guidance for Clinical Trial Protocols: the SPIRIT 2013 statement. Accessed at http://www.spirit-statement.org
7. Kaitin KI, editor. One in five procedures generates extraneous clinical trials data. Tufts Center for the Study of Drug Development Impact Report. 2012 Nov/Dec;14(6) [RS 3235]
Contact details:
Dr. Julia Forjanic Klapproth
Senior Partner
Trilogy Writing & Consulting GmbH
Falkensteiner Str. 77
60322 Frankfurt
Germany
Phone: +49 69 138 252811 Email: julia@trilogywriting.com
Biography of the author:
Julia Forjanic Klapproth
Julia started working as a medical writer in the regulatory arena after getting her Ph.D. in Developmental Neurobiology in 1997. She is passionate about the value of good medical writers and has been President of the European Medical Writers Association (EMWA) twice (2001-2002, 2007-2009). Julia is an experienced trainer of medical writers, regularly running workshops for EMWA, AMWA, DIA, and pharmaceutical companies around the world. In 2002, Julia co-founded Trilogy Writing & Consulting, a company specialised in providing regulatory medical writing and she is still involved in writing and coordinating many documents, including study protocols, study reports, and CTD submission dossiers.