Clinical studies often falter because they are poorly designed and/or communicated in the study protocols. So what makes a good protocol and how can the writing process be optimized?
Most study protocols are trying to do too much and not explaining what is required very well. The more complex a protocol becomes, the less likely it is to perform well, and more complex protocols tend to have more amendments, longer cycle times, and poor recruitment and retention rates [1-3]. For a protocol to really work, four factors are crucial in terms of the study design and its communication: precision, simplicity, practicality and clarity.
ESSENTIAL ELEMENTS
Instead of trying to cover everything, the protocol should concentrate on a few precise questions and make it clear how it will answer these. Many studies collect far too much extraneous data, which means the investigators and sites are busy doing things that will not actually contribute to the task at hand [4]. The design elements of a study should specifically provide the data needed to answer the few questions it is trying to ask.
The practicality of running the study and the clarity with which it is communicated are also critical to its success. When writing a protocol, the logistics of the study design need to be feasible in practice. Many protocol amendments become necessary because sites tell the sponsor that certain activities simply cannot be done as planned, or they misunderstand what was required.
A protocol must present the ideas and activities in a clear and consistent way to ensure that everyone involved truly understands the intention of the study, what is meant to happen, and when. The user-friendliness of the protocol plays a huge role in making sure that the same activities happen in the same way across numerous sites, and will ultimately reflect on the consistency of the data collected.
All of these factors will contribute to a protocol that is more likely to avoid misunderstanding by investigators (thereby avoiding protocol violations, difficulties in subject enrolment, and so on), is less expensive, more practical to run, and more effective overall [1,2].
COMMON PROBLEMS
Generally, the reason studies do not run well is because the protocols have not adequately considered the factors mentioned above. Common problems include too many objectives, too many inclusion and exclusion criteria and secondary variables, too many activities planned, and inconsistency and poor communication of the intentions.
Study findings reported in 2012 by the Tufts Center for the Study of Drug Development found that a typical protocol has an average of seven objectives and 13 endpoints [4]. If a study is trying to answer seven different questions, that is probably five too many to truly answer properly.
Similarly, how many secondary variables does it take to be sure that an answer is provided? Teams need to think about how the data to be collected will be used and reported later in submission documents to tell the ‘big picture’ story. Often, less is more, and having too much tangential information can simply cloud the picture and distract from what could otherwise be a crisp, clean message.
Teams need to keep in mind that drug development and the task of getting a drug approved for use is a different exercise to the scientific exploration of a drug. They need to ask what data are critical to demonstrate that the drug works and is safe in the particular indication sought. If endpoints simply give the same answer in another way, a choice should be made as to the most relevant tests, and the others can be removed. Showing that a drug works (or not) in 10 different ways is probably not needed; five good, solid ways are certainly sufficient.
CLEAR AND CONSISTENT
Even if the study design is simple and practical, it will not make a difference if the information presented in the protocol is inconsistent and confusing. The different sections of a protocol should fulfil their intended purpose with minimal repetition. For example, the introduction should make clear why this study is needed; the objectives section should clearly set out the study aims; the assessments section should identify what is to be measured to answer the stated objectives; and the statistics section should make clear how the data from the assessments will be analysed.
Protocols often confuse study objectives with the study endpoints, such that endpoints are given in the objectives section, leaving the readers unclear about the fundamental aims of the study (which must then be extrapolated by the reader, based on the endpoints indicated).
Similarly, it is often difficult to find important information in protocols, such as the exact dosing regimen, details about dietary or other restrictions during the study, or the intended timing for all activities. While key activities are usually described in detail, others are often only mentioned in passing, leaving site staff to guess how these are meant to be done.
Another common pitfall is a lack of consistency between key sections such as the study title, study objectives and the planned primary analysis (see the case study box). If these are not aligned, the investigators are left to assume what is really intended, and it is likely that different investigators will assume different things. This is not a situation conducive to obtaining streamlined data.
STAKEHOLDER INVOLVEMENT
The design and concept of a study is influenced by many different stakeholders, each of whom considers different, important aspects. Unfortunately, many study protocols are prepared by only a few of the key stakeholders, typically the regulatory and clinical functions of the sponsor, plus the study statistician. However, to be sure the study is feasible and to understand if it is being communicated effectively, early drafts should be reviewed by the investigators, study staff, laboratories and other specialist providers who will be involved, along with a medical writer.
Do not overlook the fact that the medical writer is a key stakeholder in their own right. Their experience in writing an array of protocols means they can make useful suggestions about how to depict complicated study designs to make them more understandable, or suggest which protocol structures are more effective.
In addition, when writing a protocol, a medical writer is already envisioning how the data collected will appear in the study report and can help teams to understand and perhaps choose which parameters are going to be the most meaningful when it comes to crystallising the take-home messages.
Each of these stakeholders will review the protocol from a different perspective and provide important feedback that can help iron out problems before the study starts. They will often identify key missing information, confusing parts of the document, and activities that are impractical to perform. Getting them involved in the review of early drafts increases the likelihood that the protocol will be practical and well understood, and will decrease the need for amendments later on.
IDEAL WRITING PROCESS
The difficulty in preparing a protocol that addresses all of the issues discussed above lies in the writing process. Writing a protocol needs to start by getting input and agreement from all stakeholders. To do this properly, a detailed synopsis should be prepared that does more than identify the objectives, endpoints and sample size calculation. The devil is in the detail, and we only start to understand the difficulties of a study by teasing apart how exactly it will glean the data that is needed.
The synopsis should clearly describe all inclusion and exclusion criteria, the intended treatments (with detailed dosing regimens), and have a full study schedule attached to it. It is easy for management and investigators to look at a few objectives and agree to the concept in general; it is something else to look at the details of implementing those objectives and agree to what will really be done.
That is why this detailed synopsis should be given to everyone involved in the study before the protocol itself is written. Upper management of the sponsor, as well as all the stakeholders, should have a chance to provide input at this stage.
For some strange reason, companies tend to wait until there is a well-developed or final protocol before they send it to the laboratories, site staff, investigators and specialist analysis groups for agreement on the concept. Not surprisingly, this often results in considerable reworking of the protocol at a late stage of the writing process.
PLAN OF ACTION
After everyone has had a chance to review the synopsis, there should be a kick-off meeting with as many of the key stakeholders as possible. This is to confirm the intentions of the study; to make sure that critical decisions are made as a team; to identify inconsistencies and impracticalities in the concept or details of performing the study; and to create action items assigned to the stakeholders to gather important information that is needed to prepare a complete protocol. So many fundamental points are often discovered and discussed during this meeting that can prevent potential problems before they happen.
Once the stakeholder team has provided as much input as possible, the synopsis can be turned into a full protocol. This saves unnecessary rewriting of the document, because at this point the concept is fairly clear and stable. Nonetheless, the first draft should be given to all stakeholders again, so that they can evaluate the concept after it has been turned into a plan of action.
It is essential that the medical writer is involved in review meetings to discuss the protocol, so that they are aware of the reasoning behind any changes needed and can make sure that the appropriate revisions are made. As the person consolidating ideas, the medical writer is often in the best position to direct the team.
What results is a protocol that is well designed and communicated, and easy for the study sites to follow, which will improve the consistency and quality of the study data and ultimately increase adherence and compliance rates.
REFERENCES
- Kaitin KI, Optimizing protocol design to improve clinical study performance, efficiency and cost, Tufts Center for the Study of Drug Development, R&D Management Report 8(2), April 2013
- Getz K, Predicting successful site performance, Applied Clinical Trials, November 2011
- Getz K, Protocol amendments: a costly solution, Applied Clinical Trials, May 2011
- Kaitin KI, One in five procedures generates extraneous clinical trials data, Tufts Center for the Study of Drug Development, Impact Report 14(6), November/ December 2012
Julia Forjanic Klapproth
Published in: International Clinical Trials, August 2013