Introduction:

Risk based monitoring is the process of assuring that the quality of clinical trials is maintained by identifying, assessing, monitoring, and reducing the risks which could affect the safety of a study. US Food and Drug Administration (FDA) guidance on RBM describes three steps:

 a. Identify critical data and processes – In a clinical trial, the Quality of a study data and the safety of the participants in the trial are accurately monitored, for each study sponsor should be aware that elements from the informed consent to eligibility to screening and tracking of adverse event are important.

b. Perform a risk assessment - A risk assessment is done to evaluate sources of risk and the effect of study errors on those risks.

c. Develop a monitoring plan – According to guidance of FDA’s, the monitoring plan should define monitoring methods, responsibilities of the personnel involved in trial conduct, and requirements of trial. Monitoring plan is for everyone involved in monitoring trial for communicating risks and monitoring procedures. ^[1]

 

Centralized monitoring is enhanced statistical approach to handle information available from across sites or across patients within a clinical research study. The major steps involved in centralized monitoring implementation are:

• -> Identify & Define Key risk indicators and their thresholds
• -> Designing monitoring plan
• ->Continuous assessment of risks or issues using the right technology
• ->Development of risk mitigation plan

 

Well defined monitoring plan is necessary for efficient and effective centralized monitoring implementation. Monitoring plan should define aspects of monitoring which would be achieved through centralized monitoring. The plan should indicate process for site contacts, or triggered on-site monitoring based on the critical issues identified. The plan should outline, the key risk indicators, their thresholds and performance metrics. The regulators require sponsors to implement a systematic approach to identify and justify the key risk indicators. The monitoring plan should be flexible to incorporate amendments based on new risks identified related to data, procedures, sites or overall operations. ^[2],[3]

 

Embracing the concept of RBM requires a shift in mindset from traditional monitoring, and drug developers may feel cautious of, or resistant to, this change. This may be due, in part, to uncertainty about how the FDA will respond to the use of RBM techniques. However, sponsors and CROs should also consider how regulatory agencies might respond if RBM modifications are not made. For companies who are uncomfortable making the shift to RBM, a useful exercise would be to evaluate the factors fueling that hesitation and figure out how to address them.         

Even for companies that are ready to transition to RBM, the change can be daunting, and proactive change management is essential. The move toward an RBM approach requires change management related to people, process, and technology. Commitment at the leadership level, along with staff training and a clear understanding of each team member’s roles and responsibilities, helps ensure that RBM is incorporated into the DNA of both the company and the clinical trial.

 

There are a growing number of RBM technologies available, but drug developers should keep in mind that features vary widely from software to software and many fall short of delivering the full benefits of RBM. Some RBM technologies only look at risk in isolation, without context or analysis. Others may be limited by their reliance on retrospective data, missing the critical capability to draw insight from emerging issues or placing undue emphasis on irrelevant historical information. An incomplete approach to identifying, managing, and analysing risk introduces complexity and cost without adding value. Moreover, a system that fails to fully assess risk could also introduce a new risk: a false sense of security. Sponsors and CROs may mistakenly believe that they are utilizing RBM, without fully benefiting from all that RBM can offer. ^[4]

The research done to understand the acceptance of industry in adopting the RBM model indicates that before the COVID-19 pandemic the industry adoption of RBM is less extensive than expected, majorly because companies were reluctant to completely change their existing practices and protocols. One common reason for incomplete adoption and partial implementation of RBM is a hesitance on the part of trial sponsors and CROs to reduce the amount of SDR/SDV in favour of a more targeted approach. Post COVID-19 pandemic, the companies naturally experimented with transitioning to remote-site monitoring to avoid trial disruptions. The data collected during these times clearly indicate that the effectiveness of remote-site monitoring is equivalent to that of on-site monitoring. ^{[5], [6], [7], [8], [9], [10]}

 

Methodology:

This study used a survey method to collect data about the opinions and experiences of the clinical research professionals who have worked on risk-based management systems at some point of time in their clinical research career. Based on the sample size provided by the statistician the targeted sample size was 500 responses to the survey questionnaire. The survey was available on the electronic portal and the link to the survey was shared with participants using various modes of communications like mail, phone, social media etc. This research is a questionnaire-based survey project targeting to receive the responses from 500 clinical research professionals. For handling such a huge influx of data, an electronic data capture system was built. This data capture system was developed by an external vendor taking into considerations the requirements of the project.

 

Results:

The questionnaire was sent to multiple clinical research professionals worldwide. However, the complete response to the questionnaire was received from 511 participants.

Descriptive analysis of general information collected from the participants:

 

Figure 1

59.3% (303 participants) of the respondent’s current organization is CRO, while 34.1% (174 participants) work for sponsor companies, and 6.1% (34 participants) work for other type of organization i.e., Site Management Organizations.

 

Figure 2

39.7% (203 participants) have experience of working only in the global trial, 11.9% (61 participants) have experience of working only in the local trial and 48.3% (247 participants) have work experiences of both local and global trials.

 

76.1% (390 respondents) had experience in working with risk-based monitoring while 23.9% (122 respondents) had no previous working experience in RBM. 30.50% (156 participants) have experience of 1-5 years, 33.30% (170 respondents) have 6-10 years of experience, and 12.30% (63 respondents) have 11-15 years of experience in the field.

 

Figure 3

The above figure represents the current designation of the respondent in different organizations at different roles. The majority of the respondents are clinical research associates, Project managers, clinical trial managers, and central monitors including trip report reviewers.

 

The participants were asked, ‘What according to them are the capabilities needed to support different operating models across the industry?’ 76.12% (389) of the participants said, a source system that supports the risk indicators is needed. 67.90% (347) of the participants said, a mapping engine, which allows the source data to be mapped to an internal data warehouse that supports RBM.

 

The next question asked was, ‘What are the characteristics needed for developing End-user face of the systems involved in RBM?’ 74.16% (379) participants chose the option of having a User-friendly interface (eg, wizard-based) (A source system that supports the risk indicators). Another option given to the participants was the Configurable application of algorithms at user-defined levels of study (eg. study, region, country, on-site monitoring vendor, site) which was selected by 33.85% (173) of the participants.

 

The participants were asked what are some of the common expectations from the systems that are currently in use. 78.27% (400) of the participants said that they system should be able to identify trends, patterns and identifiers. 31.89% (163) participants said that the systems should be able to calculate and display simple and easy-to-understand performance scores. 335 (65.55%) participants said that the system should be able to track issue management. 105 participants (20.54%) said that the system should provide user-friendly administration screens to administer risk indicators.

 

Figure 4

The next question was, ‘New mobile technologies like smartphones and fit bits to collect patient data can become a new operational excellence in RBM.’ Most of the participants either agree (42.85%) or strongly agree (42.46%). 8.80% (45) of participants neither agree nor disagree. 11 (2.15%) and 19 (3.17%) disagree and strongly disagree respectively.

 

Figure 5

‘What according to you are the parameters for enabling technologies for successful implementation of RBM model?’ was the next question asked. 327 participants (63.99%) said aggregation of data from various systems, 324 participants (63.40%) said flexibility in accommodating data in various formats, 305 participants (59.68%) said standardization of data.

 

The next question for the participants was what according to them are the elements ideal to RBM technology. 57.92% (296) of the participants said a holistic system is ideal. 67.90% (347) of the participants said rapid identification and the majority (461) agreed to risk identification and reporting (90.21%).

 

Figure  6

The participants were asked what are some of the challenges faced while developing the systems. 48.72% (249) of the participants said storage of data in multiple databases. 20.54% (105) of participants said access or visibility to data to numerous functional groups, 33.07% (169) of the participants said that organizations vary across several key factors, such as therapeutic area focus, standard operating procedures, and other associated organizational risks.

 

‘What are the common challenges faced in implementing the systems?’, was the next question posed to the participants. 62.03% (317) of the participants said one of the most common challenges is that data resides in several clinical systems. 52.64% (269) of the participants said having different operating models across the industry. 47.35% (242) of the participants said different technology solutions exist that drive the same process.

 

Discussion:

The primary objective of implementing risk-based monitoring model is assessing risk before actually it occurs, to fulfil this objective the system developed must be able to Identify trends, patterns and outliers, Track issue management, Provide user-friendly administration screens to administer risk indicators, Calculate and display simple and easy to understand performance scores, Include global training plan, Include user interface to manage access and security at multiple levels, Include an audit trail, Store output from the risk assessment in a centralized location, Remain adaptable as experience  with RBM grows although the survey data agrees to all the expectation it disagrees that system should be able to calculate and display simple and easy to understand performance scores. ^[11]

 

To develop a successful system, the system should have the following characteristic as per the literature and the survey data  are: a source system should–agnostic data model that supports the risk indicators, a data load engine that allows for staging of source data from multiple sources, a mapping engine which allows the source data to be mapped to an internal data warehouse that supports RBM, framework to load unstructured data in combination with structured data, framework to segregate areas for exploration and standard reporting, The framework should allow for both incremental and cumulative data and the framework should allow for creation of data snapshots to support the traceability of decisions made during the trial majority survey data indicates that system must have a mapping engine which allows the source data to be mapped to an internal data warehouse that supports RBM, framework to load unstructured data in combination with structured data, The framework should allow for creation of data snapshots to support the traceability of decisions made during the course of the trial and a source system should–agnostic data model that supports the risk indicators. ^[11]  The common challenges faced in developing and implementing the RBM system as per the literature and the survey would include Storage of data in multiple data bases, Access/Visibility to data to numerous functional groups, Organizations vary across several key factors, such as therapeutic area focus, standard operating procedures, and other associated organizational risks, Risk indicators with different weightage needs algorithms for aggregated data have to be constructed to meet these needs while the common challenges faced by people using this systems based on the literature and the survey   includes the frequency of system refresh and the understanding of what is considered sufficient or ‘‘real time’’, Determination of how to present data, the amount of data to be displayed on screen, the level of detail needed and the adequacy of actions user must to take to address and resolve the identified. ^[11]

 

The ideal technology must be able to provide drill-down capability to review and analyse source data, produce graphical/visual representations of analyses, Filter on subsets within various analysis representations, allow for generation of text-, CSV-, or PDF-based reports, Allow for dynamic redefinition and modifications to new risk indicators, thresholds, and alerts, Possess customizable risk indicator algorithms as needed for each unique protocol, work seamlessly with third-party systems if trials are externally sourced, Provide functionality to send and receive relevant information, such as alerts and recommended actions to monitors, Possess templates for core visualizations and analytics, Allow for reusability of a core or standard set of analytics across trials, Allow for creation of trial-specific visualizations and analytics, Allow for creation of trial-specific visualizations and analytics. The analysed data is indicative that RBM systems can be successfully implemented if they are able to aggregate data from various system, standardization of data, flexibility in accommodating data in various formats, trend reporting and track issues and resolution. ^[11]Both the literature and the survey data indicates that the front end configuration of the system must be able to assess study risk parameters, add/remove specific risk indicators at various study levels, having risk indicator scoring weights/thresholds to further refine risk level, role-based application of weights and thresholds at the various study levels, capability of top-down application of weights/thresholds to lower study levels, alert distribution and frequency at various study levels, ability to simulate scenarios with different parameters and perform ad hoc calculations. ^[11]

 

Additionally, the systems must facilitate easy adoption and implementation. For systems to facilitate easy adoption and implementation the system must be able to have shorter implementation timelines, easy installation and process development, simplified deployment, easy-to-run pilot projects and flexibility to make changes in the system based on the lessons learned, and interactive trainings for the staff. ^[12] The technology system must have a flexible yet robust infrastructure and allow for gradual adaptation, both for number of trials and therapeutic areas and for the number of risk indicators ^[9]

 

The literature and survey data indicated some critical competencies that led to successful RBM implementation which included expertise in clinical development concepts, critical thinking of the team members, thorough knowledge of clinical operations and data management, good communication skills and technical skills, and ability of team members to use the available technologies correctly. Implementing RBM successfully will not ensure that the RBM is used effectively the literature and survey showed effective use of RBM will need personnel’s working on the RBM to agree on common terminology, define success criteria, create a RBM process map, define and categorize site risk signals.^{[13], [14]}

 

Additionally, literature and survey suggest having an in-house expert IT team and clinical operations will prove to be a boon in data analysis using the RBM technologies.^[15] The survey tried to touch base with all the aspects of RBM right from developing RBM systems and their implementation and we see that the survey data and the available published literature runs parallel.

 

Conclusion:

While we all speak of implementing the RBM technology we must all understand that the ideal RBM technology must be holistic in nature and should be able to provide role-based access to the system. As well as to develop and successfully run the RBM tools in the organization we need to have a strong and dynamic internal IT team to support data analytics. The organizations must adopt a system that is able to assess baseline risk, identify and review risk ongoing basis and understand the monitoring requirements. As we accumulate experience working with RBM, we require our systems to remain adaptable to new requirements along with being able to identify trends, patterns, and outliers. As equipped as the RBM systems would be, we still might not be able to replace the value of people in the conduct of the trials.

 

References

1. Brosteanu et al. Risk Analysis and Risk Adapted On-Site Monitoring in Noncommercial Clinical Trials. Clin Trials. 6: 585-595 (2009). 20 Tantsyura et al. Risk-Based Source Data Verification Approaches: Pros and Cons. Drug Inf J. 44: 745-756 (2010).

2.https://www.fda.gov/downloads/Drugs/Guidances/UCM269919.pdf ; cited on 21 Jan 2021

3.http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2013/11/WC500155491.pdf; cited on 21 Jan 2021

4.A.Ghone, The Risk Based Monitoring Plan, 30 Oct 2017, applied clinical trials, available on https://www.appliedclinicaltrialsonline.com/view/risk-based-monitoring-plan, cited on 15 Sep 2020.

5.Morrison et al. Monitoring the Quality of Conduct of Clinical Trials: A Survey of Current Practices. Clin Trials.

6.Lessons Learned from An RBM Implementation; available on https://www.clinicalleader.com/doc/lessons-learned-from-an-rbm-implementation-0001 3/3 cited on 10 Oct 2020

7.7 Reasons You’re Not Ready For Risk Based Monitoring; available on https://www.clinicalleader.com/doc/reasons-you-re-not-ready-for-risk-based-monitoring-0001 2/3; cited on 10 Nov 2020

8.Barnes S, Katta N, Sanford N, Staigers T, Verish T. Technology Considerations to Enable the Risk-Based Monitoring Methodology. Ther Innov Regul Sci. 2014;48(5):536-545. doi:10.1177/2168479014546336 accessed on 24Jan2022

9.Oleson, Kimberly & Fay, Margaret. (2013). Fay MF and Oleson K. A Rationale for a Risk-Based Approach to Clinical Trial Conduct and Monitoring Practices. The Monitor. 2013;12: 9-14. The Monitor. 12. 9-14. Cited on 17 Oct 2021

10.Koreith K. 2011. The high cost and questionable impact of 100% SDV. CenterWatch Monthly 18:1, 15–7

11.Barnes S, Katta N, Sanford N, Staigers T, Verish T. Technology considerations to enable the risk-based monitoring methodology. Therapeutic Innovation & Regulatory Science. 2014;48: 536-545.

12.Rosenberg MJ. Key Considerations in the Transition to Risk-Based Monitoring. Therapeutic Innovation & Regulatory Science. 2014;48(4):428-435. doi:10.1177/2168479013519631; cited on 19Jul2020

13.Wilson B, Provencher T, Gough J, et al. Defining a Central Monitoring Capability: Sharing the Experience of TransCelerate BioPharma's Approach, Part 1. Ther Innov Regul Sci. 2014;48(5):529-535. doi:10.1177/2168479014546335; cited on 29Sep2020

14.The Foundation of Risk-Based Monitoring Success available on www.perkinelmer.com; cited on 02Oct2020

15.Ed Miseta, Clinical Insights: "RBM Is The Biggest Game Changer Since EDC"; 4 Sep 2015; available on https://www.clinicalleader.com/doc/clinical-insights-rbm-is-the-biggest-game-changer-since-edc-0001 cited on 13 Oct 2020.