CeeBee - Conversational AI for Clinical Trial Recruitment & Retention

CeeBee - Conversational AI for Clinical Trial Recruitment & Retention:

Authors:?Rajiv Marwah &?Ritesh Kumar:

In a rather animated conversation with a peer of mine, I was told that “Chatbots or Virtual Assistants have no role to play in improving Clinical Trial patient recruitment or patient retention”. For instance, we have all experienced virtual assistants as consumers with airlines, telecom, and banks. Each of us could admit that we have a varied response to chatbots, as they can provide both positive and negative experiences. After over six months of experimentation and multiple implementations, I am convinced that we should not ignore the power of smart Chatbots. It is one more step in our journey to digitalize clinical trials, an undeniable trend and one that is gaining momentum. There are clear benefits of implementing chatbots for both patient recruitment and patient retention. This tool allows for greater patient centricity which is what most stakeholders desire. One positive of Covid has been the increasing digitization of clinical trials and as this trend continues additional productivity tools will be required.

Before proceeding, it might be of value to list the elements of Conversational AI to allow for increased clarity. The terms Conversational AI, Chatbot, and Virtual/digital assistant are all used interchangeably and refer to the same functionality.

The objective is to simulate human-like conversations with a friendly user interface that can be used across the IoT. When implemented properly, it can result in an improved patient and provider experience, while being available?24X7. To reduce conversational friction, the software must be contextual, include current information, and provide personalized responses. This will be possible if the software can do both intent & entity recognition.

Intent recognition?is the ability to interpret what the user is asking so that an appropriate response can be provided.?Entity recognition, in our use case, is the ability to recognize and effectively interpret medical/clinical entities. These include eligibility criteria, procedures, conditions, drugs, and other clinical trial variables. The accuracy and number of medical entities recognized will directly impact the quality of responses. The AI and Natural Language Processing (NLP) engine under the hood of a smart virtual assistant enables successful interactions. The associated Machine Learning (ML) engine will learn continuously to improve future interactions. To deal with exceptions or when the software is unable to understand patient intent, it’s important to be able to seamlessly transfer the conversation to a human agent. At this point, a human agent could be introduced through video or email. The human agent will be shared an email with a copy of the chat conversation details for a future appointment.

Given the above, let’s review how these concepts can be implemented in clinical trials as they relate to both patient recruitment and retention. The conversion and retention rates with the deployment of Conversational AI technology should see an improvement of at least?25%

Recruitment: The Chatbot can be visualized in distinct sections:

• Enrollment: The NCTID for the trial is ingested, parsed for inclusion and exclusion criteria and standard information (e.g. demographics and reimbursement)
• Potential patients can interact with the chatbot?24×7?to determine eligibility, and obtain general and clinical answers about the trial, based on available public information through?clinicaltrials.gov?or other generally available information. This information is shared in a user-friendly manner and is easier for the patient to understand. This allows for several patient-centric interactions and digital nudges to reduce the enrolment time.
• The digital interface will encourage the participation of a younger patient population.

Retention: of enrolled patients

• Chatbot can provide sentiment analysis and data-driven digital interventions to reduce drop-off rates.
• The chatbot can recognize potential adverse reactions and raise an alert and facilitate the scheduling of meetings with clinical research staff. In these cases, the human remains in the loop and the goal is to act as an alert mechanism and at no stage is clinical advice provided by the chatbot
• Data collection for improvement based on patient feedback across a growing number of users.

Regulatory framework:

• Chatbot can be deployed in a secure manner such that there are no privacy or other compliance-related violations
• The chatbot conversations are saved and available for audit, reporting, and monitoring. It is easier to monitor textual information in real-time as compared to voice calls and static web forms.

Deployment:

• The chatbot can be hooked to any page on a website that the customer chooses
• The chatbot can be skinned to align with the customer’s brand
• The chatbot can be deployed for a single or multiple trials
• There are multiple sources of rich data sets which enable higher patient satisfaction it is now possible to access these using the web, API(s), and other technologies
• Chatbots can be scaled up or down based on trial requirements.

Technology:

The Busy CeeBee

Let us understand what makes our busy CeeBee intelligent and context-aware for the clinical trials world. The technology behind the functioning of the virtual assistant is explained in the following section

As we know, clinical studies are often listed on?clinicaltrials.gov. They have the information presented in a specific format, starting with?Study Description?to ending with?More Information. We can easily observe three unique ways in which the text is layered:

1. General FAQ-type questions?with typical answers (study start etc.) from the non-clinical text
2. Domain-specific text?as in the eligibility criteria, rich with clinical entities
3. Text in the tables?like the?Condition-Intervention-Phase?table

The differing layers provide a unique challenge to the CeeBee bot to correctly understand the?context?in which the question is asked, identify the?relevant candidate passage or table?where the answer might be, and then generate the?correct answer.

The first layer task

Our CeeBee bot uses a multi-task transformer architecture that can handle both intent classification and entity recognition together. It thus starts by capturing the?intent?of a user’s query and the?entity?within the query. While intents allow it to understand the motivation behind a user’s input, entities allow it to gather specific pieces of information that the user may have mentioned. Next, we define the user’s?utterances?for each intent which are the automated responses from the bot. But wait! The questions from the user could come in any variation. The?Question Generator?helps CeeBee Bot understand the multiple different ways in which a question may be asked by the user.

The second layer task

The eligibility criteria section of the clinical study has text rich in clinical entities. Extremely domain-centric, for the bot to understand the text, clinical texts must be parsed correctly, and clinical entities are extracted and overlaid on the text embeddings. For example, consider an inclusion criterion under the eligibility section:

“At least one lesion ≥ 1.5 cm that is seen on standard imaging (e.g. CT, MRI, mammogram, ultrasound, FDG-PET/CT).”

For the CeeBee bot to understand it, we need to mine these entity pairs:

{“condition”: ”lesion”} with attributes {“value1”: “at least one”}, {“value2” : “>=1.5 cm”}

{“procedure”: “CT”} etc

Candidate lines are fed along with the user’s query into the deep learning models for predicting the most probable answer span. We use BiLSTM, BERT, and RoBerta to generate the answer span. Our model uses the candidate lines to construct a feature vector which is then encoded. Similarly, question encoding is also created. Finally, the model uses candidate line encoding and question encoding to predict the correct offset for the answer span.

The third layer task

CeeBee bot has an in-built retriever that identifies if the section in the clinical study is text or table type. Tables need to be dealt little differently than the text. They can be thought of as multiple entity-entity relations stacked together as columns of the table. The answer to a user’s query could be in table or text or both. The CeeBee bot takes the answers from different layers and joins them together to create a coherent response.

CeeBee’s performance

Enriched with more than 30 clinical entities and their relationships, the CeeBee bot is an innovative solution in its domain with a near-real response time of less than a fraction of a second. Interspersed with?thoughtful utterances, it engages very well with the patients and answers their queries effectively. It can?fast-track the recruitment by 2-3X.

How do you engage with patients during pre-recruitment and later? Please share your thoughts in the comments and do stay tuned for the second part of this article which would discuss the tasks our CeeBee bot accomplished while the study is ON!

Authors:?Rajiv Marwah &?Ritesh Kumar:

We welcome you to join us for a webinar on this topic on the 4th of August 2022 at 11am EST. Please register at https://circlebase.com/webinars#cta