Can Oncology Workflows Run Without Human Touch? - Anant Shankhdhar, Risa Labs

Hi everyone, uh, my name is Anant Shankhdhar. I'm an AI engineer at Risa, and I'll be talking about oncoli— automating oncology workflows from end to end.

So, at Risa, we are automating various workflows in oncology. One of such workflows is prior authorizations, where we file for authorizations for drugs for cancer patients. So, uh, I'll give a brief overview about the workflow before we move further in the call.

So, the first step is that we intake the orders that we get on a daily basis. The second step is we verify whether the patient is actually very, uh, eligible for getting the drugs, based on the amount that they have in insurance available with them.

This is called eligibility and benefits verification. Next, we determine what all drugs in the— for the patient require authorization. So, a drug can basically fall into one of the following pathways. So, one is NAR, which is no auth required.

Basically, the drug does not require authorization. Another can be that the authorization has already happened for the drug for a time period, which is called auth on file. And the third is auth required, which means that the authorization needs to be performed for this drug.

So, yeah, this is the workflow. Uh, I was— so, although our bots run and perform these steps, finally a human review is required, you know, before submitting all these orders. I was ta- tasked with the, uh, with the problem to run, uh, some of these orders without any sort of human touch, uh, directly toward submission.

Which means I need to, uh, confidently identify which all orders can be proceeded without any human verification, and then build the entire flow for them as well. So, confidence is a key metric that we were working towards. So, yeah, uh, how we did this was using four agents, namely EV Agent, Auth Agent, Necessity Agent, and Submission Agent.

EV Agent is for eligibility and business verification. So, basically fetching the patient data and determining whether it is, uh, fine for moving forward. Auth Agent determines the type of the status of the drug, whether authorization is required or not.

Uh, and the Necessity Agent is the clinical brain of the system. So, for the cases where authorization is required, we determine whether it has to be done on— whether it isright for the patient, based on his vitals, or not.

And finally we move toward submission.

So, uh, starting with the first step, how do we do this without any human in the loop? So, one of the problems that we have here is that insurance details, insurance documents, everything is scattered across dozens of portals, APIs, and documents, and it is difficult to find that information at one place.

So, in order to— in order to start with our pipeline, uh, we need to get information from various portals as well as APIs. In order to do so, we built a unified service that connects to different payer sources and, uh, gives the output in a normalized, uniform format, which we can use for, uh, processing further.

And we also added a deterministic decision engine to flag the cases which would not move forward, thus, uh, deterministically, uh, fixing some of the orders without any human in the loop. So, here's how it works. So, we have a coverage orchestrator which determines whether, uh, the patient will go for the API or the RPA path.

RPA is basically the automation. Uh, we, uh, perform the actions for the RPA or call the API and get the output in a fixed coverage result format, which we then pass to our deterministic engine to determine whether the coverage is active or not.

If it is, we move it further in the PHN, else we stop thatright there. Now, one of the problems we saw here was that, uh, if we have to build this, then we will need to make custom integrations for different sorts of portals, which is not, not a very scalable, uh, process.

But how we tackled it was that we used LLMs in the loop. So, first of all, we made a huge repository of custom actions as well as popular actions that are required in, uh, to build RPAs. Then we built a LLM-based config generation which performs these actions and builds the config for a, uh, for a portal to run on, uh, which, uh, fix— which reduces our development time significantly.

And finally, uh, these, uh, automations are fragile, so it may happen that they break during the runtime. For this, we have a self-healing loop which identifies these cases during production hours and then mitigates them. So, thus preventing any sort of failures.

So, yeah, that is the first step. Uh, once we have done this, we move further in our chain. So, as we can see our diagram, the graph, uh, the patient data is fetched, we perform the eligibility verification, we see if it is fine or not.

If it failed, we stop there, else we move forward. So, yeah, moving forward, uh, the next step was that, uh, was to see, uh, what all drugs require authorization. So, if I have to, uh, remove humans from the loop, one simple case that I saw was that, uh, if a drug has already been authorized, that is, the authorization is on file, or if the authorization is not required, then we can partly solve the order.

That is, that we do not need human oversight on that part of the order. Uh, so, yeah.

So, as the first step, we built a simple LLM extraction pipeline which takes in patients' notes, performs LLMs extraction, and categorizes the drugs into these types, the default being that authorization is required. However, uh, yeah, so we thought that this would work, but we faced some issues here as well.

So, one popular issue was that the notes that we were using did not have enough data, which means that it was performing some, some errors. Uh, also, LLM extraction is sort of an indis— uh, indeterministic process. That means that whatever outputs we get from here cannot be blindly trusted.

So, we still need a human to review all these things. It might improve the efficiency, but it will not eliminate the human. So, in order to do so, uh, we thought that what if I add more evidence to it.

So, if the LLM is saying that a drug is not, uh, no auth required, then I have some information backing that. I can say that confidently, or for the auth on file cases. In order to, uh, yeah. So, in order to do so, we auth— we leveraged two other data sources.

So, one was the authorization letters. So, this was basically the previous information. So, across the previous runs that a given patient has, they have authorization letters available, which shows that these drugs were actually authorized. So, now, instead of just having one source, I have two sources that give me the same information, and wherever they conquer, I can say with confidence that this drug has already been authorized.

Second is the NAR case, the no auth required case. So, here, uh, we found some other resources wherein, from the portals or from some documents, we could find out that on a monthly basis, which are the drugs that the payer does not work for, the insurance does not work for, basically.

So, uh, we used that information to build a payer rule knowledge base, which is basically a SQL database, uh, which was made from portal checks as well as LLM extractions. So, we use all this information, we reconcile the evidence, and we extract the auth statuses, with a higher confidence.

So, if from this pipeline we get whether a drug is not authorized, not— does not require authorization, or is already authorized, we have a solid proof behind it, uh, and a higher confidence, uh, to say that this is the case.

And of course, all of this is configurable. So, uh, in order— if some mishap happens, we always have a plug over the cases. So, yeah. Uh, another good thing that happened from here is that we noticed that certain orders were completely eliminated.

We could eliminate certain orders completely from these two types of drugs. Because it may happen that an order does not actually require authorization at all. The drugs are either, uh, not requiring authorization or are actually authorized from before.

So, this enabled us to no-touch on a certain set of orders that we had. Yeah. Uh, this slide gives a big, uh, small overview on how we built the payer rule knowledge base for the NAR cases. So, uh, we had the documents which had this information on a time-wise scale.

We built an LLM extraction which uses LLMs, uh, which, uh, is configurable, so we can configure it on different types of documents, and we can perform the extraction, and we can get the drug-wise constraint that for this payer, this is the— this is how it will be treated for a period of time.

Another thing is that we leveraged some historical information. That is, that we know that a certain organization, uh, treats this drug in a certain sort of way. So, we stored that information and used that as well. Apart from this, we also used some regular portal checks to get this information.

So, yeah. Uh, performing this moved my— moved our needle forward a bit. So, now we had these deterministic checks along with our initial eligibility verification. So, whatever were not verifiable were flagged out before, and whatever moved forward, we could determine these statuses for the drugs, and we were able to eliminate some of the orders from here.

So, now that we've solved for two types of drugs, let's move forward.

So, the next problem that we faced here was that some decisions do actually need clinical reasoning. So, so far, the drugs that we were, uh, we shifted to no-touch were the ones where the information was di- directly available, or indirectly available by another source.

But we did not need to perform any sort of reasoning over here, or any sort of question-answer. However, for the drugs that are actually auth required, we need to, uh, check whether the patient is actually eligible for them, and also sup— uh, give sort of supportive evidence from where we are answering this infor— uh, we are giving this information.

So, in order to do this, we built our third agent, which is the Medical Necessity Agent. So, the Medical Necessity Agent answers simple and complex clinical questions per patient, uh, and attaches confidence score to any answer. So, we escalate only the ones that actually need a clinician.

Uh, so we did a lot of work on this space. We actually have another— uh, have a publication here as well, which I have mentioned. So, how this agent works is that, uh, we have a new order, and for every order, we read three things.

So, we read the patient notes, we read the policy criteria. So, the pol— uh, policy criteria is basically the criteria that needs to be met in order for this drug to be processed further. Uh, we— uh, from this criteria, we query the patient medical graph, which is a graph of, uh, biomarkers that are extracted for a patient, and we determine what all biomarkers exist and what is the condition of this patient from here.

And then we use this thing— these inf— this information and pass it to an LLM to get an answered questionnaire with all the supportive and contradictory facts. So, uh, yeah. And, uh, if we have the relevant information or we are able to determine this with a higher confidence, we move this forward.

And for the cases we do not have this enough information, uh, we move— keep that for human escalation. So, yeah. Uh, this is the next step. Uh, so after, uh, this Medical Necessity Validation, uh, we moved from, uh, the first step where we fetched the data, we did the eligibility verification, we flagged the orders that were not eligible, moved the rest forward, extracted the no auth needed and already authorized letters, uh, got the— got, uh, the cases where we can move that without any human intervention, and then we did the Medical Necessity Evaluation.

Uh, whatever was not met was also moved to review. So, yeah. Finally, we assemble all this information and then we submit it back to the payers. So, this is where our Submission Agent comes into place. The Submission Agent is pretty much similar as the EV, VV Agent.

We have customized integrations for every sort of payer that we have, uh, which is built using an LLM-driven config as well as our repository of tools.

So, yeah. This is the entire graph, uh, that we got through the entire journey. Uh, so the green part shows the cases that we are going without human touch. So, whenever eligibility verification passes, uh, then, uh, enough information is found to not— to prevent the drug from going for further Medical Necessity Evaluation, then the, uh, Medical Necessity Evaluation and the submission.

So, yeah. Finally, I'd like to mention that, uh, these drugs are also— although built initially for this case, are being used across multiple workflows. Uh, we have extended the functionality to a more general one, uh, where the Medical Necessity Agent can answer questions specific to any sort of workflow for the patient.

Uh, similarly for the other agents as well. So, yeah. Uh, as the conclusion, I would say the no-touch is growing on the share of every order. Uh, so we started with deterministic checks, uh, agents only for the rules that— where what rules can't decide.

Uh, we used multi-source of evidence, uh, to beat the single source of evidence to add confidence to our cases. Uh, we added self-healing in order to scale our RPAs, and finally we added the reasoning layer in order to deal with the cases where actually authori— uh, auth— authorization is required.

So, yeah. Thanks for the, uh, yeah. Thanks for your time. Uh, yeah. Bye-bye.