How to build an AI-Native Health Company — Dan Feng, Maven Clinic

It's time. We can get it started. I'm Dan, I'm from Maven Clinic. Today we'll share the experience: how we transitioned from a traditional technology company to an AI-native company. Before I start it, I would like to do a little exercise.

Raise your hand if you think you are already an AI-native company. Okay, we saw a few. Raise your hand if you thought about it but haven't started the journey yet. Okay, we saw a few. That means most of us is in between.

Hopefully this talk can help you with that one. So, Maven Clinic is the largest digital health platform. We are focused on women and their families. So we specialize in, like, maternity, fertility, parenting, menopause. We started our AI journey just two years back.

And at this moment we built something called Maven Intelligence. It's our orchestration layer across all our product to enable AI for everybody in this company and for our clients. So, AI is here and improving every day. I think adopting it is not optional.

Even you choose not to, your competitors will do. This is a code I heard, like, a couple years back. I would like to share here again. Like, tractors aren't to replace farmers, but the farmers who can operate the tractor will replace the ones who cannot.

Hopefully everybody here will become farmers who can operate their tractors. That's the goal. So first of all, I don't think there's one single definition of what it means by AI native. And more importantly, there's no predefined playbook you can just follow and bingo, you become AI native.

For us, it really comes down to three parts. One is internally, we adopt AI tools whenever it's possible. It can be as simple as, like, generating your daily summary, managing your meeting, creating a Jira task. Anything you need to do today manually, you should think about to say, can I use AI to do it?

Whenever you want to ask other people to do something for you, you should say, I can use AI to do it. A lot of leaders, in fact, today at Maven include our CIOs, COOs. They use AI tools to solve those tasks by themselves now, instead of delegating to other people.

That's for internally. Externally, we want to build AI into our product. We achieved two goals there. One is really focused on improving our user experience. Second, maybe help us reduce our operational cost. Like, AI-based, like, chatbot is a really good example.

It's 24/7, always available, can help our address issues, help our customers instantly. It's way better and cheaper compared to human agents. Thirdly, and I think it's more important, we need to think about our culture, process, the way we work, how we can change it so we can maximize what AI offers for us.

I will touch it more on the following slides. So, when we come to adopting new technologies, there's always, like, three groups of users. One, there's some early adopters,right? For them, we don't need to do too much. The only thing we need to do is enable the tools for them and encourage them to share what they learned with the company.

And what we need to really focus on is the one in the middle. That's the majority. We should build a shared AI infrastructure for them, build easy-to-use tools for them, just make the adoption as seamlessly as possible. More importantly, we should really listen to them, get feedback, consistently improving.

For example, last year, most of folks at Maven, they are using Cursor. This year, a lot of them switched to Cloud Code. For us, we need to support both. We need to meet where they are, just make sure they're comfortable to use it.

And for other places, you always have a few slow adopters. They always have concerns, worries for the new technologies. For them, we just should meet where they are, understand what their concern is. But more important, we should be crystal clear with them where the company is heading to.

So, AI is really good at execution if we know what we want to do. So let's change how we should hire new people and how should we reward it. We used to, the way we used to work is we have a senior engineer assess the problem, come up with a solution, and dedicate to other engineers for implementation.

So we can work in parallel and be faster. But these days, we find the nest engineers who would like to dedicate implementation work to other people. Because they already figured out how to solve the problem, they just use AI to solve it instantly.

Dedicating to other people means more overheads and way less efficient. That also means, like, when you have new people, you want to make sure they can solve the problem independently. They pretty much have to work as a traditional technician-lead level.

We cannot afford other people dedicating implementation tasks for them. Also, when we hire new people, we should think about what we are looking for. We definitely want to look for somebody genuinely interested in AI. The domain is moving so fast.

We want them to keep learning, also help the team to stay on track. Secondly is, with AI, engineers can do way more than they used to do. The boundaries between PM and engineers are getting blurry. We found, like, engineers who really understand the product, in fact, they can have way more contribution than a traditional engineer who only focuses on the software side.

And this is what we are looking for. And those deep understandings of the system, the ability to handle complicated, ambiguous problems is also very valuable. This is where AI knocks off. When we hire new people, this is also the people we are interested in bringing on board.

For people we bring in, we want to reward them in the proper way. Even in our performance review, we start to ask, okay, what have you done for the AI side? We definitely want to reward people who leverage AI to multiply their impact.

Although this is impactful for everybody and the company.

So now we have theright tools, we get theright tenant in place. And we need to change how we work to maximize the benefit of AI. The way we used to work is to say, okay, we spend weeks, sometimes months, to flesh out the business requirements, finalize the design, and then do the implementation.

Because implementation can be really expensive. If we didn't get the other partright in the beginning, it can be very costly to change it later. But in fact, we never get the same thingright in the beginning anyway for any of our big projects.

With AI, building is super fast. It's probably a couple minutes. You can get it done. Argument is a really expensive one. So we should really think about what's the best way we can work, how can we deliver fast.

It's still okay, you can think about what you want to deliver in one year. You can assume AI models can do anything you want in one year. Based on that one, really dream big to think what you can do in one year.

But it should only serve as inspiring and directional. What we really need to focus on is what we want to deliver in the next two to four weeks,right? What we want to get the PMs and designers is to say, okay, tell me what I need to do in this sprint.

And the engineer will focus on it and get it released at the end of the sprint, if not sooner. Meanwhile, and the PMs, they have time to flesh out the next batch of requirements. If at the end of this sprint they say, no, what we decided two weeks ago is wrong, it's totally okay.

We can switch the gear, get it fixed quickly. That also means, like, we prefer people not to write pages or pages PRD or TDD anymore. We prefer them to write just a short one or two pages. That one is really served as communication.

So we can iterate on it. The really awkward part is mid-term goals. Those, like, three months, six months. It's very hard to plan these days. The reason is, I don't know what AI models will be capable of in three months.

There may be multiple releases already. So we prefer not to focus on this one. But this one can maybe make it not very easy for most of folks who have been in this domain for a long time. Because traditionally, we get used to having quarterly planning, or we plan it for six months.

But it's our job to get used to the new AI era and learn how to work it efficiently.

So I want to talk about coding and software development a little bit more here. AI coding tools are probably the most successful AI applications. And it's really good at implementation. So you probably heard a lot of people say, okay, I have these AI tools.

Now I can even use my phone to implement software and automate every stage. If they feel comfortable doing that, it's totally okay. But you don't have to. What I'm trying to say here is, at Maven, this is our journey, how we adopt those AI tools.

We start with the lowest risk tasks, like starting with writing unit test documentation. Those things are very easy to verify, and the risk is super low. By doing that one, we build confidence. And we start to construct our own rules, scales, and build our guardrails.

And then we push to the whole engineering team and say, now you should use these AI coding tools for all the tasks. When they choose not to do, it's the time we really want to learn and say, why you don't do it.

And at this moment, we pretty much use the AI coding tools to do all our implementation. Engineers really focus on reviewing, architecturing, and evaluation.

So, and with AI coding tools, we are writing so much code these days. Code review becomes really challenging. So for a good engineer, used to, they probably write hundreds of lines of code every day. These days, they can easily write, like, thousands.

If we capable to do the code review as we used to do, we often be able to keep up. We also tried multiple, like, AI coding review tools. It helps a little bit, but we don't feel comfortable 100% relying on them yet.

We still find the feedbacks from our engineers are very, very valuable. That means we need to really change the way we are doing code review to meet where we are now. And a couple things we have done. One is we allow engineers to self-identify with the still-need code review.

If they think this PR is simple enough, I feel very confident, I don't need anybody to take a look. And we are fine with that one. We let them merge, but we still hold them accountable. And if they do want code review, we want them to stay with the best practices.

For example, each PR shouldn't have more than 500 lines of code. Because nobody can do a meaningful code review with the ones that have, like, thousands of lines of code. And we also enabled, like, stacked PR. What it means is that for big features, and engineers can break it into multiple PRs while people review the PRs, and they can keep working on it.

One thing we really want to avoid is the rubber stamp, we call it. It means, like, people submit a code review, you cannot really do anything to it. You just say, blindly approve it. This is the worst case.

We should really avoid. Because that just gives us false confidence. We think we reviewed it, it's good, and we released it. Meanwhile, we should keep working on our AI coding review tools because we are seeing that's the future.

So, at this moment, we use AI tools pretty much assisting on each step of our software development. Our goal is it will be automated the whole life cycle from end to end, from designing, implementation, until it's fully released.

More important, we want the AI tools to be able to monitor the live traffic and be able to catch the issue early and automatically fix it. That's what we are still working on, and we are not there yet.

The last thing I want to touch a little bit for this presentation is about reliability. So what it means is, like, for the traditional software, it does what we implement there. No more, no less. But for the GenAI solutions, hallucination is there.

We cannot ignore it. And completely eliminating them can be very costly. Sometimes it's not necessary either. So the way we should do is really have a holistic solution, even from the beginning. For example, we can start with identifying which failures are acceptable, which ones are not acceptable.

For our AI system, for example, we have the functionality to help our customers to schedule appointments. If we fail one out of 1,000, probably it's okay. I'm not saying it's a good experience. But the users usually can just click the button again, we will reschedule for them.

Probably it's okay. But if we help the user to, like, submit their reimbursement claim, we cannot tolerate a failure. Because if people ask for $200, we issue them $50. Or they ask $50, we give them $200. Each case will cause escalationright away.

For those cases, we have to put in extra stamps. For example, when we receive their receipt, we will use different models to review the same receipt. We only move forward if the results from different models agree with each other.

If we really have trouble to figure it out which one isright, it's okay to tell the customer, say, hey, I will trouble to process your stuff. Do you want us to get you connect to a human agent? We will move from there.

That's accepted solutions. And also, we should have a rigorous process to release our software. For us, we have, like, hundreds of integration tests which pretty much cover all the use cases we know. And we are keep adding to the integration test suite.

And when we run the integration test, not only past once is not good enough anymore. Because the ARM can do different things. So for each test case, we run it many times. We consistently require a high pass rate.

Like, for example, 90% for all the time. And more important, and after we launch the software, we have our auto-evolve system. It carefully evaluates each conversation. We have predefined a lot of rubrics. What we think is good, what is bad.

And then we will generate results. We will review the score. Besides this one, we also have a dedicated group. Their job is manually reviewing those conversations. We will spot-check our conversations. That helps us to say, whether we need to come back to improve our systems, or our rubrics are too strict or too loose, and we need to consistently improve it.

When we launch new features, that's the time we say, not spot-check probably not enough. We really want to review, like, say, 20%. And we can do it. This whole process makes sure we feel really confident we never reach something.

Although we know hallucination is there. That's pretty much all I have for today. And I can stay here to take up questions. And if you have other things, you can reach out to me.