How Forward Deployed Engineering is done at Ramp — Leo Mehr

Awesome. Thank you, guys. Awesome. It's great to meet everyone. I mean, I hope that after the talk, you know, if you want to come out and we can chat, we'd love to. Cool. So yeah, today my goal is to share with you guys the two most important principles from what we learned doing FDE at Ramp.

So just, yeah, briefly a little bit about myself. Yeah, I'm a director of engineering at Ramp. I joined the company two and a half years ago when it was just, you know, FDE was just two engineers at the time.

And today my org is about 30 engineers across Forward Deployed, Developer API, and our new AI services business. So I know this is kind of a running theme, but like no one knows what FDE is, so I'm just going to spend a moment on that.

So yeah, I actually kind of like this meme. To me it's kind of funny.

But I actually think it's like totally wrong. I don't see this as the actual, like true form of what FDE is. I don't see it as like the final evolution or like boss mode of technical go-to-market roles. Now, this might be true at some companies, but at least at Ramp it's a little bit different.

So FDE at Ramp, we live within the engineering organization and our goal is to help Ramp win up market. So with that in mind, what we do is we basically work on the core product and our new agentic features and make them work really well for our largest enterprise customers.

So that's just a little bit of intro context. I want to dig in. And today, like I said, there's just two things I'm going to share with you. Literally two things. Very easy talk.

And these are the principles that I would say have really guided us and I would say probably the two most important things that we have. Always be scoping and scale with tokens.

So let's get started with the first one. On scoping. So I would say there's this thing where like many people think that as an FDE, your job is to just say yes to the customer. But that's wrong. If you were just to say yes, you know, instead of like beautiful Waymos that we have driving us around in San Francisco, you'd have something like this, you know, horses with like rockets strapped to their legs.

And the point is you want to help the customer be successful. You want to try to figure out a way to say yes, but you actually want to deliver good software. You need to build theright thing so you don't just endlessly say yes to people.

And

I do want to share an example of something that I would say happens somewhat regularly in one form or another at Ramp. So it's Friday night and an enterprise sales rep comes to us with anurgent request that this super important strategic logo is only going to close if we build out an SAP S4 HANA integration.

And I think that the default engineering reflex is like, shit, like what are the SAP API docs? Like where do I find them and how do I build this integration? But what a well-trained FDE would do is like pause for a second and say, okay, first of all, like what's driving theurgency here?

Like one thing I've seen is I've seen sales reps who like go kind of crazy because it's like the end of the quarter and they're trying to hit their quota and close the deal and not because the customer is the one driving theurgency.

So that's like one example. But you know, as an FDE, you're asking tons of questions to gather context about what's important and what actually is theright thing to build. And so you might ask like, who's using this integration?

Have we exhausted all the different workarounds? Is there something manual that we can do in the meantime? Does the customer have like technical resources? Can they hit our APIs such that we don't have to build this thing? But I'd say the most important thing that an FDE does is also looks beyond this one request and looks at the other prospects that are coming down the pipeline and other customers to see if anyone else would benefit from this as well.

And the point is that by gaining all this context, you can do a better job of building theright thing. So I want to share another story that was really, really painful for us in the early days. We had this large enterprise customer and they needed this reimbursement feature on mobile.

Unfortunately, our mobile team was totally swamped. We basically just had to roll up our sleeves as FDEs and just find out a way to get things done. And we had two of the engineers on the team just like learn how to do iOS and Android development.

And it was awesome. We were super excited. We're like, okay, we're going to ship this feature. It's going to be so good. Like hell yeah. So we grinded for a couple of weeks, got the feature done on both platforms, and we go to the customer and we're like, awesome.

Like can you send us your list of beta users for Android? And that's when they told us

they only, they require, they mandate all of their employees to use iOS devices.

So we were like, what the fuck? Like not to the customer, you know, just internally. But like obviously it was super disappointing for us because we'd put all this effort in. And so it was a big lesson for us to remember the importance of scoping.

Even some of the most basic assumptions, like which mobile platform you build on, it's super important to validate them and thus kind of emphasizes the importance of scoping upfront. Now, okay, so let's say that you and your team have become masters of scoping.

You're amazing. In today's world, this is not enough.

So unless you are scaling with model capabilities, you are going to fall behind. Now, I'm not going to belabor this point too much. I think like every talk in this conference is probably some flavor of this, but like the point is that we basically have to reinvent our jobs constantly now.

So whatever work we are doing today, you know, for the most part, it's knowledge work. We have to figure out how to have models and agents do it for us. And so that brings me to the second half of this talk and the other point that I want to convey today, which is all of us have to figure out how to scale with tokens.

And the way that I interpret scaling with tokens for FDE is take a look at the whole life cycle of what an FDE does. From gathering context to scoping out a request to writing out a spec and then implementing the feature, each stage of that pipeline can be replaced with agents.

And at first, it seems kind of daunting. You're like, how are you going to go and approach and like solve that? But if you break the problem down and then make progress on it, it's actually pretty tractable. And so I'll share with you guys one example of something that we've done at Ramp.

So we have this internal Slack channel called FDE requests. And this is where account managers, solutions, sales reps will post whenever there is a blocker for a prospect or customer that's large enough, basically. And so we get these requests.

In this case, actually, one of the CSMs on our team, Greg, posted here. And if you were to, it's actually a Notion workflow. If any of you work at Notion, by the way, thank you. We like use Notion so much.

If you were to click open in Notion, you'd see like a pretty long request that has all the details of what exactly it is. And the problem is there's a super high variance. Like some people will submit like really detailed, good requests from the customer and others are just going to submit like one line like we need, you know, we need this SAP integration.

And before, what would happen is we would have FDEs manually kind of go through this request. We'd read the whole thing, understand it, figure out what exists in the product, do a bunch of back and forth with the customer.

And this is like exactly what the first half of the talk was about. Always be scoping. You know, we would spend a lot of time really digging in and validating what exactly was, you know, absolutely necessary. And so you can see here what we did then was we basically used Notion agents to build a V1, which literally just took the request and asked a couple of questions.

That was it. And

after, it was kind of astonishing. Literally, after a couple of weeks, we found that it was like saving us a lot of time because first of all, like the latency of replies went from like hours or days to like, you know, seconds.

And immediately, like the account reps, the account managers, the reps would start kind of engaging with this agent. And one of the things that we did was because it went so well, this is actually a more recent iteration of it.

It's very cute. You know, the little penguin actually helps make it seem a little more friendly and approachable. And what it does is it actually goes and does several rounds of back and forth questioning with the submitter until it deems that it's ready to create a spec, basically.

And it's actually been incredible how helpful this has been for us. I would say it's probably saved us like a large percentage, I don't know, 20% of the time that we'd spend on scoping out these requests. So, you know, this is a great example.

For us, this has been really helpful. I'm super excited about this. It's going to help automate a lot of the work that we've been doing manually. But it's really just the first stage of this pipeline that I was alluding to.

So if you look at the first part here, like we've been able to make some progress on it. The last step as well, going from a well-shaped spec to like a working product, obviously like frontier models can like one-shot medium-sized features.

And so the last part is also a lot easier for us. It's this middle part that I would say is super like gnarly and like unformed and difficult. And I'm really excited about our team kind of investing a lot more and spending a lot more of our time just like building out this factory, building out agents to replace each one of these steps.

And the thing is, if you look at, if I were to say 6 to 12 months from now, like what does FDE at Ramp look like? Like these are the sorts of applied AI problems that we're going to be spending all of our time on, I think.

Like, you know, making sure that the agent harness that's running each of those steps is running super smoothly. Making sure that the output quality of each of the outputs of the pipeline is actually good. That, you know, with evals, with rubrics, with human feedback.

And there's, of course, like one of the biggest challenges, which is getting your agent theright context. When you're making the LLM call, ensuring that it has theright context. So there's like a lot of historical data, data about the product.

Imagine like all the knowledge that a product manager has in their head about their product. Like how do you get that into an agent? Like Notion docs and all your existing knowledge base and help articles only give you so much of that.

Yeah, skills, memories, tools. I could go on for a bit, but ultimately, the most important thing here is that as an FDE, we still have the responsibility of taste and judgment over the final output. So that's going to be like the underlying kind of through line.

Okay. So let's say that you've done an amazing job building out this factory. But the problem is, and to tie this to the first half of the talk, if you don't do a good job of scoping out requests or building upon the principles of scoping things well, you're going to get a token maxing slop cannon.

And so the whole point is that you have to do these both because the other way around is actually quite bad as well. If you are, you know, amazing at scoping, but don't invest in building out this, you know, agent factory,

you know, it's going to be over for you. Like your agent native competitors are just going to overtake you and outcompete. And so that's why

in the end here, I want to close with the most important thing is that if you have both of these, it can set you up for success in the future. Always be scoping and scaling with tokens. The future of FDE needs both.

That's all. Thank you, guys.