How to run a product team focused on crisis prevention

What I learned from co-founding Lyft's Regulatory Policy product team

Regulatory policy and compliance can be a matter of life or death for a startup. Maybe even for its users. Not complying with laws that regulate your business will get your entire business shut down, and if your work impacts people in real life it could also affect their safety (eg: vehicle inspections and background checks). Getting this right is paramount. It is also incredibly complicated: defining success and prioritizing work is more ambiguous than typical product teams focused on metric impact.

When I joined Lyft, these were the sort of problems I found myself tackling. I and several others felt it was paramount for Lyft to succeed here and founded a team solely dedicated to preventing crises before they happen. This was perfect timing since over the past several years the number of regulations around ridesharing have exponentially grown, making enforcing those rules exponentially more complex. We developed a bespoke framework for executing in the regulatory policy space and drove a ton of impact making sure the risks mentioned above would never happen at Lyft.

In this post I'll outline what I would do if I were to join a new company to work on this problem from scratch. I'll use regulatory compliance as an example, but this exact framework could be leveraged in other ambiguous problem spaces like social media content moderation, data privacy/security, and general engineering infrastructure reliability.

I'll focus on the following themes:

Setting up an engineering team whose only job is proactively keeping the company compliant and building all the necessary platforms for that. And why you should keep this team separate from other areas like growth
Working alongside subject matter experts to figure out how to prioritize work for the team at scale
Having a clear way to measure success with leadership buy-in and why that's critical for both the team's success and individuals' success on the team

Step 1: Aligning incentives for your new team

If you don't get this part right, nothing else will work.

Most teams across tech companies focus on metrics like user acquisition, engagement, retention, and monetization. These types of goals are exactly what you need to create a product people will love and use a ton, but they should not come the expense of other priorities, such as security, integrity, and infrastructure reliability.

For teams like these, anything to prevent a future crisis (compliance or otherwise) is inevitably going to be treated like a box to check rather than something to proactively invest into.

What can go wrong?

The costs of not investing proactively into these areas can be astronomical. For Lyft or Uber, compliance with local regulations is essential to their ability to operate in those cities. For other companies this could look like a massive data breach, foreign election interference, or adding months of extra work to your engineering team's roadmap a year from now due to tech debt.

And while there are no headlines for this one, the insidious impact of slower technological progress due to bogged down engineering infrastructure can compound over time in serious ways. Growth-oriented teams' engineers often have to cut corners today to meet short-term metric goals. Which means in a year or two, when an engineer (often a different one) has to build something new in the same code base, it could take them weeks or months longer to do so. That's always problematic, but would be especially problematic if the new project was something urgent - like meeting a new regulatory requirement due in a couple days that'd otherwise prevent your company from being able to operate one of your major markets!

The answer: a "one goal team"

If you want your company to never run into any compliance problems even as your industry goes through hypergrowth, the only way to ensure that is building a team with exactly that as its sole goal. A "one goal team."

This team shouldn't have to worry about growing users, cutting costs, or improving profits. You have other teams focused on that already. All this team should worry about every day is how to ensure that you can keep up with accelerating compliance requirements and get ahead of potential crises.

Step 2: Helping your new team prioritize work

You now have a team empowered to focus on crisis prevention proactively, unencumbered by incentives to directly contribute to business metrics. Next up is answering the question any new team has to ask: "Where do we start?"

No matter how big or small your company is, you must always face the reality that your team isn't big enough to work on all the things you want to accomplish. Trade-offs exist in every organization. Security, integrity, compliance - these problem spaces are no different. If you have 10 engineers who will work on average 40 hours a week this quarter, that's around 5000 total hours of effort you can allocate towards a roadmap that may very well include tens of thousands of hours worth of projects.

While getting there is still hard of course, prioritizing work is pretty straightforward when the goal is something like "grow number of new users." If you can reliably project the potential impact of each project and how much effort it'll take to execute, you can stack rank everything easily.

This falls apart, however, for compliance. With a list of myriad regulations that are being added or changed across many jurisdictions you operate in, how can you easily figure out which one is the highest priority to work on?

Here are three steps you can follow for even the most ambiguous of problem spaces:

Use "pre-mortems" to qualitatively understand what you're trying to prevent
Use "priority scorecards" to quantify impact for each opportunity (with subject matter experts)
Use "product case law" to iterate on your approach over time

Pre-mortems

A pre-mortem is basically imaging now what things might go wrong in the future, so that you can proactively prevent them. You can read more about pre-mortems here.

There are many ways to leverage pre-mortems. You can think about what might happen if you didn't comply with each forthcoming regulation (ie: getting a large market shut down, bad press, safety concerns, etc.). You can think about how much wasted engineering effort it would take to fix compliance problems later without investing in more scalable infrastructure now. The main thing here is to be holistic and paranoid about what might go wrong if you don't prevent it now.

Especially for an area like compliance, but also for any other ambiguous space like integrity or security, leverage your subject matter expert partner teams for help. Of course for compliance this primarily means working with lawyers to imagine what bad scenarios you want to avoid.

Priority scorecards

The qualitative insights from pre-mortems on their own won't be enough to scale as your team grows. If you want any engineer on the team to easily and precisely pick the highest impact project to work on next without any help, you need to find a way to stack rank your roadmap in a way that compares everything apples-to-apples.

The best way to do this when all the options are ambiguous is to develop a "priority scorecard" for your team.

For compliance, have your legal partners fill out for each project a scorecard spanning a set of factors contributing to how high-priority each project is, calculate an aggregate numerical score for each project, and then stack rank based on the aggregate priority score. Your engineers now just have to pick with project at the top when they're ready to work on their next thing.

You can also tiebreak projects with similar "scores" with a quantitative metric like how much of the platform is impacted by the issue (eg: if you're Instacart, what % of total deliveries would be impacted if you had to shut down all markets affected by the regulation in question)

Product case law

Set up a cross-functional team combining your tech leads and subject matter expert team leads and frequently go over outcomes. I recommend meeting at least several times a month. This is your chance to go over which projects the "priority scorecard" method prioritized for the team, which issues we therefore got around to solving, and which ones we did not end up making time to solve.

If the decisions you made thus far given the above framework don't feel right to the team - for example, you didn't address an issue that you all think you should have because the scorecard rated it too low - update how the scorecard works for future projects. Think of this as first establishing the spirit of the law (initial framework) for the team, and leverage case law (actual decisions the framework led to and whether we qualitatively agree with them) to update the framework over time.

Step 3: Ensuring your new team gets the recognition it deserves

Congratulations, you now have a "one goal team" solving the right problem, with the right incentives, with a solid roadmap! Now you should make sure everyone else you work with agrees.

Thankless work

This can be especially tricky for teams working on problems like compliance because the measuring stick most people (including most of your leadership team) are used to using when grading teams and doing individual performance reviews is business metrics. It's easy to compare the team or the engineer who grew revenue by $50M this year against the other who grew by $25M, but how do you compare that $50M against... probably preventing a $100M loss due to a crisis that never ended up happening?

That "probably" is the problem. If you can't see the problem, you can't be sure that you made an impact on it. This is why people who put out fires after they happen often get more recognition in companies than people who build the infrastructure that prevented the fire from ever happening in the first place.

Leadership buy-in

This sort of problem is systemic in nature and requires a systemic solution. And since the systems your organization operates by are largely influenced by your organization's leadership team, it is them who you need to get bought in.

Take the time to educate your leadership team on how your new team sets goals, how your scorecard methodology works, and what "pre-mortem" scenarios you believe you'll be avoiding through these proactive investments.

You'll be even more effective at making your point if you can point to past crises that could have been prevented with more proactive work. If your company has no precedent for this, point to other companies. No company wants to go through the next #DeleteUber or Facebook Russian election interference scandal.

Do all of this, and you can be more confident that your engineers will get the performance ratings they deserve, and that your team can secure and even grow headcount as more opportunities arise.

Org-wide culture

Eventually, this can even turn into a cultural shift across the broader organization. With org leaders publicly recognizing your new team for success in your newly defined goals, and with individuals on your team seeing their careers grow, it will become clearer to everyone else that focusing on more ambiguous areas like compliance and platform reliability was valued just as much as growing the business.

-------------------------------

Hope this helps! Feel free to reach out here if you have any questions or feedback.