How to Answer Root Cause Analysis Questions
The most common type of execution question is root cause analysis.
Root cause analysis (RCA) questions present a scenario. Something unexpected has happened with a product, and as a PM, it’s your responsibility to figure out what’s going on. For example, an interviewer might ask:
"Imagine you are a PM at Lyft and there’s been a 20% increase in ride cancellations. What would you do about this?"
These interviews are unlike most others in the PM loop in that they are conducted via a “role-play” interviewing style. Your interviewer expects you to ask questions about the context of the problem, the product, and any relevant metrics. Using the information you gather, you’ll iterate through increasingly precise hypotheses until you identify a root cause. Often, your interviewer will play the role of an uninformed data analyst, requiring you to ask concrete questions about reasonably available data.
For example, one of your first clarifying questions might be “Is there anything unusual going on with drivers?” In other PM interviews your interviewer would answer directly, but in execution interviews, they'll likely respond with something like “As a data analyst, I can help you if you clarify what you want to know in terms of data we might have about drivers?”
The interviewer will push you to be more specific and concrete about your requests for data throughout the interview.
What interviewers are looking for
Your ultimate goal in working through RCA questions is to identify the root cause of the problem, but your interviewer will care most about your approach.
RCA questions are different from product design, strategy, and analytical questions in that they do have a correct answer, but a logical and thoughtful approach to problem-solving can land you a "strong hire" recommendation even if you don’t arrive at the exact right solution. Be sure to demonstrate your ability to:
- Make sense of a situation given limited data
- Generate and continually refine reasonable hypotheses
- Gather information iteratively
- Identify a root cause
A framework for answering RCA questions
RCA questions at most companies are designed to be solved in 25 minutes so a framework to keep you on track is critical. Follow this 6-step process.
- Step 1: Clarify and gather context
- Step 2: Form high-level hypotheses
- Step 3: Gather data
- Step 4: Refine hypotheses and repeat
- Step 5: Identify the root cause
- Step 6: Evaluate
Let’s continue the Lyft RCA question as an example. Assume you’ve been asked:
"Imagine you are a PM at Lyft and there’s been a 20% increase in ride cancellations. What would you do about this?"
Step 1: Clarify and gather context
First, ask clarifying questions about anything that will help you make sense of the issue, scope the problem, and align with your interviewer.
Don’t hesitate to ask broad questions like “How does this affect our core business?” but be aware that your interviewer may not give you direct answers. Instead, they’re likely to prompt you to ask for specific data that you’d reasonably have access to as PM.
A helpful tip for getting into the right mindset is to envision what it’d really be like to be in the situation you’ve been given. What would your priorities be as PM? What kind of data would you need to begin to diagnose the issue?
Step 2: Form high-level hypotheses
Once you’ve gathered some context, you can start forming high-level hypotheses about what may have happened. Keep in mind that you have limited time to narrow your scope, so a good strategy is to first eliminate broad areas from consideration.
At this stage, you’re simply breaking up the universe of possible root causes into meaningful chunks that you can dive into later and eventually prioritize or discard. Here are a few common sources of problems:
- An unintended technical issue or a product bug
- A product change — possibly intended — but with unintended consequences
- An operational change within the company
- An external event
Remember that communicating your process is critical, so share your full list of hypotheses with your interviewer before you begin prioritizing in the next step.
Step 3: Gather data
At this point, you’ve got a solid set of high-level hypotheses. The next step is to determine which of these is worth expanding on. You’ll do that by gathering more data.
One helpful strategy for identifying what’s driving the problem is to isolate key variables. For the sake of an execution interview, a key variable is a variable that correlates highly with the issue at hand.
For example, say you’re diagnosing a problem with a smartphone app and the problem is evenly distributed between users across age groups and geographic locations. If, when you look at the distribution of errors experienced by device, you see that 90% of users experiencing the issue use a Google Pixel, you might consider device to be a key variable.
You’d then gather more data to figure out what Google Pixel usage correlates with the issue. It could be a certain version of the app installed on Pixels, something about the device itself, or even a factor that users who buy Pixels have in common. To isolate key variables, ask yourself these questions:
- Which variables could reasonably be correlated with the variation I’m seeing?
- How might I test whether these variables are having an impact without confounding variables affecting the outcome?
When you’re ready, ask your interviewer whatever questions feel meaningful to help you gather the right data. Be sure to explain why you’re asking each question and how the answer changes your understanding of the problem.
Step 4: Refine hypotheses and repeat
The goal here is to gather evidence to either support or deprioritize each hypothesis as you drive toward the root cause of the problem.
With more data, refine your hypotheses, generate new questions to ask, and refine further. You may find yourself repeating this step a few times. This is normal. Every additional piece of data helps to make your hypotheses more specific.
Be sure to communicate every step of your thought process to your interviewer and check in frequently. Doing so can preempt tough follow-up questions after you give your final answer.
Step 5: Identify the root cause
Once you have enough data, explain what you think the root cause of the issue is. If you’ve communicated your thought process throughout, you’ll have made a logical case for your choice, so you’ll only need to give a quick summary here before moving on to evaluation.
Step 6: Evaluate
If you followed the framework and communicated throughout, your answer will be backed up by data. Quickly recap your major findings, and then spend a few minutes discussing what to do about the root cause you’ve identified. Consider:
- Whether the root cause should be fixed. Sometimes, a change in a metric is caused by a product change that the company is otherwise happy with. The next steps may just be to keep an eye on things and to act only when it’s clear user experience is being degraded.
- If a fix is needed, consider what mitigation makes sense. Many candidates reach for a product change immediately, but that’s not always the best option. Use your judgment as a PM.
“Okay” vs. “good” vs. “great” answers
- In an okay answer, the candidate asks questions that are relevant and offers a light interpretation of the situation but doesn’t explain their overall thinking on the problem. It feels like they have an intuitive sense of what could be going on, but it’s not clear if their approach is comprehensive or deeply thought out. Alternately, they may quickly jump to plausible but specific explanations and focus on proving or disproving them, slowing down their ability to make sense of the situation.
- A good answer is one where the candidate shares what possibilities they’re considering and it is clear why they’re asking the questions they do. The candidate interprets responses and explains how they change their understanding. The candidate confidently navigates the possibility space and it feels like they make steady progress toward an answer.
- A great answer does the above but also displays deep insight and ability to contextualize the problem and uses that to guide their approach. For instance, the candidate might draw on product insights, explaining why and when users or drivers cancel, then apply that to quickly narrow in on the most important metrics to check. While regular answers to these questions usually feel like an exploration, great answers often feel like they quickly and intelligently cut through the possibilities and lock on to the heart of the problem.
Common pitfalls
- Immediately guessing at the root cause. If you are trying to prove or disprove a narrow hypothesis, it will take a long time to find the answer (or the interviewer will give up and guide you toward the answer to keep moving.)
- Not explaining what you’re considering or why you’re asking particular questions. If you don’t explain your approach, you’re more likely to get lost. To the interviewer, it may feel like there’s no rhyme or reason to your process. It also makes it harder for the interviewer to guide or help you.