How to know if you're interviewing at a strong product company
I'm seeing this sentiment pop up with alarming frequency. Many product people are dealing with career uncertainty — and not just the "will my company make it through the pandemic?" variety. Without the usual distractions of life available, things that were easily overlooked are now front and center. And what are many of these product people realizing without the distractions of normal life?
They're set up to fail.
Their work environment and culture is not set up to create strong products. It is distinctly not set up to empower the collaboration of product, design, and engineering to build things that matter.
This has led many to realize it's time for a job change.
As product people, we want to work in a strong product company. Note: this does not mean that the product org is "in charge," giving orders and engineering/design have to get on board. It does mean a place that truly values the craft and contribution of product, and that empowers individuals and teams to work to their highest potential. A place that is built around creating amazing products that truly make life better for the people they're trying to serve.
This is inherently and fundamentally a collaborative process that no single functional org rules by fiat. (The most common counterexample is a sales-led organization.)
We all know it's possible. It's what the best product companies are doing. Yet despite many attempts tried, there are many frustrated PMs that can't seem to get their work environment to change. Why not?
Part of the reason is that transformation is extremely hard, even in the best case.
But the real reason is they joined the wrong company in the first place.
They took a role where good product work occurs in spite of the dominant practices and culture, not because of them.
Jobs like this waste precious years of your career. I've made this mistake too, and it hurts. I hope this article helps you avoid it in the future.
It's actually really hard to know how strong of a product culture and environment that a team has prior to living in it. Even within the FAANG companies, every team and manager is different and you need to find out the truth.
After reading this, you should have a go-to set of questions to ask the next companies you interview. These will help you suss out the reality of how the company operates before you commit to an offer. I'm writing this for product managers, but it's equally relevant to design and engineering.
So, what can you do to make sure that your next company is actually a good place to be a product person?
The key to solving this is to interview companies the same way we interview users and customers.
Think of this as "The Mom Test," applied to product job interviews.
Interviewing companies like users/customers
Why you need to do this
One of the most common complaints in the job search process is "this place isn't what I thought it would be."
People join a company full of vigor and excitement, and within a few months they are bored, going through the motions, and fantasizing yet again about being somewhere that "gets product."
It doesn't have to be this way.
As PMs, we invest so much getting ready for those damn product interviews that we often forget: we need to interview the company just as hard as they're interviewing us.
This is called "reverse interviewing." As product people, we already know how to do this.
We know how to interview customers and users. We know to ask questions to account for bias and speculation, surface actual behavior, and understand how decisions are really made.
When applying for a job, we need to apply those same skills to interview our interviewers.
As Marty Cagan said to me in our podcast conversation:
From your point of view, your job is to learn as much as possible about how that company really works, and especially how that manager would be like to work for.
Let's see how to do just that.
Principles for interviewing
Two excellent resources on interviewing users—which you really should read if you haven't—are this post by Teresa Torres and The Mom Test.
I distill their points into these two overarching principles for interviewing:
Separate your research questions and interview questions
Ask questions that uncover actual past behavior, rather than speculative or aspirational future behavior
Separate research questions from interview questions
The user research community has learned that we often can't directly ask our questions and get reliable answers.
The way around this is to map our research questions to interview questions.
Research questions are what we really want to know. Interview questions are what we actually ask to get the interviewee telling stories that show their behavior. (This is a concept I learned from Teresa Torres in her excellent interviewing course.)
For example, imagine you work at Spotify and your team is assigned to work on reducing the churn rate.
Let's say you've learned that users who play a given playlist consistently at the start of their workday are retained ~20% longer than those who don't. Your research questions might be:
Why are people using the same playlist every day?
What are people looking for in such a playlist? How do they choose playlists?
When, where, and how do people find these playlists?
What is unique about playlists that have high retention rates?
Are users more likely to exhibit the desired behavior with a playlist they follow, or one they create?
Some interview questions to get stories containing this information could be:
Tell me about your morning playlist? How did you first find it?
Walk me through your day. Start with the moment you woke up.
Tell me about how you start your workday
Tell me about the last playlist you discovered that you really liked
Tell me about the last playlist you made yourself
Each of these prompts a story about actual behavior, and we can then ask more questions to go deeper. A well-extracted story can often answer multiple research questions at once.
Now let's talk about how to apply this to finding a great product environment to work in.
The droids we're looking for
What we're seeking in our job search: empowered product teams.
Empowered product teams—as opposed to delivery or feature teams—only exist in environments with strong product leadership.
Empowered product teams:
are small, cross-functional, and durable
address product risks early with collaborative product discovery, and regularly "kill their darlings" en route to ideas that work
are accountable to delivering results rather than output
These teams exist in product orgs with an environment that has:
a focused and insight-rich product strategy
regular, ongoing cadence of lightweight research directly with their user/customer, rather than handing this off to a research group
people connected to the product vision in a visceral, energetic way
managers that are proactively and regularly coaching and developing team members
an equal partnership with the rest of the business
Teams like this are great places to be a product person. These are where magical career experiences and products come from. And they are made up of ordinary, hard-working, sincere people just like you.
(To go deep on the idea of empowered teams, read Marty Cagan's latest book, EMPOWERED, and listen to the deep dive conversation I had with him about the ideas in the book.)
Our research question is: "is this a place that empowers product people/teams?"
Now that we know what we're looking for, how do we assess a given job opportunity?
This is where our interview questions come in.
Reverse interview questions
Since every company we talk to will say they're a good product company, we need to dig deeper. Just as with users, we can't take them at their word. We need evidence from real behavior.
Based on my own career, research, listener questions and podcast interviews with product people, here are my top reverse interview questions (click to jump to the section for the given question):
What were the last few things your team has built and shipped, and how did you decide to do those?
When's the last time you talked with your customers? How often have you done that in the last month?
What was the last feature or product your team killed?
Can you describe your product vision?
Tell me about the last coaching session you had with your manager. How often did that happen last month?
Let's discuss the rationale for each of these, and what you want to hear/avoid.
1) What were the last few things your team has built and shipped, and how did you decide to do those?
Why you're asking
We need to understand the core of how work happens here. How is work assigned? Who decides what gets built, and how?
This is the biggest indicator of whether this team is empowered. This is the motherlode. Plan to dig deep into this one. Pull every thread and see where it takes you.
What you're looking for
We want to see that leadership has assigned the team clear objectives — customer or business problems to solve — and empowered the team to come up with solutions to those problems. We're looking to see that leadership has pointed the team in the right direction, and empowered them to figure it out.
We want to hear things like "well, our goal this quarter is to increase the average 4-week retention for a new user cohort from 33% to at least 40%. We interview users every week and dug into this pattern from last quarter's data. The PM/designer/tech lead ran a fast series of prototypes/experiments to figure out what worked with users, and to make sure our stakeholders could support it. After we had enough confidence that it was the right thing to build, we put it on the backlog and built it out for production."
What's wrapped up in that statement? Tons of goodness:
A clear goal, with metrics, that the whole team is focused on
A regular cadence of user/customer contact
Insights that inform the product strategy and shape the objectives the teams pursue
Rapid, iterative, and collaborative prototyping and continuous discovery practices
Addressing the four big risks early on in discovery, before building for production
A collaborative, rather than subservient, relationship with stakeholders—we hear that the product team exists to delight the customer, in a way that works for the business
We'd also love to dig into the tradeoffs and tough calls that were made in that process. This is a place we'd love to hear the product vision and principles shaping the daily product decisions that the team is making.
What you don't want to hear
"Well, it was on the roadmap…"
Ah yes, the roadmap issue. This is where it's going to come out. Roadmaps aren’t inherently bad, but we definitely don't want to hear that the team just built stuff because they were told to by a stakeholder, or because it was on the roadmap. (Which is equivalent in many companies where real product discovery does not exist.)
We don't want to hear that "well, our goal for the quarter was to ship feature X, so that's what we did."
We want to see that the team is building things because they are accountable to delivering results, not output.
As Christina Wodtke shared:
Success is not checking a box.
Success is having an impact.
If you complete all tasks and nothing ever gets better, that's not success.
We don't want to see that any one person — especially not the PM — tried to dictate what would happen by fiat.
We don't want to hear egos and opinions. We want to hear about data, customers, and the results of experiments/prototypes being used to answer hard questions.
2) When's the last time you talked with your customers/users? How often have you done that in the last month?
Why you're asking
Good product teams are constantly engaging with their end users and customers. They are not just doing product discovery, they are doing continuous product discovery (and delivery).
They are also addressing risks up front, prior to committing to build something. This means they are addressing the big risks—value, usability, feasibility, viability, and ethics—before committing production engineering resources to delivering it.
What you're looking for
We want to hear that they've talked with their customers and/or end users very recently, and that it's a totally normal thing there. Ideally, at least once a week. It should be weird and cause cultural dissonance if a team goes 2+ weeks without customer contact.
We want to see that the product team has its own regular cadence of lightweight, ongoing customer research. And that research is being done by the same team that is building/delivering the features.
We want to hear that the company has structures in place to support this continuous interviewing—especially interviewee recruiting and scheduling, which is the biggest hurdle to establishing this cultural habit.
We'd also love to hear that all three functions—product, design, and engineering—participate in interviewing. Especially engineers—they are closest to the enabling technologies. Magic happens when engineers are directly exposed to user/customer problems, because they are the ones who will know what's just now possible to solve those problems.
What you don't want to hear
"Hmm, let me think… you know, I'm not sure. I'd have to double check with our research group."
We also don't want to hear that it didn't happen, uncertainty, or that interviewing was a one-off / uncommon thing.
We don't want to hear that user research is "owned" by a separate team. (It's great if there is a strong research capability we can tap into, and the product team should be doing its own research on the opportunities it's pursuing.)
I'm also leery of places that talk about doing "voice of the customer (VOC)." This very entreprise-y term is usually code for "we did a big batch of interviews one month, then stopped talking to people and went into a cave to build for two years."
That doesn't work.
It's good to have a larger phase of generative research before undertaking a big new product initiative or vision. It's also not a replacement for the fresh data that product teams need on an ongoing basis to make hundreds of product decisions from week to week.
As Rich Mironov says, if you don't do the work to deeply understand user needs, you lose the right to make decisions on their behalf.
That right needs to be continually earned.
3) What was the last feature or product your team killed?
Why you're asking
Most ideas don't work at all, and those that do almost never work right the first time. It takes iteration.
We also want to understand the decision making process used to build/launch/kill features, and any weird power dynamics going on.
What you're looking for
We want to understand how they build things. Again, is the team addressing product risks early, before building it?
We want to hear a process of continuous discovery and a regular cadence of testing out ideas. If so, the team should be learning constantly, mostly about all the ideas that won't work.
Good teams kill way more ideas than they build. Successful products leave lots of prototypes and tests on the cutting room floor.
We'd also like to hear the team being thoughtful about why it didn't work. Were they deconstructing the assumptions that needed to be true for their idea to work, and testing those assumptions? Or were they just throwing stuff at the wall to see what sticks?
We'd also be happy to hear about the teams goals and metrics, that they are data-informed in their decision making. Their goals and associated metrics should guide their week-to-week product work. Data-wise, ideas should be thrown out because testing them didn't move the needle, didn't move it enough, or moved it (or a proxy/check metric) in the wrong direction.
What you don't want to hear
We don't want to hear that they are building and shipping features without doing the discovery work to find out if they're worth building ahead of time. People doing this are probably locked into an output-centric culture where they are subservient to stakeholders. They may also be way too precious about their ideas and be afraid of exposing them to the cold light of testing.
As Melissa Perri puts it in her excellent book on building a product-led org: this is the build trap that we need to escape. (Credit to her for this question specifically—check out her episode on the podcast.)
We don't want to hear about fuzzy goals, output-centric goals, or no goals at all.
Above all, we don't want to hear that they haven't killed any ideas, or have barely killed any. The worst form of this is something like "we built the whole product out and then found out nobody cared, so we shut it down after six months."
The idea kill rate should at least 50%, ideally 75%+. Again: most ideas don't work, and never the first time.
Overbuilding is a far worse sin than under-building. We can always add more. We can't get back what we wasted building stuff that doesn't matter.
4) Can you describe your product vision?
Why you're asking
A strong product vision is the North Star that aligns the work of the entire product organization. It's how the product organization is bringing the company mission/purpose to life! The whole reason to have product org in the first place is to fulfill this vision.
Every single person should know the direction, and be able to see how their work contributes to it. This is huge for alignment and motivation.
More importantly, we want to work on things that matter! We only have one life to live and our time is precious. Don't build bullshit.
Good leadership is about inviting people into creating a shared future together. This means leadership needs to be leading somewhere.
The vision tells us where "there" is. Know the destination before you get in the car.
What you're looking for
We want to see energy when we ask this question. We want to see eyes light up, people get more animated, talk faster, get excited. We want to see that this vision exists, it's alive in the culture, and that people care deeply about it.
We want to see the company calling their shot. We want to see that this company is going somewhere.
To go deeper, you can also ask each person, "for you, what is compelling about that vision?"
What we don't want to hear
Silence. Stumbling. Platitudes.
Frankly, most companies have no real vision. So people will stumble, fumble, look away. Because it's kind of awkward to get called out on that.
You may also hear some vague but nice sounding statement, or get a low energy response.
By the way, it's completely fine if there is a clear vision that they are excited about, and that vision just doesn't do it for you. That just means it's not a fit.
Go where you'll be excited about the future you're helping build.
5) Tell me about the last coaching session you had with your manager. How often did that happen last month?
Why we're asking
Coaching is the number one responsibility of managers. A manager's job is literally to help the people on their team succeed. They succeed when their people succeed. This means that those people need to be growing.
What we're looking for
Above all, we want to hear that coaching is happening, and that it's regular and normal for it to be happening. It should be happening at regular 1:1s for every person on the team (they are doing 1:1s, right?)
In addition to actively coaching their reports, we'd love to hear that the manager has worked with the person to create a clear development plan and is proactively and intentionally helping the person grow.
What you don't want to hear
We don't want to see blank looks, or any response that suggests that coaching is weird, uncommon, or frowned upon.
We don't want to hear anything that suggests a negative view toward coaching and development plans. If anything they says suggests that they view a coaching/development plan like a PIP, be very cautious.
Recap
Our central research question here is: "is this a place that empowers product people/teams?"
What we're really assessing with these questions is the strength of the environment for product people. Creating this environment is the responsibility of product leadership. We're assessing the quality of product leadership through the lived experience of the product teams.
Without strong product leadership, product people are set up to fail.
When you zoom out and review what you learned in the reverse interview process, look for teams that:
are small, cross-functional, and durable
address product risks early with collaborative product discovery, and regularly "kill their darlings" en route to ideas that work
are accountable to delivering results rather than output
are guided by an insight-rich product strategy
have a regular, ongoing cadence of lightweight research directly with their user/customer, rather than handing this off to a research group
connect with the product vision in a visceral, energetic way
have managers that are proactively and regularly coaching and developing team members
serve customers in ways that work for the business, rather than being subservient to the business stakeholders
My secret hope is that if every product person screened for these things in their job search, the competition for talent would force more companies to get their act together at the leadership level. That would lead to better teams, more fulfilling careers, and a world full of better products.
It took ~3,600 words to get to this conclusion. This was an enormous PSA telling you to be thoughtful about where you choose to invest your time. Life's too short to build things that don't matter. Go where you're excited about the future you're helping to create, and where leadership is doing its job: creating the environment that enables you to build that future.
Next steps
There is a huge body of resources out there about how to break into product, how to interview, how to get started on the job, etc. They're excellent and worth digging into.
But I haven't seen anything that really went deep into how to interview the company and ensure you're in a good product place in the first place.
I'm developing more on this topic—subscribe for future articles and let me know what this brings up for you in the comments below or on Twitter.
