A product manager can describe past launches well and still struggle with the real work. They may not know how to frame a fuzzy problem, weigh conflicting evidence or defend a trade-off. Interviews reward memory, confidence and rapport as much as judgment. A work sample is different. It asks the candidate to do a small version of the job: diagnose retention, rank a backlog or write a decision memo. Every candidate then produces evidence you can compare, made under the same limits.
Hiring research likes work samples because they test the job itself. But the design decides whether that pays off. And recent studies say they predict performance less well than people used to think.
A vague prompt gets you polished documents you can't compare. A realistic task with clear scoring gets you evidence the panel can weigh next to the rest of the process. Use the exercise to answer one question: can this person make the most important recurring decisions in this role, at the level you need?
Map performance before you write prompts
Start with the outcomes the role owns. A generic list of PM skills will not tell you what to test. A senior growth PM, a platform PM and a PM building something new from zero share a title. But they face different decisions and fail in different ways. Interview the hiring manager, a strong peer, a design or engineering partner and whoever uses the role's output. Ask them where poor judgment causes costly rework or harm.
Map work you can observe. "Strategic" is a label. "Identifies a target segment, states a choice, explains the trade-off and names the evidence needed before committing" is something you can score.
| Performance area | Observable evidence | Suitable simulation |
|---|---|---|
| Problem framing | Separates symptom, need, constraint, assumption | Signal review and problem brief |
| Prioritization | Ranks opportunities against a clear outcome and cost | Backlog ranking |
| Product judgment | Chooses under uncertainty, names what would change the choice | Decision memo with partial data |
| Execution design | Splits the work, finds dependencies, defines what proves a release worked | Delivery plan or story map |
| Stakeholder alignment | Explains a decision and handles a fair objection | Live review or role-play |
| Measurement | Defines success, safety metrics, segments, time windows | Metrics plan or experiment critique |
Limit any single assessment to two or three of these areas. If you test discovery, strategy, delivery, analytics and leadership at once, you get a long exercise with weak scoring on each. Spread evidence across structured interviews, references and calibration. And keep skill with tools separate from reasoning about user data.
How ambiguity should scale with seniority
A good task copies the judgment, the output and the limits of the real work, whatever it looks like on the surface. Match the amount of uncertainty to the level.
For associate and early-career roles, give a clear problem and a small set of evidence: funnel data, six customer quotes and a release limit. Ask for the most likely point of friction, one test and the event that would show whether it helped. That measures reasoning and communication without asking for company-wide strategy.
For mid-level PMs, add competing needs and missing information. Ask the candidate to choose between three options: improve an invite flow, cut setup time or build a permission request for enterprise clients. Then ask them to explain the affected segment, the assumptions, the upside, the cost and the rule they used to decide. You are looking for logic you can follow and honesty about what stays unknown. There is no single "right" answer to find.
For senior, staff and lead roles, test whether the person can shape how a decision gets made. Use a scenario that changes as it goes. A leader wants a feature for a large prospect. Engineering flags reliability debt. Research finds onboarding problems. Ask for a decision memo, then a 20-minute review with two interviewers playing stakeholders. Watch how the candidate protects customer value, negotiates scope and changes course when the evidence changes. Do not mistake forceful pushing for leadership. A strong candidate states a recommendation and the conditions that would reverse it.
At every level, the brief decides how much of that judgment you get to see. Give enough context to reason with. Make sure no hidden answer depends on insider knowledge. A workable brief states a business goal, customer context, evidence of mixed quality, known limits, the format and time allowed, how the work will be judged, and whether outside research or AI tools are allowed. It should feel like real product work, where the hard part is choosing what to ignore.
Design the uncertainty on purpose. Endless uncertainty just rewards made-up assumptions. So set boundaries, such as "You are not expected to estimate market size."
Keep to a reasonable time limit. For early- and mid-level screens, 60-90 minutes is usually enough. Long take-home tasks hurt people with caring duties, disabilities or little free time. Never ask applicants to solve a live problem you plan to ship. Use made-up data or a past decision with no commercial value. Nobody on the panel should get free consulting out of a hiring exercise.
Score decisions, not visual polish
Write the scoring guide before anyone starts. Each criterion needs clear levels of evidence linked to the performance map. Turn vague labels like "executive presence", "culture fit" and "good product sense" into behavior you can observe.
| Criterion | Limited evidence | Solid evidence | Strong evidence |
|---|---|---|---|
| Problem definition | Repeats the brief without a diagnosis | Identifies a likely user or business problem | Separates symptom, root cause, segment and uncertainty |
| Prioritization logic | Names an idea without criteria | Uses outcome, reach, effort or risk to compare | Makes trade-offs clear and names a signal that would prove it wrong |
| Metrics design | Lists broad metrics such as active users | Defines a success metric and a safety metric | Specifies event, group, denominator, time window and threshold |
| Communication | Presents unconnected observations | Gives a clear recommendation and reason | Expects objections, adapts to new information, stays clear |
Use three to five criteria. For each, write a short description of what a 1, a 3 and a 5 look like. A 10-point scale pretends to a precision nobody has. For important roles, have two trained assessors score on their own before they talk. That way the first confident opinion does not become the verdict. When you compare scores, ask for a specific sentence, choice or calculation behind each rating. "I'd enjoy working with them" is not evidence from the work sample.
Worked example: apply the rule, not the average
Two mid-level candidates take the retention exercise. Each gets 1, 3 or 5 on four criteria. The decision rule was fixed in advance: a solid score (3 or higher) in both problem definition and metrics design, and no serious concern.
Candidate A scores 5 on problem definition, 5 on prioritization logic, 1 on metrics design and 5 on communication. Candidate B scores 3, 1, 3 and 3 on the same four criteria, in that order.
Candidate A averages 4.0 and B averages 2.5. One blended number puts A far ahead. The rule reverses that. A never separates a success metric from a safety metric (metrics design 1). So the panel cannot trust any claim about whether the proposed change worked, and that is exactly the gap the exercise was built to find. B is ordinary and weak at prioritization, but passes both required criteria. A does not move forward on this task. B does, with prioritization flagged for the next interview. The average would have hidden the deal-breaker. That is why the rule names the criteria nothing can make up for. It is also why the scale stays at 1, 3 and 5, instead of producing a 4.0-versus-2.5 gap that looks decisive. (Scores are illustrative.)
Bias, access, and the coaching gap
Giving everyone the same task does not make it fair. Some candidates have practiced case interviews, know product jargon and write business memos every day. Others never have. Unless you design against it, that coaching shows up in the scores.
Give every candidate for a role the same prompt, preparation time, materials, scoring guide and follow-up format. Also provide accessible documents and reasonable adjustments. In the United States, the Equal Employment Opportunity Commission notes that employers may need to provide reasonable accommodation during application and hiring under the ADA. Rules vary by country and state, so involve qualified employment counsel or HR partners.
For written exercises, hide names, schools and past employers for the first round of scoring. It will not remove bias in live discussion, but it cuts status signals that don't matter. Also watch for industry knowledge built into the case. A marketplace case full of unfamiliar unit economics tests marketplace experience more than product skill. If that experience is required, say so and score it. If it is only nice to have, supply a glossary.
Does the sample actually predict?
Treat the work sample as a hiring hypothesis. Write down what it measures, then test whether its scores match later job performance. Run a pilot and keep the scores without names. Compare them once enough hires have finished a work cycle. Decide in advance what counts as evidence: results at 90 days, manager ratings at six months, peer feedback or documented delivery. Do not rely on a single rating after hiring, which can reflect the team or the rater's bias. Where it is legal and useful, look at patterns by level, hiring source and demographic group.
If assessors often disagree, the scoring guide is usually too vague. Rewrite the level descriptions, practice on past submissions and test again before the task becomes a gate. Over time, track a few numbers: how many candidates finish, how long they really take, how scores spread on each criterion, pass rates by demographic group (with legal advice), and how scores relate to later results. SIOP's selection principles offer a workable standard: link methods to a study of the job, document the evidence and review how the method is used.
Checking whether the task predicts anything takes more hires than most product teams have. Suppose the link between task scores and later performance is moderate, a correlation of about 0.3. To see it reliably with standard statistical tests, you need about 85 hires with both a score and a performance rating. A company that hires eight PMs a year would wait a decade. There is a second problem. Only people you hired ever get a rating, and they all passed your bar. With the low scorers missing, the link looks weaker than it is, even when the task works. Small teams should not pretend to run a prediction study. They can do three cheaper things well: check whether the two assessors agree on every submission, show that each criterion traces back to the performance map (SIOP calls this content validity), and pool anonymous scores across years until the numbers mean something.
Some hiring leaders go further and question work samples altogether. Newer reviews of hiring research rate them lower than the figures quoted for decades. A well-run structured interview now tends to come out ahead. If the sample is a long, loose take-home scored by gut feel, the skeptic has a point, and the interview is the better buy. The picture changes when the sample has structure too: a fixed prompt, scoring levels written in advance, two independent raters and a live follow-up. Then the work sample works like a structured interview built on real material. It has one extra advantage: candidates show a decision instead of only describing one.
A retention scenario for SaaS PMs
A B2B SaaS company gets new users started well, but many leave in the first month. A mid-level candidate gets activation by sign-up group, weekly retention by account size, five support excerpts and a note that setup automation needs eight weeks. The task: 75 minutes for a two-page memo. It must name a priority segment, a hypothesis, one product change, one measurement plan and one safety metric.
A weak answer treats all churn the same and lists features. A solid one sees that small teams get started but don't invite colleagues. It defines retention as coming back to a shared workflow in weeks two to four. A strong one also flags other explanations, such as bringing in the wrong customers, and proposes comparing sign-up groups before an expensive build.
Then add a live twist. An enterprise customer wants custom onboarding that would take up the same team. Now the score depends on how the candidate updates the recommendation and weighs the revenue trade-off.
Placing the sample in a disciplined loop
A work sample should never be the only gate. Pair it with structured interviews about past behavior, a conversation about working together that fits the role, and references that test the same claims. Set the decision rule in advance. For example: at least a solid score in problem framing and metrics design, and no serious concern about ethics or teamwork. Write down every exception. If a panel hires below the bar, record why and review how it turned out. Exceptions nobody sees slowly turn a process into personal preference.
For validation and documentation of selection methods, see the Society for Industrial and Organizational Psychology's Principles for the Validation and Use of Personnel Selection Procedures. For research background, see Schmidt, Oh and Shaffer's 2016 review, The Validity and Utility of Selection Methods in Personnel Psychology. For U.S. guidance on accommodation, review the EEOC's guidance on job applicants and the ADA.
Use these as inputs to your design. Have HR, legal and talent teams review the process against the laws and policies that apply to you.
Three groups have to be able to stand behind the process: candidates who found the task fair, assessors who can explain each rating, and leaders who can show how the exercise maps to work that matters. Until all three can, treat the sample's scores as one input among several. Let the interview and the references carry more of the weight.