Product Frameworks — Sushmita Mallick

Framework 01

The Confidence Calibration Matrix

Conviction is not knowledge. This matrix forces you to separate what you actually know from what you're assuming — before you commit resources, timelines, or team bandwidth to a roadmap bet.

The problem this solves

Most roadmap debates are confidence debates in disguise. Teams argue about priority when what they're really arguing about is how much they trust the underlying belief. The person with the loudest conviction wins — not the person with the strongest evidence.

The Confidence Calibration Matrix stops that. It gives every roadmap item a structured evidence score before it enters the prioritisation stack. If you can't score it, you can't prioritise it. That's the point.

When to use it Quarterly roadmap planning · New bet evaluation · Post-discovery prioritisation · Any moment someone says "I'm pretty sure users want this"

The 2×2 Decision Matrix

Score each item on Confidence (0–100) and Impact (0–100). The quadrant determines the action — no exceptions.

HIGH IMPACT ↑

⚠ VALIDATE FIRST

Low Confidence · High Impact

The Danger Zone

High stakes, weak evidence. Run the cheapest possible validation before committing. A wrong bet here is expensive.

✓ COMMIT

High Confidence · High Impact

Ship it

You know what needs doing and you know it matters. Stop deliberating. Prioritise, resource, and execute.

✕ DROP

Low Confidence · Low Impact

Kill it

Weak signal, weak value. This item is cluttering your backlog and consuming attention. Remove it with zero guilt.

→ DEFER

High Confidence · Low Impact

Batch & backlog

You know it's real but the payoff is modest. Time-box it, batch with adjacent work, or fold into a sprint buffer.

LOW CONFIDENCE → HIGH CONFIDENCE ↗

Confidence Scale — How to Score

Score confidence based on the strongest evidence you actually have — not the evidence you plan to collect.

Assumed

0 – 25

Gut feeling, single anecdote, or internal opinion only

Directional

26 – 55

A few interviews, indirect signal, or secondary research

Validated

56 – 80

5+ interviews, usability data, or a working prototype test

Proven

81 – 100

Quantified A/B test, live cohort data, or replicated research

The 7 Calibration Questions

Ask these before you assign any confidence score. If you can't answer at least four cleanly, your score is probably inflated.

1 Where did this idea come from? A single executive opinion and three corroborating user interviews are not equal weight.
2 How many independent sources confirm it? One source is anecdote. Three converging sources start to become signal.
3 When was this belief last challenged? Stale research decays fast. A finding from eighteen months ago may not reflect who your users are today.
4 Who in the room actively disagrees — and why? If no one disagrees, you're not calibrating. You're validating groupthink.
5 What evidence would prove this belief wrong? If you can't answer this, you don't understand the belief well enough to score it.
6 Is validation cheaper than commitment? If a two-week spike would move your confidence from 30 to 70, that spike belongs in the roadmap before the full bet does.
7 Are we solving the root problem or a visible symptom? High-confidence solutions to the wrong problem still fail. Diagnose before you score impact.

Live Calibration Calculator

Confidence Score 50 Directional evidence

Impact Score 50 Moderate impact

CCM Score (Conf × Impact / 100)

→ Defer & batch

Moderate confidence, moderate impact

Evidence Rules

User demand claims

Requires direct quotes from ≥5 independent interviews confirming the same unmet need. "Sales said customers want this" does not qualify as user evidence.

Retention / engagement impact claims

Requires historical data from a comparable feature launch, a proxy metric with a causal argument, or a controlled cohort analysis. Analogies from other companies count as directional, not validated.

Competitive parity claims

Requires verified feature audit of the named competitor — not a salesperson's account of what the competitor does. Screenshots or documentation required.

Engineering estimates

Rough order of magnitude counts as directional (max 40 confidence on the effort side). Task-level breakdown with dependencies mapped counts as validated.

Framework 02

RICE + Regret Scoring

RICE tells you what to build first. Regret scoring tells you what you can live with being wrong about. Add the regret layer to every high-stakes call — especially the ones that feel obvious.

The problem this solves

Standard RICE treats a reversible experiment and an irreversible architectural decision as equivalent bets. They are not. You can undo a feature flag in an afternoon. You cannot undo a pricing change, a data model, or a public commitment to an enterprise customer.

Regret Scoring adds a second axis: how much you will regret this decision if it goes wrong — and how much you'll regret not making it if you hesitate. The delta between those two regrets is your decision pressure signal.

When to use it Bets with multi-quarter consequences · Irreversible decisions · Calls where the team is split · Anything your CEO will ask about in six months

Base RICE Score

Reach × Impact × Confidence % ÷ Effort = RICE Score

REACH

Users or events affected per quarter. Use a conservative estimate — not an aspirational TAM slice.

IMPACT

0.25 minimal · 0.5 low · 1 medium · 2 high · 3 massive. Be honest. Most features are 0.5.

CONFIDENCE

Use your CCM score here. Assumed = 20%, Directional = 50%, Validated = 80%, Proven = 100%.

EFFORT

Person-weeks to ship. Include design, eng, QA, and post-launch monitoring. Scope creep is not an excuse for a low number.

The Regret Layer

Score each dimension 1–5 independently. The delta tells you which direction to pressure-test.

Regret of Action (RoA)

How bad is it if we build this and it fails, backfires, or locks us in?

1 Fully reversible. Undo in a sprint. Low sunk cost, low visibility.

2 Recoverable within a quarter. Some rework required.

3 Significant rework or trust damage. Recovery takes 2–3 quarters.

4 Major reputational, contractual, or architectural consequence.

5 Irreversible. Pricing change, data model, public commitment, or partnership lock-in.

Regret of Inaction (RoI)

How bad is it if we skip this and competitors move, users churn, or the problem compounds?

1 No consequence. Problem self-resolves or nobody notices.

2 Minor friction persists. Users adapt or find workarounds.

3 Measurable churn or NPS drag. Competitor may capitalise.

4 Significant market share or pipeline risk. Window closing fast.

5 Existential. Missing this is a category-defining mistake.

Final Score Formula

Regret Delta = RoI − RoA

Regret Multiplier = 1 + (Regret Delta × 0.1) · clamped between 0.5× and 1.5×

Adjusted Score = RICE Score × Regret Multiplier

Positive delta = inaction is riskier → bias toward shipping. Negative delta = action is riskier → pressure-test harder before committing.

Live RICE + Regret Calculator

Reach (users/quarter) 500

Impact (0.25→3) 1

Confidence % 80

Effort (person-weeks) 4

Regret of Action (1–5) 2

Regret of Inaction (1–5) 3

100

Base RICE

Regret Delta

1.10×

Multiplier

110

Adjusted Score

Bias toward shipping — inaction is the bigger risk

Pressure-Test Triggers

When any of these conditions fires, stop. Write the narrative before you proceed.

🔴

RoA ≥ 4 — Write a one-page failure narrative before committing. Who owns the decision? What does rollback look like? What is the minimum viable version that reduces irreversibility?

🟡

RoI ≥ 4 — Write a competitive risk brief. What are competitors doing right now? What happens to retention and pipeline if this ships in six months instead of six weeks?

⚡

Regret Delta ≥ 3 in either direction — Asymmetric risk. Escalate to leadership for a final call. This decision is too lopsided for a product team to absorb alone.

⚠️

High RICE + Low RoI (≤ 2) — Sanity check your Impact score. If skipping this has no consequence, you may be overvaluing the upside. Re-run with a 0.5 impact multiplier and see if it still tops the stack.

Evidence Rules

Reach estimates

Must be grounded in actual active user counts or confirmed pipeline volume — not registered users, not projected growth. Use last-quarter actuals unless you have a firm contracted commitment.

Impact scores of 2 or 3

Require a comparable precedent: a prior feature of similar scope that produced a measurable outcome. "This feels big" is not evidence for a 2 or 3. Default to 1 when in doubt.

Regret scores

Score both dimensions independently, then compare. Do not anchor RoI to RoA or vice versa. They measure different failure modes. Have at least one dissenting voice score both before accepting the team average.

Framework 03

The Silent User Audit

Support tickets only capture the vocal minority. The users who leave without saying a word are your most expensive problem — and the hardest to fix, because they left you no diagnosis. This method recovers the signal they didn't know they were sending.

The problem this solves

Churn analysis usually starts too late — at the moment of cancellation. By then, the decision was made weeks or months ago. Silent churners made their decision at a specific moment of friction, confusion, or disappointment, and then continued using the product on autopilot until the subscription renewed or the trial expired.

This audit works backwards from the exit to find that moment. It combines behavioural archaeology (what the data shows) with structured re-engagement interviews (what the user actually experienced) to produce a pattern — not a one-off complaint.

When to use it Churn rate rising with no obvious cause · NPS drop without a corresponding ticket spike · Onboarding completion falling · Healthy activation but poor retention at day 30, 60, or 90

The 5-Phase Audit Flow

Phase

Segment

Classify churners before you contact anyone

Phase

Dig

Behavioural archaeology in product data

Phase

Reach

Structured re-engagement interviews

Phase

Pattern

Signal vs noise — when to act

Phase

Fix

From pattern to roadmap priority

Phase S — Segment: Four Silent Churner Types

Classify your churners before you contact anyone. The interview questions, behavioural signals, and fixes differ by type. Misidentifying the type means solving the wrong problem.

The Ghost

Never Activated

Signed up. Logged in 1–3 times. Never reached the activation milestone. Disappeared silently. Often blames themselves, not the product.

Data signal: Zero feature adoption beyond account creation. Last session under 5 minutes.

The Explorer

Never Had Intent

Signed up to compare or benchmark. Browsed broadly, activated nothing deeply. No purchase intent from day one. Your ICP description didn't filter them out.

Data signal: Wide shallow usage across features. No repeat session on any single workflow.

The Switcher

Found Something Better

Was an active user. Usage declined gradually, not suddenly. Found a competitor or alternative that solved a specific gap you didn't know mattered to them.

Data signal: Decreasing session frequency over 30–60 days before exit. Feature usage narrowed to one workflow before stopping entirely.

The Misled

Expectation Gap

Came in with a specific expectation set by marketing, sales, or a referral. Reality didn't match. Disengaged after the first serious attempt to use the product for their actual job.

Data signal: High early engagement followed by a hard stop after one deep workflow attempt. Often correlates with a specific campaign or sales motion.

Segmentation without asking

Classify each churner using product data alone before any outreach. Ghost: check activation event. Explorer: measure feature breadth vs depth ratio. Switcher: chart session frequency trend over last 90 days. Misled: cross-reference last-used feature against their acquisition source or sales notes.

Phase D — Dig: Behavioural Archaeology

Pull the usage trail for each churner. You are looking for the moment the relationship broke — not the moment they left.

What to pull from product data

→ Last action before exit. What was the final thing they tried to do? This is usually the breaking point.
→ Session frequency trend. Was drop-off sudden (event-driven) or gradual (slow disillusionment)?
→ Incomplete workflows. Which multi-step flows did they start and abandon, and at which step?
→ Feature footprint. Which features did they never touch? These are often the ones you assumed would drive retention.
→ Help-seeking behaviour. Did they search the docs, open the tooltip, or visit the help centre before the drop-off point? Or did they just silently stop?

Warning signals to flag

⚠ Multiple users dropping off at the same workflow step → systemic friction, not user error.
⚠ Churners concentrated in one acquisition cohort → expectation set upstream, not a product problem.
⚠ High-value churners (high ACV, power users) → escalate to exec level immediately, don't wait for pattern.
⚠ Drop-off correlates with a specific release → regression introduced, not a strategic problem.

Phase R — Reach: The Interview Protocol

Run these as 20-minute conversations, not surveys. Order matters — let them set the scene before you show them what you found. Never lead with your hypothesis.

"Walk me through what you were trying to accomplish when you were using [product]."

Sets their frame, not yours. You want their job-to-be-done in their own words — not a response shaped by your product vocabulary.

"Was there a moment where you felt like it wasn't going to work for you?"

This is the pivot question. You're hunting for a specific moment — a feature, a workflow, an error. Most users will point to it clearly once you ask directly.

"What did you do instead — did you find another tool, go back to your old process, or just leave the problem unsolved?"

Identifies whether you lost to a competitor, to inertia, or to the user giving up entirely. Three completely different strategic responses.

"When you look back, what would have had to be different for you to stay?"

The only question that directly tests whether this is fixable. Listen for product answers vs. expectation answers — they point to different parts of the org.

"I noticed you [specific behavioural observation from your data]. What was happening for you at that point?"

Shows you did the homework. Ties behavioural data to lived experience. This is where the archaeology becomes actionable — they'll confirm or reframe what you found.

Interview rules

Never offer credits, discounts, or win-back offers during the interview. It poisons the data — they'll tell you what you want to hear. Contact within 14 days of churn for best recall. Target a 15–20% response rate from your outreach list; if lower, your outreach framing is the problem, not the topic.

Phase P — Pattern: Signal vs Noise Thresholds

A single user saying a feature is confusing is a complaint. Three independent users hitting the same wall is a signal. Five is a mandate.

Minimum Signal

Same friction point, same segment type, independent interviews. Log it, watch for more. Don't act yet.

Act Threshold

Five corroborating instances. Bring to roadmap review. Correlate with behavioural data before committing a fix.

Escalate Immediately

Systemic. This is affecting a category of users, not individuals. Escalate, size the ARR at risk, and prioritise above the current sprint.

Phase F — Fix: Pattern to Priority

Classify each confirmed pattern into a fix type. This determines which team owns it and what the right response looks like.

Pattern	Segment likely	Fix type	Typical owner	Wrong response
Drop-off at the same onboarding step across Ghosts	Ghost	UX / Flow	Product + Design	Adding a help article at that step
Activation milestone never reached despite multiple sessions	Ghost / Misled	UX / Onboarding	Product + Growth	Sending a "tips" email sequence
Users who activated churn when they hit a specific feature ceiling	Switcher	Feature Gap	Product	Better documentation of workarounds
Churners from one specific campaign or sales source	Misled	Messaging Fix	Marketing / Sales Enablement	Adding a product feature to match the promise
Explorers who never intended to buy but consumed trial quota	Explorer	ICP Filtering	Marketing / RevOps	Improving onboarding for non-buyers

The accountability question before shipping a fix

For every pattern-driven fix, state: "If we ship this and the churn rate for this segment does not improve within 60 days, what do we look at next?" If you can't answer that, you don't have a fix — you have a guess dressed up as a fix.