◎ OtherBeta — synthesis improving

How do you know if you have a product problem or a distribution one?

3 views

3 modelsComplete

70%Consensus Score — 3 models mostly agreed, with some nuance. Read the breakdown.

🔴Semantic Entropy — Highly divergent · 3 distinct meanings across 3 responsesH=100%

70%

consensus

Mostly aligned

How much the models agreed on this answer

High agreement — the answer is well-supported across models.

Models agree on

✓Retention as the primary diagnostic for differentiating product vs. distribution issues
✓Product problems manifest as low retention and weak organic referrals
✓Distribution problems manifest as high retention but low new user acquisition
✓Need to validate willingness to pay via actual transactions (not surveys)

Points of disagreement

~Mercury 2 introduces the possibility of mixed problems (flat usage *and* growth), while the others focus on a binary split.
~GLM 5.2 simplifies testing to manual user acquisition, whereas Mercury 2 emphasizes structured matrix experiments.

How to Diagnose Product vs. Distribution Problems

Shared Framework

At the core, determining whether you have a product or distribution problem hinges on user behavior after initial interaction with your product. The key unanimous signal is retention:

·
Product Problem: Users try it but don’t stick around. Common indicators:
- ·Low retention rates (e.g., Day 1/7/30)
- ·Few organic referrals or word-of-mouth
- ·Negative/neutral feedback (NPS, CSAT)
·
Distribution Problem: Users love it but you can’t reach enough of them. Common indicators:
- ·High retention but low new user acquisition
- ·Organic growth signals (referrals, unsolicited praise)
- ·Willingness to pay validated via actual transactions

All models agree on this diagnostic approach, emphasizing retention as the primary differentiator.

Key Metrics Split by Problem Type (Expanded)

Product Problem Indicators

·
Activation rate: Users never reach the “aha” moment.
- ·Validation: Track sign-up → first meaningful action (e.g., task completion).
·
Retention curves: Weak Day-1/7/30 retention → product isn’t sticky.
- ·Validation: Cohort analysis; survival curves.
·
Feature usage: High abandonment or unused features.
- ·Validation: Heatmaps and event tracking.

Distribution Problem Indicators

·CAC vs. LTV: High CAC with low volume → inefficient channels.
·
Channel conversion: Low impression-to-sign-up rates.
- ·Validation: A/B test creatives/landing pages.
·Traffic quality: High bounce rates → wrong audience fit.

Diagnostic Tools Added by Individual Models

·GLM 5.2’s "Decisive Test": Manually acquire 10-50 users and observe their behavior over weeks.
·Mercury 2’s Matrix Experiment: Run 2 product variants × 3 channels to isolate product vs. distribution effects.
·Llama 4’s Customer Feedback Analysis: Leverage negative reviews (product) vs. positive-but-low-sales (distribution).

Contradictions & Nuances

·Mercury 2 highlights cases where both problems coexist (e.g., flat usage and growth) requiring parallel experiments.
·GLM 5.2 warns against the common trap of misdiagnosing poor retention as a distribution issue.

Actionable Next Steps

·
For Product Problems:
- ·Fix onboarding/core features.
- ·Conduct user testing to uncover friction.
·
For Distribution Problems:
- ·Reallocate budget to high-ROI channels.
- ·Optimize acquisition funnels (e.g., landing pages).

Bottom Line: If users try but don’t stay, fix the product. If they stay but you can’t find more, fix distribution.