Prediction Process

How will I know if I was right or wrong?

Prompts

For every prediction you make:

1. "What's my conviction level (0-5)?" — Quantify confidence
2. "What's the time horizon?" — When will this resolve?
3. "How will I know I was right/wrong?" — Define success criteria
4. "What would change my conviction?" — Identify update triggers
5. "What's my position/allocation?" — Match stakes to conviction

The Prediction Record

Field	Question	Example
Prediction	What exactly am I predicting?	"AI coding assistants replace 50% of junior dev tasks"
Time Frame	When will this resolve?	"By end of 2026"
Conviction	How confident am I (0-5)?	4/5
Positioning	What's my stance?	Early adopter, investing in AI tools
Allocation	What resources am I committing?	20% of learning time, $500/year in tools

Source Analysis

• Source: Identify where the prediction originates (e.g., expert opinion, market research, historical analysis).
• Domain: Categorize the prediction into macro-environmental dimensions using frameworks like STEEP (Social, Technological, Economic, Environmental, Political).
• Scope: Assess whether the prediction impacts a niche area or has global, cross-industry implications.

Evaluate Key Drivers

• Trends: Analyze historical trends that support the prediction. Trends are relatively stable and often predictable over time.
• Emerging Issues: Identify nascent signals or weak indicators that could evolve into major drivers (e.g., new technologies, policy shifts).
• Discontinuities: Consider potential disruptions that could alter the trajectory of the prediction (e.g., economic crises, regulatory changes).

Assess Timeframes

Use forecasting models to estimate when predictions might materialize:

• Time Series Models: Analyze historical data to project future trends.
• Scenario Planning: Develop multiple plausible scenarios with varying time horizons to account for uncertainties.
• Expert Judgment (Delphi Method): Aggregate opinions from a panel of experts to refine timelines.

Quantify Confidence Levels

Assign personal conviction scores (e.g., 0/5 to 5/5) based on:

• Quality of evidence supporting the prediction.
• Alignment with broader meta-trends.
• Credibility of the source or expert making the prediction.

Ripple Effects

Map out potential ripple effects of the prediction:

• What industries or systems will be impacted?
• What secondary trends might emerge as a result?

This step can be visualized as a mind map branching out from initial trends to consequences and opportunities.

Strategic Positioning

Define your positioning in response to the prediction:

• Are you an observer, early adopter, or market leader in this space?
• What actions or investments are required to align with this future?

Track Predictions

Continuously monitor real-world developments to validate or adjust predictions:

• Use tools like Google Trends, social listening platforms, and sales data analysis to track signals in real-time.
• Compare past forecasts with actual outcomes to refine your methodology.

Key Trends

For large-scale predictions like those from Framework Ventures (e.g., blockchain adoption, DeFi growth), apply additional filters:

• Market Size Estimates: Validate assumptions about user base growth (e.g., "1-3 billion MAUs on blockchains by 2030").
• Technological Feasibility: Assess whether enabling technologies (e.g., Layer 2 blockchains) can scale within the predicted timeframe.
• Policy and Regulation: Factor in geopolitical and regulatory shifts that could accelerate or hinder progress (e.g., global crypto policy).

Context

• Prioritization Algorithm — Every 5P score is a prediction; integral calibration closes the loop
• Validate Demand — Evidence that grounds predictions in real pain
• Probability — Expectations of outcomes
• Investing — Cultivating narratives and sizing bets
• Perspective — Reality leaves a lot to the imagination
• First Principles — Real world engineering
• The Ledger — Your prediction track record is the book others read
• Capabilities — Run to where the ball is going to be

Links

• ai-2027.com
• Good Judgment Project
• Preparing for the Intelligence Explosion
• The Basics of Game Theory
• Are you a super-forecaster
• Ten Commandments for Super-forecasters

Questions

When you score a PRD's pain at 4/5, what prediction are you actually making — and how would you know you were wrong?

• Which of your past conviction scores turned out most wrong, and what pattern does that reveal about your blind spots?
• If prediction tracking shows your demand scores consistently overestimate, is the rubric wrong or is your evidence gathering weak?
• At what point does a prediction with high conviction but no time horizon become an unfalsifiable belief?