Tech Review Process
Which gap in the capability register justifies this tool — and can you prove it?
A tool evaluated in isolation is a toy. Anchored to a feature-matrix gap with scored evidence, it becomes a decision. This process closes the loop: gap → candidates → verdict → updated matrix.
Five Gates
Every tech review runs five gates in order. No skipping.
| # | Gate | Job | Output |
|---|---|---|---|
| 0 | ANCHOR | Find the maturity gap first. State the JTBD. State the workaround cost. | Capability ID(s), JTBD statement, workaround cost |
| 1 | SOURCE | Find 3–5 candidates using sources matched to the type. Record where each came from. | Candidate list with source URL |
| 2 | CLASSIFY | Software vs Hardware vs Hybrid. Routes to the correct scoring path. | Classification + scoring path |
| 3 | SCORE | Apply 5-dimension scoring matrix. Each 1–5 with quoted evidence. | Score table + geometric mean |
| 4 | VERDICT | Map composite to Buy / Build / Hybrid / Defer. | Verdict + rationale |
| 5 | UPDATE | Write verdict back to the capability register. Save receipt. | Updated docs + receipt |
Gate 0 is a hard stop. No sourcing without a named capability ID.
Software Scoring
Five dimensions. Geometric mean. Score each 1–5 with evidence — "good fit" fails.
| Dimension | Score 1 | Score 3 | Score 5 |
|---|---|---|---|
| JTBD Fit | Covers <30% of job | Covers core, gaps in edge cases | Covers 100% including edge cases |
| Data Sovereignty | Vendor owns all data, no export | Export available, some lock-in | We own all data, open formats, no lock-in |
| Team Capability | Requires skills we don't have | Some upskilling needed | Team can run it today |
| TCO (3yr) | >$50K/yr | $10–50K/yr | <$10K/yr |
| Build Cost | Build would take <2 weeks | Build would take 1–3 months | Build would take >3 months |
Verdict thresholds:
| Composite | Data Sovereignty | Verdict |
|---|---|---|
| ≥3.5 | ≥3 | Buy |
| ≥3.5 | <3 | Build |
| 2.5–3.5 | Any | Hybrid — pilot before committing |
| <2.5 | Any | Defer |
Hardware / DePIN Scoring
Five dimensions. Geometric mean. Never buy without demand side ≥ 2.
| Dimension | Score 1 | Score 3 | Score 5 |
|---|---|---|---|
| Operator ROI | >36mo payback | 24–36mo payback | <24mo payback |
| Network Health | <100 nodes | 100–1,000 nodes | >1,000 nodes, geographic spread |
| Token Sustainability | Unlimited emissions, no demand | Decreasing emissions, no demand driver | Decreasing emissions + usage-driven demand |
| Demand Side | No identifiable buyer | Pilot buyers only | Enterprise buyers + growing demand |
| Exit | Hardware worthless if network fails | Some resale value | Strong secondary market |
Verdict thresholds: ≥3.5 → buy. <3.0 → defer.
Sourcing by Type
| Type | Primary Sources | Secondary Sources |
|---|---|---|
| SaaS / AI-native | Product Hunt, G2, a16z/YC portfolios, Perplexity | Competitor stack pages, LinkedIn job ads |
| OSS | GitHub trending, CNCF landscape, Hacker News | Awesome lists, pkg.go.dev, npmjs |
| DePIN devices | DePINscan.io, DePINhub.io | Protocol Discord/Twitter, node sale announcements |
| Robotics | NVIDIA Isaac ecosystem, Boston Dynamics partners, FrodoBots network | arXiv embodied AI, YC robotics batch |
| AI compute | io.net, Akash, Render, Aethir | GPU node announcements, Twitter lists |
POS Stackmates
Retail POS is not only checkout software. It is the edge system where payments, stock, customer response, loyalty, tax, and finance evidence become business truth.
Use this frame when POS software is the stackmate under review. The provider choice should be judged by the retailer data footprint it creates and the LLM harness it can support, not only by terminal price or screen polish.
Anchor To Model
Start with the retailer's business model, not the vendor demo.
- Discount retail: price trust, catalogue rhythm, SKU availability, margin exposure, and fast buying loops.
- Specialty retail: assortment depth, service quality, customer memory, returns, and replenishment timing.
- Hospitality: table flow, modifiers, kitchen display, tipping, service charges, and shift reporting.
- Services: appointments, packages, memberships, staff calendars, and usage tracking.
Gate 0 question: which revenue, inventory, or customer-decision loop gets faster if this POS works?
Model TCO
Treat total cost as a constraint, not a target.
- Hardware: terminals, tablets, cash drawers, receipt printers, scanners, and refresh cycles.
- Software: base subscription, registers, locations, advanced inventory, analytics, loyalty, and support tiers.
- Payments: blended fee at current average order value, gateway costs, chargebacks, settlement timing, and payout reconciliation.
- Change cost: migration, training, dual-running, lost floor time, and internal support overhead.
Output: a three-year TCO model plus the operating friction it removes or creates.
Score Fit
Score the candidate against the actual store flows.
- Time-to-competence: a new team member can complete 80% of common checkout, return, discount, and split-payment tasks within one shift.
- Integration map: accounting, e-commerce, inventory, CRM or loyalty, analytics, and any ordering portal have native sync, API, webhook, or reliable export paths.
- Inventory ledger: variants, transfers, purchase orders, stocktakes, adjustment reasons, movement logs, kits, and location stock are visible without spreadsheet offsets.
- Payment flow: cards, contactless, wallets, gift cards, cash, refunds, settlement reports, and PCI-aligned security are simple for staff and customers.
- Scale path: new locations, registers, users, role permissions, support coverage, and offline mode are proven before rollout.
Discard any option where a critical flow scores below 3 out of 5.
Demand Proof
Before signing a long contract, demand a small proof.
- Can the POS produce clean transaction, SKU, store, tender, timestamp, return, tax, and margin fields?
- Can sales subtract stock in near real time across store and online channels?
- Can accounting reconcile tenders, refunds, tax codes, and payouts without manual repair?
- Can one customer-response signal connect to a receipt or sale event with consent and privacy controls?
- Can the vendor explain the Saturday-peak failure plan: offline mode, recovery, support path, and data replay?
The verdict should be a one-page POS option canvas: context, three-year TCO, fit scores, integration map, risks, kill signals, and "why this, why now."
The strategic question after the provider decision is: what harness gets built around the POS events so LLMs can explain variance, draft buyer briefs, surface stock exceptions, and recommend bounded tests without inventing truth?
Hardware Strategy
Hardware strategy is simple: buy. We don't manufacture DePIN devices or robots — we operate them.
The evaluation question is: does this device earn its cost in a reasonable time, on a network that won't collapse?
Apply the five hardware dimensions above. If any of these are true, stop:
- Demand side < 2 — no buyer for the data/service
- Network health < 1 — launch-phase protocol, unproven
- Token emissions still climbing — the reward schedule is inflationary
See DePIN devices for the current hardware inventory.
Software Strategy
Software strategy: engineer what we need, buy commodity.
The distinction is data sovereignty. If the tool touches data that trains our competitive advantage, we own it. If it's a commodity pipe, rent is fine.
The build cost dimension accounts for AI-assisted development. A feature that took months in 2022 takes weeks now. This shifts more tools into "build" territory — update scores quarterly as capability improves.
See Buy or Build for the full sovereignty framework.
Closing the Loop
Every verdict feeds back:
| Verdict | Action |
|---|---|
| Buy | Tool added, capability record updated with adopted-tool evidence |
| Build | Capability spec created, capability record stays unresolved until capability ships |
| Defer | Documented with trigger criteria for re-evaluation |
Receipts accumulate in the receipt store. Run quarterly per category, or on-demand when a venture activates a new vertical.
Failure Modes
- Demo-first review: sourcing starts before Gate 0 names the capability gap, so the team buys polish instead of evidence.
- Single-axis score: feature fit hides weak data sovereignty, poor exit options, or a build cost that has changed since the last review.
- Receipt-free verdict: the decision never updates the capability register, so the same category gets re-litigated next quarter.
- Hardware optimism: network health and demand-side proof are accepted from protocol marketing instead of independent operator evidence.
Context
- Capability matrix — maturity gaps are the starting point for every review
- Buy or Build — Sovereignty framework and applied verdicts
- All RaaS Functions — Function catalog with IDs and ROI scores
- DePIN Devices — Current hardware inventory and evaluation history
- RFP Process — Procurement lifecycle from need to sign-off
- Horizontal SaaS — JTBD specs with top products per category
Zoom Out
- Up: Applications — the hub that frames software-selection decisions.
- Across: Build or Buy — apply the sovereignty framework once the tech review verdict is in.
Questions
What makes a tech review different from a product demo — and what evidence would a demo never show you?
- If Gate 0 requires a named capability ID before sourcing, what does that imply about the order of discovery — market-first or gap-first?
- When build cost drops because AI coding improves, which previously-bought tools cross the threshold into "build" territory first?
- A tool scores 4.0 on JTBD fit but 1.5 on data sovereignty — what does the verdict tell you that the feature score doesn't?
- Which hardware dimension is most likely to be faked in a protocol's marketing materials — and how would you verify it independently?