ETL Data Tool — Pictures
Pre-flight maps for the data acquisition pipeline.
Maps
| Map | Question | Status |
|---|---|---|
| Outcome Map | What does success look like? | Complete |
| Value Stream Map | Where does time die? | Complete |
| Dependency Map | What must happen first? | Complete |
| Capability Map | What can we actually do? | Complete |
| A&ID | How do agents orchestrate? | Complete |
Key Finding
The three-layer architecture (Structured APIs → AI Enrichment → Trust Scoring) emerged from the research. NZBN API is the authoritative foundation. Crawl4AI enriches. Trust scoring validates. Each layer has a known cost ($0) and known trust level.
Pictures Bridge
| Map | Key Insight | Feeds |
|---|---|---|
| Outcome | 100 NZ businesses, trust-scored, queryable in < 2s | Spec: Quality Targets |
| Value Stream | NZBN (bulk) → CO (directors) → Crawl4AI (enrich) → Score → Load | Spec: Build Order |
| Dependency | NZBN API key is the only external gate. Everything else is open-source. | Spec: Sprint 0 |
| Capability | Repos exist (73 types), ETL CLI exists, connectors exist. Gap = acquisition layer. | Spec: Build Ratio |
| A&ID | ETL Agent (extracts) + Trust Instrument (scores) + Schedule Controller (triggers) | Spec: Agent-Facing Spec |
Context
- PRD Index — ETL Data Tool
- Prompt Deck — 5-card pitch
- Spec — Engineering depth
Questions
What is the most important visual missing from the etl data pipelines picture set — and why does it matter?
- Which relationship between elements in this diagram is most underspecified — and what would happen if it were wrong?
- If this picture were shown to a new engineer on day one, what would they misunderstand — and how should the picture be changed?
- What assumption does this visual make that should be made explicit in the spec?