Testing Infrastructure

Where do your tests run — and what does that cost?

COMMIT → TYPECHECK → UNIT/INTEGRATION → PREVIEW DEPLOY → E2E → SHIP
  │          │              │                  │            │       │
  ▼          ▼              ▼                  ▼            ▼       ▼
 Local     <1s            5-30s            Vercel builds  30-120s  Production
 Free      Free           DB required      Free tier      Browser  Monitored

Testing infrastructure is the machinery that runs tests — CI pipelines, databases, runners, preview environments. Get this wrong and you pay in flaky tests, blocked terminals, or expensive CI bills. Get it right and every push gets verified automatically against a real deployment.

The Core Principle

Test against deployed artifacts, not dev servers.

Running E2E tests against localhost means port conflicts, RAM contention, and the false choice between "dev server running" or "run tests." Preview deploys eliminate this entire class of problems. A real URL. Real infrastructure. No local resource cost.

Pipeline Design

Two pipelines. Different triggers, different budgets.

Local Pipeline (every commit)

Fast, free, catches 90% of issues.

tsc --noEmit → vitest run → done
     │              │
   Types OK?    Unit + Integration pass?
   <10s         <60s

Budget: Under 90 seconds total. If it exceeds this, tests will be skipped. Set a hard ceiling and enforce it.

Requirements:

• Test database running (Docker container on a dedicated port)
• No dev server needed — unit and integration tests don't use the browser
• Memory-safe typecheck — use affected-file checking, not full project

Command	Purpose	Trap to Avoid
tsc --noEmit or pnpm tc	Type verification	Full project typecheck can OOM on large monorepos. Use affected-only.
vitest run or jest --bail	Unit + integration	--bail stops on first failure. Fast feedback.

CI Pipeline (every PR)

Thorough. Runs against the preview deployment. Catches integration issues.

Install → Typecheck → Unit/Integration → Wait for Preview → E2E → Report
                                              │
                                    Vercel builds preview URL
                                    from the PR branch

The key insight: Vercel gives you a preview URL on every push for free. The E2E tests just need to point at that URL instead of localhost.

Preview Deploy Testing

The single highest-leverage infrastructure change. Decouples test environment from development environment.

How It Works

git push → Vercel builds preview → CI runs Playwright against preview URL
                │                          │
          Free on all plans         Tests hit real infrastructure
          Unique URL per commit     No local server needed

Playwright Configuration

One config change makes Playwright work against any URL:

// playwright.config.ts
export default defineConfig({
  use: {
    baseURL: process.env.BASE_URL || 'http://localhost:3000',
  },
  // Only start local server when NOT in CI
  webServer: process.env.CI ? undefined : {
    command: 'pnpm dev',
    url: 'http://localhost:3000',
    reuseExistingServer: true,
  },
});

Locally: Playwright starts the dev server. In CI: BASE_URL points to the Vercel preview. Same tests, different target.

Deployment Protection

If Vercel has deployment protection enabled, Playwright requests hit an auth page instead of the app. Solutions:

Approach	Cost	Trade-off
Disable protection for previews	Free	Previews are publicly accessible
Automation bypass header	Pro plan ($20/mo)	Previews stay protected, CI gets a bypass token
Password protection with env var	Pro plan	Playwright sends the password in a setup step

For most teams, disabling protection on preview deployments is fine — they're ephemeral and the URLs are unguessable.

Test Database Infrastructure

Integration tests (L2) need a real database. Not a mock. Mocking the database hides the bugs that matter — constraint violations, transaction behavior, query performance.

Docker Test Database

Dedicated container, dedicated port, isolated from development:

Concern	Approach
Isolation	Separate port (e.g., 5433) from dev database
Startup	docker compose up with health check, wait for ready signal
Cleanup	Each test cleans its own data. No shared state between tests.
Reset	Drop and recreate between test suites if needed
CI	Service container in GitHub Actions / self-hosted runner

Port Allocation

Dedicated ports prevent collisions between dev and test:

Service	Dev Port	Test Port
Database	5432	5433
App server	3000	4300
API	3001	— (use preview URL)

Cost Management

Browser testing in CI is expensive. Playwright with 3 browsers for 5 minutes, 20 runs/day on GitHub Actions: ~$50/month. Here's how to reduce that.

Reduce What Runs

The testing strategy page covers the layer model. The infrastructure implication: every E2E test you convert to an integration test saves 30-120 seconds of CI browser time. At scale, this is the biggest cost lever.

Action	Savings
Convert server-action E2E → integration test	~60s per test
Run only affected tests (nx affected)	Skip unchanged projects entirely
Bail on first failure in PRs	Stop wasting time after a break
Parallelize with sharding	Same wall-clock time, more runner-minutes, but faster feedback

Cheaper Runners

GitHub Actions minutes are premium-priced. Alternatives for browser-heavy workloads:

Provider	Model	Relative Cost
GitHub Actions	Per-minute, managed	Baseline (1x)
Self-hosted runner	Your machine, free compute	Free (your electricity + RAM)
Ubicloud	Drop-in replacement, bare metal	~0.3x
RunsOn	Your AWS account, spot instances	~0.1-0.15x
Blacksmith	Managed, faster builds	~0.5x

Cheapest viable path: Self-hosted runner on a spare machine for private repos. Ubicloud for open source.

Simplest path: Run E2E locally against preview URLs. Zero CI minutes for browser tests. The preview URL is the infrastructure — BASE_URL=https://your-preview.vercel.app npx playwright test.

NX and Caching

Monorepo build tools provide two cost-saving mechanisms:

Mechanism	What It Does
Affected commands	Only test projects changed by the PR. Skip everything else.
Computation caching	If inputs haven't changed, reuse previous test results.

In CI: nx affected -t test instead of nx run-many -t test. On a 10-project monorepo, this typically skips 60-80% of test runs.

Preflight Checklist

Before any test work:

• Test database running and healthy
• Correct environment identified (local vs CI vs preview)
• Known issues checked (issue log, failing tests)
• Test type identified → correct layer selected

Never Do

Action	Why	Alternative
Full-project typecheck on large monorepos	OOM crash	Affected-only typecheck
npm run dev in CI	Blocks the runner	Build + serve, or use preview URL
Skip database readiness check	Tests fail silently with connection errors	Always wait for ready signal
Claim tests pass without running them	Breaks trust	Evidence or nothing
Arbitrary sleep() in E2E tests	Flaky, slow	waitForSelector(), waitForResponse()

Feedback Loop

Tests are instruments. They measure the gap between intent and reality.

TEST FAILURE
     │
     ├─► Implementation gap → fix the code (engineering concern)
     │
     └─► Vision gap → update the spec (product concern)

Every failure is classified: is the code wrong, or is the spec wrong? This distinction routes feedback to the right team and prevents the cycle of fixing code to match a broken spec.

Log failures with evidence: test file, line number, expected vs actual. Structured feedback compounds — pattern recognition across failures reveals systemic issues that individual fixes miss.

Context

• Testing Strategy — Layer model, selection rules, hexagonal testing
• Testing Tools — Vitest, Jest, Playwright, RTL
• Monorepo Build Tools — NX affected commands, caching
• Dev Environment — Docker, containers, isolation
• Deployment Checklist — What happens after tests pass
• DevOps — CI/CD, security, git practices