Browser API Labs
What happens when you stop building pages and start building instruments?
Eight lab prompts. Each proves one browser modality within five seconds of interaction. Copy-paste into any code generation tool.
Prompt Pattern
Every lab prompt follows five parts:
| Part | Job |
|---|---|
| Modality | The one thing being demonstrated |
| Transformation | What input changes what output in real time |
| Fallbacks | Unsupported browsers, permissions, demo mode |
| Legibility | Transcript, oscilloscope, raw event panel — make the API visible |
| Magic moment | One interaction that creates surprise within five seconds |
Meta-Prompt
Reusable template for inventing new labs:
Build a single-page browser lab that showcases [API/modality] as the primary interaction primitive. The experience should prove the modality within 5 seconds of use, include a clear realtime feedback loop between input and output, expose the raw signal visually, handle permissions and unsupported browsers gracefully, and feel like a polished experiment rather than a generic app. Add one magic moment, one fallback mode, and one simple explanation of what the browser is actually doing under the hood.
Voice Shapes Reality
Layer: Input (Speech Recognition + Speech Synthesis)
Core APIs: Web Speech API, Canvas or WebGL
What it proves: Voice as a first-class control surface, not a gimmick.
Build a single-page browser experiment called "Voice Shapes Reality." Use the Web Speech API for speech recognition and speech synthesis, plus Canvas or WebGL for visuals. When the user speaks a command like "make it red," "more chaos," "slow down," or "spawn circles," the scene should change immediately and the system should answer back in a calm synthetic voice explaining what changed. Include a visible transcript, a command log, fallback UI when speech recognition is unavailable, and a reduced-motion mode. The goal is to make voice feel like a first-class control surface rather than a gimmick.
Screen Melt
Layer: Pixels (Screen Capture + MediaRecorder)
Core APIs: getDisplayMedia(), Canvas, MediaRecorder
What it proves: The browser can capture and remix the desktop as live material.
Build a browser lab called "Screen Melt." Use
getDisplayMedia()to capture the user's screen or window, render the live stream onto a canvas, and let the user apply realtime visual transformations like pixel sorting, motion trails, edge detection, zoom tunnels, and color quantization. Add recording support with MediaRecorder, a split view for original versus transformed output, and a safety note explaining permission flow. The point is to show that the browser can capture and remix the desktop as live material.
Living Synth
Layer: Sound (Web Audio)
Core APIs: Web Audio API (OscillatorNode, BiquadFilterNode, AnalyserNode)
What it proves: A browser tab can be a musical instrument with reactive visuals.
Build an experiment called "Living Synth." Use Web Audio to generate sound from oscillators, filters, envelopes, and analyzers, and connect the audio output to animated visuals that react in real time to frequency and amplitude data. Add a small sequencer, keyboard shortcuts, and a modulation panel with controls for tempo, filter cutoff, waveform, and chaos. Make it feel like an instrument, not a settings page.
Physical Interface
Layer: Devices (WebHID)
Core APIs: WebHID API, Canvas, Web Audio
What it proves: Physical hardware can shape browser-native workflows.
Build a browser app called "Physical Interface Lab." Use WebHID to connect to supported HID controllers, knobs, pads, or unusual input devices, and map incoming signals to visual parameters, sound modulation, or navigation states. Show raw input events, let the user map controls to actions, and include a simulated virtual controller for browsers or devices that do not support WebHID. The purpose is to prove that physical hardware can shape browser-native workflows.
Haptic Compass
Layer: Devices + Input (Vibration + DeviceOrientation)
Core APIs: Vibration API, DeviceOrientation, Fullscreen, Web Audio
What it proves: A phone is a physical object, not just a screen.
Build a mobile-first experiment called "Haptic Compass." Use the Vibration API, DeviceOrientation or motion-related device events where available, fullscreen mode, and ambient audio to create a tactile navigation experience. As the user rotates the phone, the interface shifts directionally and vibration patterns confirm alignment, warning states, or discoveries. Include graceful fallback for unsupported devices and a clear permission explanation.
Shared Presence
Layer: State + Pixels (WebRTC + Canvas)
Core APIs: WebRTC (data channels + media), Canvas, Screen Capture
What it proves: The browser is a space of direct shared presence.
Build a two-person browser lab called "Shared Presence." Use WebRTC for peer-to-peer audio/video or data channels, plus a shared canvas where both users can draw, place objects, or trigger effects in real time. Add cursors with names, ephemeral reactions, optional screen-sharing, and a mode where one user controls sound while the other controls visuals. The point is to showcase the browser as a space of direct shared presence, not just chat.
Spatial Thought Room
Layer: Pixels + Input (WebXR + WebGL)
Core APIs: WebXR Device API, WebGL, Pointer Events, Gamepad
What it proves: Information becomes navigable as space.
Build an experiment called "Spatial Thought Room." Use WebXR when available and provide a non-XR 3D fallback with WebGL if not. The app should place notes, ideas, or data points in a 3D room that the user can walk through, connect, cluster, and inspect. Add hand/controller interactions where possible, and preserve a mouse-and-keyboard mode so the concept works on desktop too. The goal is to make information navigable as space.
Folder Mind
Layer: State (File System Access + IndexedDB)
Core APIs: File System Access API, IndexedDB, Web Workers
What it proves: The browser can be a sovereign workspace over real local data.
Build a browser app called "Folder Mind." Use the File System Access API where available, plus IndexedDB for cached metadata, to let a user open a local folder and turn its contents into a live navigable map of files, relationships, and change history. Add drag-and-drop fallback for unsupported browsers, local-only processing, and a visible explanation that no files are uploaded unless the user explicitly exports something. The point is to demonstrate the browser as a sovereign workspace over real local data.
Best Showcase Set
Four labs that prove the browser is a sensorium:
| Lab | Proves |
|---|---|
| Voice Shapes Reality | Perception — speech in and speech out |
| Screen Melt | Media — capture, transformation, recording |
| Physical Interface | Hardware — native browser device interaction |
| Shared Presence | Presence — direct realtime human-to-human collaboration |
That set covers perception, media, hardware, and presence. It tells one story: the browser can listen, see, transform, connect, and respond.
Context
- Browser APIs — Modality checklist and combination patterns
- AI Modalities — Same matrix thinking applied to AI transformations
- Components — React layer that consumes these APIs
Links
- MDN Web APIs — Complete reference for all browser APIs
- MDN Web Speech API Guide — Speech recognition and synthesis walkthrough
- MDN Screen Capture Guide —
getDisplayMedia()usage and permissions - MDN Web Audio Guide — Audio graph, oscillators, filters, analysers
- MDN WebHID Guide — HID device connection and communication
- MDN WebRTC Guide — Peer-to-peer connections, data channels, media streams
- MDN WebXR Guide — VR/AR device access and spatial tracking
Questions
Which lab prompt, when built, would most change how you think about what a browser can be?
- If voice becomes a first-class control surface, which existing apps should have had it all along?
- What happens when you combine two labs — Voice Shapes Reality feeding into Screen Melt?
- Which lab has the shortest path from experiment to shipped product?