diff --git a/README.md b/README.md new file mode 100644 index 0000000..9fadc1b --- /dev/null +++ b/README.md @@ -0,0 +1,282 @@ +# HandConnect AR + +A browser-based augmented reality experience that tracks your hands in real time using your webcam and renders interactive visual effects on top of the camera feed. No installation, no dependencies to install -- open the HTML file in any modern browser and go. + +   + +--- + +## Quick Start + +1. Download `index.html` +2. Open it in Chrome or Edge *(recommended -- best WebGL + camera performance)* +3. Click **Initialize** +4. Grant camera permission when prompted +5. Hold your hands up in front of the camera and explore + +> **Safari note:** Works but may require enabling camera permissions manually in System Settings -> Privacy -> Camera. + +--- + +## How It Works + +HandConnect uses [MediaPipe Hands](https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker/web_js) to detect up to two hands and track 21 landmark points on each. Every frame, landmark positions drive visual effects rendered across a stack of HTML5 canvas layers. Audio is generated via the Web Audio API. Nothing leaves your device -- all processing is local. + +--- + +## Features + +### Hand Tracking + +Detects up to **2 hands simultaneously** with 21 landmarks per hand. The HUD (top-left) shows live stats: + +| Stat | Description | +|---|---| +| **HANDS** | Number of hands currently detected | +| **FPS** | Frames per second | +| **GESTURE** | Current detected gesture (OPEN, FIST, PINCH) | +| **SPREAD** | How open/closed the hand is (0-100%) | + +--- + +### Visual Effects + +All effects can be toggled independently from the settings panel. + +#### Matrix Rain +Animated falling katakana characters scroll down the background. Speed increases when you move your hands quickly. + +#### Hand Skeleton +Neon glowing lines trace the bones and joints of each detected hand. The default render now uses a brighter layered glow, and you can stack additional styles from the settings panel for more dramatic looks. + +#### Skeleton Styles + +Five optional skeleton looks can be enabled individually or layered together from **Settings -> Skeleton Styles**. + +| Style | Look | +|---|---| +| **Hologram** | Cyan/magenta ghosting with pulsing fingertip rings | +| **Constellation** | Dotted light bridges with star-like joint flares | +| **Circuit** | Segmented neon rails with traveling signal pulses | +| **Mystic Sigil** | Palm-centered rings and ritual geometry | +| **X-Ray Plasma** | Bright inner bone shafts with icy plasma glow | + +The styles stack, so you can test them one at a time or combine them for denser visuals. + +Suggested combinations: + +- **Hologram + Circuit** for a polished sci-fi AR look +- **Constellation + Mystic Sigil** for something more ceremonial and atmospheric +- **Hologram + X-Ray Plasma** for the brightest, most dimensional hand read + +#### Fingertip Sparks +Particle sparks emit from each fingertip as you move. Particles have physics -- gravity, drag, and fade over time. + +#### Lightning Arcs +When the fingertips of both hands get within ~150px of each other, electric arc bolts jump between them. + +#### Mandala +With two hands in frame, a rotating geometric mandala pattern is drawn connecting fingertip positions across both hands. + +#### Pinch -> Shockwave +Bring your **thumb and index finger together** to trigger a ripple shockwave at the pinch point plus an electric zap sound. + +--- + +### Themes + +Eight color themes control the palette of all effects. Switch themes from the bar at the bottom of the screen or from the settings panel. + +| Theme | Vibe | +|---|---| +| **Neon** | Cycling purple -> cyan, electric and fast | +| **Vapor** | Soft pastel lavender and rose | +| **Ocean** | Blue-cyan shifting waves | +| **Synthwave** | Hot pink alternating with electric blue | +| **Toxic** | Acid green and chartreuse | +| **Inferno** | Deep orange to near-white-hot | +| **Ice** | Cool blues and whites, slow breathing pulse | +| **Cyberpunk** | Hard red and cyan, high contrast | + +The active theme also updates the UI accent color throughout the settings panel in real time. + +--- + +### Camera Filters + +11 filters that process the camera feed. Select from the **Camera Filter** section in the settings panel. + +| Filter | Description | +|---|---| +| **Normal** | Default camera with slight brightness/contrast adjustment | +| **Grayscale** | Black and white | +| **Sepia** | Warm vintage tone | +| **Invert** | Color-inverted feed | +| **Hyper Color** | Extreme saturation boost | +| **Noir** | High-contrast black and white with crushed blacks | +| **Matrix** | You rendered as glowing green characters -- see below | +| **Thermal** | Heat-map palette: blue (cold) -> green -> yellow -> red (hot) | +| **Pixelate** | 14x14px block averaging | +| **Glitch** | RGB channel split with random horizontal tears | +| **Comic** | High saturation + halftone dot overlay | + +--- + +### Matrix Filter + +The Matrix filter makes you appear to be constructed entirely from falling green characters, similar to the visual effect from the films. + +**How it works:** The camera image is sampled each frame, the luminance of each pixel is inverted (so dark areas like you become bright, and bright areas like walls become dark), and then a grid of katakana characters is drawn where brightness of each character corresponds to the inverted luminance. A separate layer of falling "rain" leader characters is composited on top. + +#### Matrix Tuner Panel + +When the Matrix filter is active, a **Matrix Tuner** panel appears at the bottom-left of the screen with 6 live sliders. All changes apply instantly. + +| Slider | What It Controls | Tip | +|---|---|---| +| **Dark Cut** | How aggressively the background is eliminated | Raise if your background bleeds through; lower if you're disappearing | +| **Edge Fade** | Softness of the transition at your silhouette edge | Higher = softer, more cinematic edge | +| **Contrast** | Gamma curve -- controls punch between dim and bright zones | Lower = more dramatic contrast | +| **Char Size** | Size of each character cell in pixels | Smaller = more detail; larger = chunkier movie look | +| **Rain Trails** | How long falling rain streaks persist | Lower value = longer ghosting trails | +| **Rain Density** | How many falling leader characters appear per frame | Higher % = heavier rain | + +Click **Reset Defaults** to restore all sliders to their baseline values. + +> **Lighting tip:** The Matrix filter works best when you are darker than your background. If results are inverted (background shows as characters, you appear as holes), sit in front of a brighter wall or increase the Dark Cut slider. + +--- + +### Creative Features + +Five additional creative modes, all off by default. Enable them from **Settings -> Creative**. + +#### Finger Piano +Each fingertip plays a different musical note when you tap downward: + +| Finger | Note | +|---|---| +| Thumb | C4 - 261 Hz | +| Index | E4 - 329 Hz | +| Middle | G4 - 392 Hz | +| Ring | A4 - 440 Hz | +| Pinky | C5 - 523 Hz | + +A floating note label rises from the triggering finger, and the note name flashes in the center of the screen. Both hands play independently, so you can chord. + +#### Ghost Trails +Your last 12 hand positions are stored each frame and drawn as fading skeleton echoes behind your current hands, creating a motion-blur ghost effect. + +#### Fireworks +Raise an **open hand quickly upward** and a firework launches from your palm. It arcs up and explodes into a burst of 80-120 colored sparks with physics (gravity, drag). There is a 1-second cooldown between launches. + +#### Aura Field +A soft radial glow radiates from each palm center. The aura size pulses with your movement speed -- move fast and it expands, hold still and it contracts. Rendered in screen blend mode for a natural neon-light look. + +#### Kaleidoscope +After each frame renders, the canvas is mirrored three ways (horizontal, vertical, both) and composited back, creating a 4-way symmetric pattern from whatever effects are active. Works especially well combined with Ghost Trails and Aura Field. Automatically pauses below 26 FPS to protect performance. + +--- + +### Draw Mode + +Enable **Paint w/ Index Finger** from the settings panel. Your index finger on either hand will paint a persistent glowing trail that stays on screen. Trails use the current theme color. + +- Works with both hands simultaneously +- Trails persist until you click **Clear Canvas** +- Switch themes to change the paint color mid-session + +--- + +### Audio + +Three independently controllable audio layers: + +| Control | Description | +|---|---| +| **Master Audio** | Global on/off -- smoothly ramps gain to avoid clicks | +| **Ambient Hum** | A sine tone that activates when two hands are detected. Pitch rises as your hands move closer together. | +| **Pinch Zap** | A sawtooth burst sound that fires on every pinch gesture. | + +All audio is generated locally via the Web Audio API. No audio files are loaded. + +--- + +## Settings Panel Reference + +Open with the gear button (top-right). Click again or click outside to close. + +```text +SETTINGS +|- DISPLAY +| |- Stats HUD - show/hide the stats panel (top-left) +| `- Theme Bar - show/hide the theme switcher (bottom) +|- EFFECTS +| |- Matrix Rain - animated background characters +| |- Hand Skeleton - glowing joint/bone overlay +| |- Fingertip Sparks - particle emitters at fingertips +| |- Lightning Arcs - bolts between close fingertips +| `- Mandala - rotating geometric pattern (2 hands) +|- SKELETON STYLES +| |- Hologram - cyan/magenta ghosted skeleton +| |- Constellation - star-map joints and dotted bridges +| |- Circuit - segmented neon traces with moving pulses +| |- Mystic Sigil - palm circles and occult geometry +| `- X-Ray Plasma - bright bone shafts and plasma nodes +|- CREATIVE +| |- Finger Piano - notes on finger tap +| |- Ghost Trails - motion echo ghosts +| |- Fireworks - palm-launch particle bursts +| |- Aura Field - palm glow field +| `- Kaleidoscope - 4-way canvas mirror +|- DRAW MODE +| |- Paint w/ Index Finger +| `- Clear Canvas +|- AUDIO +| |- Master Audio +| |- Ambient Hum +| `- Pinch Zap +`- CAMERA FILTER + `- [11 filter buttons] +``` + +--- + +## Browser Compatibility + +| Browser | Status | Notes | +|---|---|---| +| Chrome / Edge | Yes - Recommended | Best performance | +| Firefox | Yes | Slightly lower FPS | +| Safari | Yes | Enable camera in System Settings if blocked | +| Mobile Chrome | Partial | Works but performance varies by device | + +--- + +## Performance Tips + +- **FPS dropping?** Turn off Kaleidoscope first -- it reads the canvas every frame +- **Testing skeleton looks?** Start with Hologram alone, then add one extra style at a time so you can feel what each layer contributes +- **Hands not detecting?** Ensure good lighting on your hands. Avoid backlit situations +- **Matrix filter laggy?** Increase Char Size in the Matrix Tuner (larger grid = fewer cells to process) +- **Too many effects at once?** Kaleidoscope + Ghost Trails + Aura Field is beautiful but GPU-heavy -- disable one if needed + +--- + +## Technical Stack + +| Technology | Usage | +|---|---| +| [MediaPipe Hands](https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker/web_js) | Hand detection and landmark tracking | +| HTML5 Canvas (5 layers) | Rendering pipeline | +| Web Audio API | Synthesized sound effects and ambient tone | +| CSS Custom Properties | Live theme color propagation | +| Google Fonts | Space Mono + Syne | + +No build tools, no framework, no npm. Single HTML file. + +--- + +## Credits + +Original project by [Axshatt](https://github.com/Axshatt/HandConnect). Feature expansion and documentation by [waynefullam](https://github.com/waynefullam). diff --git a/index.html b/index.html index fd10f19..5509655 100644 --- a/index.html +++ b/index.html @@ -1,723 +1,1269 @@
- - -