Implement stochastic music generation web app with webcam interaction

[Copilot]

This PR implements a complete creative web application that generates music through webcam-based interaction with moving shapes and objects, fulfilling the requirements specified in the problem statement.

Overview

The application transforms the existing Python-based video processing concept into a real-time, interactive web experience that uses computer vision and generative music principles to create a unique musical interface.

Key Features

🎥 Real-time Webcam Integration

• Uses MediaDevices API for live video capture
• Implements motion detection through canvas pixel comparison
• Graceful handling of camera permissions and error states
• Real-time motion area detection with bounding box visualization

🎵 Dynamic Music Generation

• Web Audio API for real-time sound synthesis
• Support for 5 musical scales: Major, Minor, Pentatonic, Blues, and Chromatic
• Polyphonic audio with up to 8 simultaneous voices
• Stochastic elements including random pitch variations and timing
• Built-in reverb and delay effects for rich soundscapes

🎨 Interactive Visual Elements

• Animated geometric shapes (circles, triangles, squares, diamonds) on HTML5 Canvas
• Collision detection between user movement and shapes
• Dynamic particle effects on collision
• Smooth animations with pulsing, rotation, and color changes
• Stochastic movement patterns for organic feel

🎛️ User Interface

• Minimalistic, responsive design that works on desktop and mobile
• Real-time status indicators for camera, motion, and audio
• Adjustable motion sensitivity slider
• Music scale selector with instant switching
• Intuitive start/pause controls

Technical Implementation

The application is built with pure web technologies:

• HTML5: Semantic structure with canvas elements for rendering
• CSS3: Modern styling with gradients, backdrop-filter, and responsive grid layout
• JavaScript ES6+: Modular architecture with separate concerns:
  • camera.js: Webcam handling and motion detection using frame differencing
  • shapes.js: Shape management, animation, and collision detection
  • audio.js: Web Audio API sound synthesis with effects chain
  • script.js: Main application coordination and UI management

Motion Detection Algorithm

The motion detection uses a custom implementation that:

1. Captures video frames to canvas at 30fps
2. Compares pixel brightness differences between consecutive frames
3. Uses flood-fill algorithm to identify connected motion regions
4. Filters motion areas by size and creates bounding boxes
5. Provides relative coordinates for device-independent collision detection

Audio Architecture

The sound system features:

• Multi-oscillator voices (sine, triangle, sawtooth) for rich timbres
• ADSR envelope shaping for natural attack and decay
• Real-time effects processing (reverb, delay, filtering)
• Scale-aware note mapping with octave spanning
• Ambient generative layers for atmospheric background

Testing

The application was tested in a browser environment and demonstrates:

• ✅ Proper initialization and UI rendering
• ✅ Graceful camera permission handling
• ✅ Scale selection functionality
• ✅ Debug functions for audio testing
• ✅ Responsive design across screen sizes
• ✅ Error handling for unsupported browsers

Browser Compatibility

• Chrome 66+ (recommended)
• Firefox 60+
• Safari 14+ (iOS Safari supported for mobile)
• Requires HTTPS in production for camera access

Usage

Simply open index.html in a modern browser, grant camera permissions, and start moving to create music through natural body interaction with the animated shapes.

This implementation transforms the repository from a Python-based video processing tool into an accessible, creative web application that anyone can use to explore the intersection of movement, visuals, and generative music.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.