Piano Teacher

A Java-based interactive piano learning system that detects piano notes in real-time and provides visual feedback through LEDs, OLED display, and spectrograms. The system integrates audio processing with Arduino hardware to create an engaging learning experience.

Features

• Real-time Note Detection: Detects piano notes (C4-B4) from microphone input or MP3 files
• Audio Processing: Uses TarsosDSP for pitch detection and signal processing
• Arduino Integration:
  • Controls 7 LEDs corresponding to piano notes
  • Displays note information on SSD1306 OLED display
  • Button input via resistor ladder for interactive control
• Visual Feedback:
  • Real-time spectrogram visualization
  • Frequency domain analysis
  • Time-domain waveform display
• Recording & Playback: Record practice sessions and play them back for analysis
• Calibration System: Calibrate button inputs for accurate note selection

System Architecture

The application consists of several key components:

• Main.java: Application entry point
• PianoNoteApp.java: Main controller coordinating all components
• PianoAudioProcessor.java: Handles audio recording, processing, and note detection
• ArduinoController.java: Manages Arduino hardware communication (LEDs, buttons, OLED)
• VisualizationManager.java: Creates spectrograms and audio visualizations
• NoteEvent.java: Data model for detected musical notes
• PianoNoteDetector.java: Legacy standalone implementation

Hardware Requirements

Arduino Setup

• Arduino board (Uno, Mega, or compatible)
• Arduino Grove Base Shield or equivalent
• 7 LEDs connected to digital pins 2, 3, 7, 8, 9, 10, 11
• 7-button resistor ladder connected to analog pin A0
• SSD1306 OLED display (128x64) via I2C (address 0x3C)
• StandardFirmata firmware loaded on Arduino

Voltage Configuration

Button resistor ladder voltage levels:

• Button 1 (C4): 2.5V
• Button 2 (D4): 3.3V
• Button 3 (E4): 3.7V
• Button 4 (F4): 4.0V
• Button 5 (G4): 4.1V
• Button 6 (A4): 4.2V
• Button 7 (B4): 4.3V

Software Dependencies

• Java 8+
• TarsosDSP: Audio processing and pitch detection
• Firmata4j: Arduino communication via Firmata protocol
• Apache Commons Math3: FFT and signal processing
• JFreeChart: Data visualization and charting
• Java Sound API: Audio input/output

Installation

1. Clone the repository

   git clone https://github.com/jazzhatcoder/Piano-Teacher.git
   cd Piano-Teacher

2. Install StandardFirmata on Arduino

   • Open Arduino IDE
   • File → Examples → Firmata → StandardFirmata
   • Upload to your Arduino board

3. Add required JAR dependencies Ensure the following libraries are in your classpath:

   • TarsosDSP-x.x.jar
   • firmata4j-x.x.jar
   • commons-math3-x.x.jar
   • jfreechart-x.x.jar
   • jcommon-x.x.jar

4. Compile the project

   javac -d bin src/*.java

5. Run the application

   java -cp bin Main

Usage

Getting Started

1. Connect Arduino: Plug in your Arduino board via USB
2. Launch Application: Run Main.java
3. Calibrate Buttons (first time setup):
   • Click the "Calibrate" button
   • Follow on-screen instructions to press each button
   • Calibration data is saved to button_calibration.txt

Recording and Detection

1. Record Audio:

   • Click "Start Recording"
   • Play piano notes (C4-B4 range)
   • Click "Stop Recording"

2. Load MP3 File:

   • Click "Load MP3"
   • Select an MP3 file containing piano notes
   • The system will process and detect notes

3. Play Detected Notes:

   • After detection, click "Play"
   • LEDs light up corresponding to detected notes
   • OLED display shows current note information
   • Visual spectrogram displays frequency analysis

Interactive Mode

• Press physical buttons on the Arduino to manually trigger specific notes
• LEDs provide immediate visual feedback
• OLED displays the selected note name and frequency

Audio Processing Details

• Sample Rate: 44.1 kHz
• Buffer Size: 1024 samples
• Overlap: 768 samples (75%)
• Frequency Range: C4 (261.63 Hz) to B4 (493.88 Hz)
• Frequency Tolerance: ±15 Hz
• Minimum Note Duration: 0.2 seconds
• Pitch Detection: YIN algorithm via TarsosDSP

Visualization

The application provides two types of spectrograms:

• Raw Audio Spectrogram: Shows the unprocessed frequency content
• Filtered Audio Spectrogram: Displays filtered audio focusing on piano note frequencies

Spectrograms use Fast Fourier Transform (FFT) with:

• Window size: 512 samples
• Hop size: 256 samples
• Multi-threaded processing for performance

Testing

Unit tests are provided for key components:

• ArduinoControllerTest.java: Tests Arduino communication and hardware control
• PianoAudioProcessorTest.java: Tests audio processing and note detection

Run tests using your preferred Java testing framework.

Acknowledgments

• TarsosDSP: Excellent audio processing library
• Firmata4j: Seamless Arduino integration
• Apache Commons Math: Robust mathematical functions
• JFreeChart: Powerful visualization capabilities