An attempt at parsing sheet music for guitar using computer vision. The goal of this project is two-fold. First, an image of a page of sheet music (for guitar) is processed using the OpenCV computer vision library to ultimately generate an ordered list of musical notes. These notes are then arranged into usable fretboard locations, seeking to maximize playability for a solo guitarist, and transcribed into tablature format.
Python 3.7, plus three external libraries, are required to use this project:
OpenCV, version 3.3.1 or greater, but recent development is done using 4.0.1
Numpy, for manipulating the arrays generated by OpenCV
Matplotlib, for displaying the results of various stages of the program
Linux and Windows environments should both work, but MacOS has not been tested.
Currently, a given image can be separated into individual staffline regions, and the presence and location of notes can be identified within them, with moderate accuracy. Red represents the boundaries of the staff itself, and green its overall area of influence. Detected notes are circled in blue.
Detected notes can be assigned musical names and octave values based on their location on the staff, but key signatures and accidentals are yet to be accounted for.
A manually selected group of notes can be arranged into a statically playable shape, if possible, and a series of notes and chords can be arranged into a dynamically playable pattern of frets. This takes into account the notes prior and subsequent to each given note to maximize overall playability, with mixed, but interesting, results.
A series of fretboard locations can be transcribed into tablature, preserving as much timing info as possible, although some is always lost with tab compared to proper sheet music.
Images can be directly transcribed into tablature, although overall accuracy is poor. Chosen fingerings are not always sensible, and the timing is all wrong; every note is assumed to be a straight quarter note, and time signatures are ignored entirely. The detector also often fails to detect every note present, and may detect notes that don't exist. Naturally, these are included in the transcription anyway.
detect.py will attempt to process any .png images in its directory. It will display detection results in a matplotlib figure, then print its transcription to the terminal, which should be run in interactive mode, i.e. 'python -i detect.py'. Accuracy of detection, although incomplete, is greater with higher resolution images with full length staffs. Transcription accuracy remains poor.
test_detect.py will do the same for the images in the 'static/' directory of the repo.