Audio Transcription & Sheet Music Tool

Project Summary

This desktop application converts multi‑instrument audio into clean, printable sheet music. It integrates Magenta’s Onsets and Frames and DrumRNN transcription models with Spleeter for source separation, pretty_midi for quantization, and MuseScore for score generation. Users can separate stems (e.g., piano, drums), transcribe them into MIDI, and export quantized scores — all through an accessible tkinter-based GUI with drag‑and‑drop support.

Developed as my Computer Engineering senior project at Cal Poly SLO, the tool is designed for musicians, educators, and developers interested in AI-assisted music transcription.

System Design & Workflow

Audio Import & Conversion — Supports MP3, WAV, FLAC. Converts to 16kHz mono WAV using ffmpeg for model compatibility.
Source Separation — Uses Spleeter to split into five stems: vocals, piano, drums, bass, other.
Transcription — Passes selected stems into Magenta’s Onsets and Frames (piano) or DrumRNN (drums).
Quantization — Uses pretty_midi to simplify rhythms and improve sheet readability.
Sheet Music Generation — Calls MuseScore CLI to produce PDF sheet music from MIDI.

The GUI supports running each step independently, enabling iterative workflows — for example, retrying transcription with different stems without repeating earlier steps.

Implementation Challenges & Solutions

Dependency Conflicts — Managed with isolated Conda environments for TensorFlow, librosa, and Spleeter compatibility.
Guitar Transcription Limitations — Trained a custom Onsets and Frames model on GuitarSet dataset; results were poor, reinforcing the need for better datasets.
Score Readability — Quantization reduced unique rhythmic values, improving clarity without major loss of musical detail.
MusicXML Workflow Issues — Direct post-processing in music21 caused layout issues; settled on quantized MIDI → MuseScore for consistency.

Results & Evaluation

Testing on piano-heavy tracks such as "Don’t Stop Believin’" demonstrated strong rhythmic accuracy and pitch detection when stems were cleanly separated. Quantization consistently reduced rhythmic complexity (e.g., unique durations), leading to more readable scores.

Drum transcription was moderately accurate for basic patterns but less reliable for cymbals and hi‑hats. Guitar transcription still requires improvement.

Example quantized sheet music output pg1

Example quantized sheet music output pg 2

Example quantized sheet music output pg 3

Example quantized sheet music output pg 4

Example quantized sheet music output pg 5

Example quantized sheet music output pg 6

Future Improvements

Support for more instruments (bass, vocals) via new or fine‑tuned models.
Batch processing and real-time audio transcription.
Chord symbol overlay and automatic key detection.
Standalone .exe packaging for easier installation.

Explore the Project

Source code, setup instructions, and example sheet music are available on GitHub.

View on GitHub