Piano Accompaniment & Guide Vocal
Practice with piano accompaniment and a guide vocal that sings the solfege syllables with you. Mute the vocal to sight-sing on your own, then unmute to check your answers.
Piano Accompaniment with Deep Learning
Ladukhin's original scores are monophonic (melody only) with no accompaniment. The piano parts in this app were created using BachBot.
BachBot is a deep learning model developed by Feynman Liang at the University of Cambridge. It uses LSTM (Long Short-Term Memory) networks trained on approximately 350 Bach chorales to automatically generate four-part harmony in the style of Bach.
All of Ladukhin's melodies were fed into BachBot to generate harmonizations, and the output was then manually rearranged by the developer. Some pieces could be used as-is, while others required significant editing, but overall about 70% of the accompaniments are based on BachBot's harmonizations.
Vocal Synthesis
The guide vocals are generated using singing voice synthesis technology.
The primary system used is Sinsy, developed at the Nagoya Institute of Technology. Sinsy is a singing voice synthesis technology based on HMM (Hidden Markov Models) that can synthesize singing voices from melody and lyrics input in MusicXML format. It is released as open source (modified BSD license) and is designed to run with minimal memory. The developers of Sinsy later founded Techno-Speech, Inc., a university startup, and developed the commercial singing voice synthesis software 'CeVIO.'
Some tracks also use Synthesizer V, developed by Dreamtonics Co., Ltd. Synthesizer V is a singing voice synthesis engine that combines concatenative synthesis with neural networks to produce more natural-sounding vocals.