Cloning musical performances

I suppose if you're a good cook and you taste something new made of ingredients you know, you could more or less derive exactly what would have to be done to reproduce it. But it would be quite another accomplishment to, say, pour a martini into a machine and have it determine the recipe and manufacturing process. ("The booze was Bombay Sapphire, the vermouth was brand X in proportion Y/Z to the gin, the olive was from southern Spain, and it was shaken, not stirred")

Musical performances offer similar issues - any idiot can record something, but it's quite another thing to record the *way it was played*. That is, to capture the performance in such a way as to be able to duplicate it on other instruments to the extent applicable, right down to the tiniest nuances in pitch, tempo and dynamics. Or if you're an electronic music geek, to convert a complex performance to MIDI without human intervention. But listen to this:

First, an original recording is digitized at the highest possible quality. Then, their custom-designed software then models the exact combination of pedal, hammer velocity and impact angle, and timing needed to produce each note in the song. They claim their system is so robust that line noise, voices, or singing have no effect. The software records this data in a special MIDI format file which has 7 times the descriptive variables and 10 times the resolution of a standard MIDI file. Then, this file is "performed" by a Yamaha Disklavier PRO Grand piano. Unlike the "roboplayers" you may have heard in Malls and Restaurants, this machine has orders of magnitude more precise movement and sensing, as well as separate CPUs for pedal and keyboard movement.

All this pulls together to deliver the impossible: Zenph claims that their system reproduces the actual performance of the musician, past the level of precision that the human ear can detect.

The reproduction is so perfect that re-performance isn't the only thing the setup is good for. Great pianists travel to Zenph's studio just to record themselves on the piano, and then have it replayed while sitting in various places around the auditorium. Having never heard yourself play like this, it can be a very moving experience. The robotic piano is so precise that artists record pieces into memory and return later with sound engineers to turn that program into sound

More here.

Via Gizmodo.