Google NSynth Super Puts Magenta AI Into Open-Source Synthesizer

Google's Magenta AI has spawned an unexpected hardware device, the NSynth Super synthesizer that uses machine learning to create new sounds. Based on the Magenta research project, it's built using the NSynth neutral synthesizer that Google released last year, embodying the AI smarts in a tactile physical interface.

That interface resembles a Korg KAOSS PAD in many ways, its fascia dominated by a sizable touchscreen and flanked by some programmable physical controls. Designed in collaboration with the Google Creative Lab, it can take sounds from up to four different external sources and then evolve them into something new, using deep neural networks.

NSynth takes an external sound as an input, and then uses an auto encoder to pull out 16 "defining temporal features" from that. Those features are used to generate mathematical representations of each sound, which Magenta calls "embeddings"; generated embeddings can be decoded into new sounds, with the acoustic qualities of multiple inputs. In short, it takes the core aspects of different sounds and then generates new timbres and dynamics that would be hard – or even impossible – to generate with a traditional analog or digital synthesizer.

NSynth Super gives those algorithms physical form. Starting with sixteen original source sounds – spanning 15 pitches – the NSynth algorithm cooked up over 100,000 new sounds. They were loaded into the hardware itself, with four source sounds assigned to each of the four dials.

Spinning the dials chooses between the different source sounds, and then dragging a finger across the touchscreen blends the acoustic qualities in different ways. Any MIDI source – like a keyboard or sequencer – can be used to control the NSynth Super.

"Rather than combining or blending the sounds," Google's NSynth team explains, "NSynth synthesizes an entirely new sound using the acoustic qualities of the original sounds – so you could get a sound that's part flute and part sitar all at once." The engine was released as an open-source project, so that the tech-savvy can build their own AI synthesizer if they choose.

Excitingly for musicians, Google is also releasing NSynth Super as an open source project. That includes not only the software side – which uses TensorFlow and openFrameworks – but the schematics and design templates for the hardware, too. Your NSynth Super may not look as stylish as Google's does – depending on how much you have to spend on materials, and what you're using to make the casing – but it should be functionally identical.

MORE NSynth Super