Shimon Is A Singing And Songwriting Robot

Researchers from Georgia Tech have created a robot called Shimon that can write songs, play music, and sing. The marimba-playing robot is going on tour to support a new album that he has composed and sings. The album will have eight to ten songs, and Shimon wrote them in collaboration with humans. The human collaboration starts with the human giving the robot a theme, like space, and Shimon will write lyrics around the theme.

The robot is also able to generate some melodies for researcher Gil Weinberg to use as he puts together a song. A human band will play the songs with Shimon as the robot sings the songs. Researchers working on the project trained the robot using datasets of 50,000 lyrics from jazz, prong rock, and hip-hop genres. The robot uses deep learning to generate words to go with the themes given form the data set.

The deep learning is combined with semantic knowledge since rhyme and rhythm are required. Collaborators created the voice that the robot sings with at Pompeu Fabra University in Barcelona. Machine learning was used to develop the voice, and it was trained on hundreds of songs.

Other changes to Shimon include new hardware to improve how he plays and moves on stage. The robot is mostly stationary but has a new mouth, eyebrows, and head movements to convey emotion and interact with the human bandmates. Upgraded hardware allows the robot to play much faster than before, in the 25 to 30 hertz range. It can also play both soft dynamic range and strong dynamic range.

Shimon can now count at the start of a song to cue the band, and at times he will wave his mallets around in time to the music. The brushless motors give the robot a greater range of motion and control of that motion.