Adam Friedli - http://jaiachi.blogspot.com/2011/02/paper-reading-6-adaptive-mouse.html
Evin Schuchardt - http://csce436spring2011.blogspot.com/2011/01/paper-reading-6-shimon-interactive.html
Reference Information
Title: Shimon: an interactive improvisational robotic marimba player
Authors: Guy Hoffman, Gil Weinberg
Presentation Venue: CHI 2010: 28th ACM Conference on Human Factors in Computing Systems; April 10-15, 2010; Atlanta, GA, USA
Summary
This paper describes Shimon, a robot that can play the marimba while perceiving and interacting with someone else playing an instrument and improvise according to what is heard. To allow for a wide range of motion, the robot has four arms and to better interact with people, the robot has a socially expressive head with a built in camera. The robot can match the human player’s style, tempo and harmony as the music is played.
The complete performance is made up of three different modules. In the first module, called Call-and-Response, the robot responds to a musical phrase using a sequence spotter and beat estimator. In the second module, Opportunistic Overlay Improvisation, the robot tries to play notes included in the perceived chord while also making rhythmic gestures with his head. In the third module, Rhythmic Phrase-Matching Improvisation, Shimon tries to match the style of the human while generating improvisational phrases inspired by the player.
The researchers also focused on the interactions between the robot and player by having Shimon turn away from the player when it is in the third module as if it is focusing. They also made him appear to blink and have a “slow breathing-like behavior.”
Discussion
After reading this paper, the first thing I did was look for it on Youtube. There are a lot of videos of Shimon there. To me this is a great example of Human Computer Interaction. The researchers could have stopped when the robot was able to listen to the human and play music with him, but the researchers went even further. They tried to make him seem more human by having him bob his head to the beat and turn his head either towards the player or the instrument. As a future work they even plan to enhance Shimon’s eye contact. In an interview on Youtube they discuss how they never plan to have a robot band. Their focus is the interaction between robot and human.
 |
| Taken from the paper: Shimon playing the marimba |
Do they have any plans to directly branch out this research to create a robot that interacts with humans outside of the context of music?
ReplyDeleteP.S.
I really appreciate how you always have your blogs up early, it saves me from having to scrounge around at the last minute for something to comment on.
Someone else might want to branch out but these researchers sounded like they planned to focus only on the music aspects for now.
ReplyDeleteAnd awesome! I'm glad I can help you out like that!
The first thing I did was youtube Shimon as well. I watched the highlighted version and it was really awesome! In my opinion the researchers went above and beyond what you normally see done in terms of both robotics and music!
ReplyDeleteI especially like the effect Shimon's head does. It's movement is synchronized and and interlligent. A very good implementation of human behaviour.
ReplyDelete