Curiosibot One half of ROBATON PIERLUIGI BARBERIS Presented to Music Technology Innovation Department Berklee College of Music Valencia As a requirement for the Culminating Experience And the degree of MASTER IN MUSIC TECHNOLOGY AND INNOVATION Berklee College of Music, Valencia July 2014   2   Gratitude I will like to express my gratitude, first to my mother Martha for giving me the life and the supports me on every decision I need to make. To my father Nardo which whom I spent some beautiful years, and taught me to appreciate music and its value. To my aunt Marcela for being a support and pushes me to be a better person every day. To Luis Alberto an Ofelia that also support me in not so happy moments and enjoy with me grateful times. To my cousins Santiago and Andres, for being more than friends, confidents, and companions on a growing path that taught us to appreciate life and become better people. To all my friends in Ecuador Gaby, Dani, Martina, Marivi, Vero, Karlha and Maria Isabel; with whom I spent great times. To Patty and Cristina at USFQ and to Esteban for all his help in my educational path. To all my friends in Valencia from which I learn a lot and enjoy life and music. To Alan for an amazing project together; Alex and Nick for their great friendship and to Matt, Michael, Misty, Bowie and Jaime also for their friendship and support. To Alayna, the person that supports me, helps me grow, keep me in shape, etc. The person I love.   3   Pero mas que nada quiero dedicar este trabajo y año de estudios a mi madre, especialmente por la vida que me ha dado y por todas las enseñanzas que me ha entregado. “El que tomó a su madre irradia. Quien así irradia será querido. Inmediatamente será querido, inmediatamente atraerá a los demás” Bert Hellinger   4   Table of Contents Gratitude  ..........................................................................................................................................  3   Culminating Experience  ............................................................................................................  6   Introduction  .............................................................................................................................................  6   Creating presence on social networks  ........................................................................................  7   Description  of  the  Culminating  Experience  Project  ..........................................................  8   Curiosibot develops into Robaton  .................................................................................................  8   LynxMotion, Arduino and Max.  .......................................................................................................  9   How  the  Work  Incorporates  Innovation  ...............................................................................  9   What is Innovation?  .............................................................................................................................  9   How  the  Work  Contributes  to  the  Growth  of  the  Student  ............................................  10   Develop of other skill that passion you  .....................................................................................  10   Challenges,  both  anticipated  and  unexpected  .................................................................  11   Rocks at the middle of the road? Fix them and jump them!  .............................................  11   Future Ramifications  ...............................................................................................................  12   What’s Next?  ........................................................................................................................................  12   Conclusion  ..................................................................................................................................  12     5   Culminating Experience Introduction The proposal was to design, build and program a “robot” that can be consider part of a band. This “robot” could be controlled by software that can send pre-programed pieces of a song and also send real-time information about the piece that is being play by the musicians. So the main idea is to interact with it. The robot would receive real-time information via MIDI. Curiosibot works with MIDI signals that are sent from Max/MSP to an Arduino Mega board. The board receives the signal and via a patch that I use inside Max. Each of the solenoids reacts to a specific note or MIDI message and I multiply this for the 2 octaves. An electrical current of 24 volts activate each solenoid and thanks to a spring, the metallic bar of the solenoids hit the Crotales. Each one would be physically attached to a note on the acoustic instrument, so when the note is activated on the program, it hits the note on the instrument. I preferred to use solenoids because they have a time of reaction much faster than a servo or other types or devices. The project was being divided into stages. The culminating experience would focus on the first stage that is creating a complete working Curiosibot and program it to play with real musicians a pre programmed piece, and in some part of the song receive information from some of the musicians and interact with them on the stage.   6   The next stages would focus on improve the way the robot obtains data and how it interact with the band in real-time. At this stage comes the idea of joining the projects with Alan Tishk, and the Virtual Conducting Experience. (VCE) The VCE is a combination of hardware and software that allows the user to experience conducting a virtual orchestra through hand gestures. This is accomplished via a Microsoft Kinect, and all the data is being processed by Max and then send via MIDI to Curiosibot. Creating presence on social networks As a parallel project, I am sharing the story of the culminating experience on social networks as Facebook, Twitter, YouTube, etc. We were posting as the project evolved with pictures, advancement, and improvements. On Twitter, Facebook, we have special accounts for Robaton, and we also design a website. The idea of making this parallel project is mainly to show the Culminating Experience to the people that would be interested in doing something with Robaton. For every project when you are developing it, you need to think what are you going to do after you finish it.   7   Description of the Culminating Experience Project Curiosibot develops into Robaton The idea of utilizing a machine as a musician is something that has been around for a while. Right now there are several bands that incorporate robots or musical machines into their performances, but these machines are programmed to do something at a specific time. The idea of interacting with a machine in real-time as a musician will open new opportunities of performances for musicians because it will be something that people will like to see. Also, it will create bridges for other professionals such as electronic engineers to develop their creative side. At the beginning, Curiosibot as an independent project was going to be able to analyze the information that the musicians around him can provide and then create something new. When we decide to create Robaton as a joint project we decide that we need to create milestones. Before joining the projects, I created my first prototype with a sequence and also a MIDI controller so my classmates could play with it. Then, we began working with a good way of communicating between the two computers via a network for Robaton; improve aspects of delay in the signal and also some sole of information between the VCE and Curiosibot. When we found a good way to communicate and the system was working I create the visual part of the project. Build a robot face that moves according to the notes that is being played and that can also interact with the conductor. This is mainly because the robot itself cannot be seen in action easily; it needs something extra to show what it is actually doing. I also developed a light show that is controlled by the   8   music and the MIDI information that Curiosibot receives. These lights are 3 Phillips Hue Bulbs that have the possibility of changing color and intensity because they are RGB LEDS. These bulbs give different colors and tonalities depending on the notes and intensity that is being play, the illuminate Curiosibot from bellow. Other light is a Phillips Hue RGB light strip that is situated from one side to the other of the crotales at the white keys. LynxMotion, Arduino and Max. The most important resources that I used for this project are a set of robot parts that are develop by LynxMotion, and the Arduino board plus the program that they develop. Also very important is the software that is going to be the communication bridge between other musical DAW (Digital Audio Workstations) and the robot, this program would be Max/Msp 6 How the Work Incorporates Innovation What is Innovation? Innovation is creating something or inventing something from scratch, or improving upon something that already exists. In this project I improve the idea of machines as musical instruments and “robots” as a musicians. Also develop an important aspect, which a lot of people don’t think in projects like this, that is the performance aspect the show.   9   The way that I accomplish this was to create an interface that reacts from the point of view of a musician. I created an idea that is suitable for them and make them feel comfortable with it. I developed a new way to program these machines in an easy way in order to be suitable for musicians who are not technologically savvy. Then, with Robaton, the idea of having a robot and a virtual orchestra at the same time and being able to control them together is something that was never before made. All the robots that play as musicians always have been controlled by a sequence and never receive information in real time, and that is what makes this project innovative. How the Work Contributes to the Growth of the Student Develop of other skill that passion you I developed a lot of skills during the process of creation of this experience, such as electronic design, working with metal, wood and plastic. I also developed new skills in programming, building upon my basic foundations. My knowledge in hardware interfaces such as Arduino and Raspberry Pie have grown a lot. From programming simple lines and play with them to create a fully functional project that is consider as a master thesis. My presentation skills also improve thanks to the showcase at Sonar+D and the Museum of Science. These skills will help me in future projects.   10   This project contributes to my growth because everything involved is something that I am already passionate about, or something that I would like to learn. The difficulties and challenges help me grow throughout the process. The process of preparing everything and checking that all works in the way it should, help me develop skill that beside being related with the project are also related with organization and management. Challenges, both anticipated and unexpected Rocks at the middle of the road? Fix them and jump them! Like any other major project, this one was not without its share of failures and setbacks. Starting with ideas to affix the solenoids to the instrument, in this case the crotales to fried Arduinos, solenoids jumping and getting fried, problems with the Kinect tracking properly, and the list goes on. But that’s something you would always find on projects like this. Sometime the best solutions are not the more fancy one, like supporting the solenoids with two shower tubes, they work and look nice so they fulfill their purpose. My major challenge was to create the circuit board of the crotales. When I started this project I knew something about electronics, but Curiosibot need a complete other level, so I started to investigate and create the prototypes of that circuit board. For the first octave I have a prototype board with all the components solder in place. It works well and is perfect for this type of process where you are prototyping but for the second octave I create a complete new one with a PCB or printed circuit board. This one is more professional and helps me being more accurate.   11   Future Ramifications What’s Next? Now Curiosibot and Robaton, as two projects that have grown together, need to separate. Curiosibot as an instrument will grow as another musician that will help improve shows and performances. Virtual Conducting Experience will grow also into a more develop system that would help people how to conduct an orchestra. There’s no doubt that a some point Robaton will hit the stages again but now each project will grow separately and become better. Curiosibot would find another instrument to play, but the idea is that he can play any percussive instrument. I would also improve his reaction time and managing data in order to have more information played at the same time. Conclusion With this project I learn a lot of interesting things, starting from the technical part of using programing software as Max and Arduino, to more hardware develop such as creating a circuit board and prototyping and putting ideas in physical objects. Another important thing that I understand in this master and also in this project, is that a lot of people can teach you a lot of things, but it depends on you how do you managed that knowledge, and if you want to improve it and learn more. There’s a lot of ways to search for information: teachers, classmates, friends, family, Internet and books, but it will always depend in how much do you want to learn.   12       13