admin
Fri, 10/14/2022 - 16:46
Edited Text
Berklee College of Music
SPATIAL ILLUSIONS:
AN AUDIO-VISUAL EP THAT WILL ENCOMPASS
VIVID, DYNAMIC PROJECTION MAPPED VISUALS
Submitted in Partial Fulfillment of the Degree of
Master of Music in Music Production, Technology and Innovation
Supervisor: Pierce Warnecke
By Li-Ying Ooi
Valencia Campus, Spain
July 2019
Table of Contents
Abstract
iii
Acknowledgements
iv
1. Introduction
1
2. Review of The State of Art
1
2.1 One of A Kind
1
2.2 Portable Projection Mapping
2
3. Description
3
3.1 Audio Portion
4
3.2 Visual Portion
4
4. Innovative Aspects
5
5. New Skills Acquired
6
5.1 Music Creation Skills
6
5.2 Visual Creation Skills
7
5.3 Projection Mapping
9
5.4 Kirigami
9
6. Challenges
10
7. Future Ramifications
11
7.1 Projection Mapping for Dummies
11
7.2 Marketing and Distribution
12
8. Conclusion
13
Appendix A. Scaled-down Nett of the Sculpture
14
Appendix B. Google Searches for the State of Art Review
15
Bibliography
17
ii
Abstract
The EP, Spatial Illusions, will be presented in the form of a traditional CD album, but
instead of opening up to a compact disc, a pop-up structure will emerge when opening the
CD case, and that will be the object that is to be projection mapped with some original music.
The music itself will be in a hybrid genre called Melodic Dubstep. This genre was chosen
because it embodies the beautiful and lush elements of genres like Trance and Future Bass,
but also have the grittiness and the sprightly energy of Dubstep. The music will be
accompanied with vivid, vibrant visuals to create a unique audiovisual experience which will
be executed via projection mapping. This paper will not only address the different audio and
visual aspects of the project, but also the challenges faced and next steps of the EP.
Keywords: projection mapping, video mapping, projectors, melodic dubstep, pop-up, CD
case, CD album, abstract, audiovisual.
iii
Acknowledgements
Throughout the course of this project, I have received a great amount of support and
assistance. I am truly grateful to my supervisor, Pierce Warnecke, for his guidance and
encouragement, especially during times of doubt when I was uncertain about the project. I
would also like to express my appreciation to Ben Cantil for mentoring me with the music
aspect of the project. It was the most nerve-wrecking portion of the project and his
knowledge and insight has been extremely helpful to my development as an artist. Many
thanks to Devon Bryant as well for providing me with so much new knowledge on an art
form that I have never encountered before, it was immensely eye-opening, pun intended. My
gratitude also goes to the panel, Pablo Munguía, Pierce Warnecke, and Ben Cantil, for their
time and constructive criticisms during the prototype sessions, and also Stephen Webber and
Katerina Becic for their encouraging feedback during the final defense. I am also grateful to
all my peers in and out of Berklee for their praise and support. Their compliments have given
me confidence in my artistry, visually and musically. And finally, my parents, who have
supported and provided for me all these years, for which I would not have gotten this far
without them.
iv
1. Introduction
The uprise of streaming on digital platforms has caused the decrease of the physical
means of music consumption,1 creating a dent in the music market as artists lose an extra
outlet for revenue.2 Hence the idea of this unconventional EP was born. It will utilize the
lowly CD case as the physical media, but with a non-traditional purpose. To be clear, this
project does not aim to bring back the use of CDs, but just to revive the tangibility of holding
the physical object in hand. The user will be greeted with a pleasant surprise when opening
the CD case. A paper sculpture appears before them, unexpectedly replacing the CD. Its
unorthodox usage will be used to advocate the art of projection mapping, making it an
extremely unique CD album, and at the moment, the only one of its kind. The EP is titled
Spatial Illusions due to the visual aesthetics of the projections creating light movements onto
layers of mapped space, then combined with energetic music to create a unique audiovisual
experience.
2. Review of the State of the Art
Spatial Illusions is not just another audiovisual EP, but a way of innovating old media
with new objectives. This section will assess the individuality of the EP’s physical nature and
the technology of projection mapping in a cozier setting.
1
Meira Gebel, “The Walkman Just Turned 40 – Here’s How Listening to Music Has Changed Over the Years,”
Business Insider, June 1, 2019, https://www.businessinsider.com/history-listening-to-music-recorded-walkman2019-6?IR=T.
2
Tim Ingham, “The Album is in Deep Trouble – and the Music Business Probably Can't Save it,” Rolling Stone,
November 9, 2018, https://www.rollingstone.com/music/music-features/the-album-is-in-deep-trouble-and-themusic-business-probably-cant-save-it-753795/.
1
2.1 One of A Kind
Presently, it seems like there isn’t any other body of work that is anywhere similar
to Spatial Illusions. Google searches like ‘Projection Map Album’, ‘Projection Map CD
Album’, ‘Projection Mapping CD Case’, and ‘Pop-up CD Case’ do not yield any relevant
results. Most results refer to projection mapping as an installation or an event. Some also
refer to projection mapping as a geographical item due to the word ‘map’. Google
searches about repurposing the CD case only show concepts that destruct the CD case
instead of embracing it. It is worthwhile mentioning the first pressing of Michael
Jackson’s Dangerous, which was a collector’s edition album with some pop up artwork
like a children’s pop up story book, but it does not contain any animations or moving
parts and is purely for display3. Therefore at this present time, it’s safe to call Spatial
Illusions a one of a kind projection-mapped album.
The art of Kirigami was used to construct the physical structure built into the CD
case. Unlike Origami, Kirigami was used because it involves more than just folding of the
paper. In addition to folding, Kirigami also involves more cutting of the paper that will
result with a 3D paper creation.4 But this is where most of it ends, a final product that is
just an intelligently cut and folded piece of paper in a card or book. By squeezing it into a
CD case to compress it and using it for projection mapping with music, it creates an
innovative piece of art using projection mapping to highlight its beauty and design.
3
“Michael Jackson - Dangerous Collector's Edition First Printing 1991 Unboxing 4K | MJ Unboxing,” October
8, 2017, YouTube video, 2:33, https://www.youtube.com/watch?v=fCeQqwDh0qE.
4
Kelly Richman-Abdou, “Kirigami: The Ancient Art of Paper Cutting and How Artists Are Keeping It Alive,”
My Modern Met, April 22, 2019, https://mymodernmet.com/kirigami/.
2
2.2 Portable Projection Mapping
Most of the projection mapped pieces out there are usually large in scale on stage,
in a festival, or in a live show. So in order to enjoy a projection mapped installation in
person, one would usually have to attend an event or go to a large venue to enjoy this art.
Whilst that is great, it is not common practice to have projection mapped art in a more
intimate setting. There is now an increasing number of affordable small and portable
projectors on the market today. Which means it is possible to enjoy the art of projection
mapping in a smaller, more intimate setting. Major brands like Sony and LG not only
have projectors for professional use, but they also have a wide range of projectors for
home use, big and small. The Sony MP-CD1 Mobile Projector 5 and several of the
Minibeams from LG 6 are palm-sized and can project from one meter to three meters.
Other shapes and sizes like Anker’s Nebula Capsule 7 in the shape of a soda can and
Merlin Digital’s PocketBeam Cube 2 8 in the shape of a small cube can project a similar
range which should be sufficient enough to projection map a small object. Therefore,
there are opportunities to enjoy projection mapping art on a consumer level, rather than
just using the projector to watch movies or play games. Hoffman (2019) compiled a list
of mini projectors in the market indicating that there is a demand for compact projectors.9
Therefore, with a portable projector and a portable sculpture, no fancy technology is
required for an average consumer to participate in projection mapping.
5
“MP-CD1 Mobile Projector,” Sony.com, accessed 4 July, 2019,
https://www.sony.com/electronics/projector/mp-cd1#product_details_default.
6
“LG PW 1500G”, Lg.com, accessed 4 July 2019, https://www.lg.com/pa_en/projectors/lg-PW1500G.
7
“Capsule Projector ,”Anker.com, accessed 4 July 2019, https://www.anker.com/store/capsuleprojector/D4111111.
8
“PocketBeam Cube 2”, Merlin-Digital.com, accessed 4 July, 2019, https://merlin-digital.com/pocketbeamcube2.html.
9
Tony Hoffman, “The Best Portable Projectors for 2019.” PC Mag, 17 June, 2019,
https://uk.pcmag.com/projectors/65529/the-best-portable-projectors.
3
3. Description
Spatial Illusions not only aims to create visual stimulation, but auditory invigoration
as well with the melding of the audio and visual. Both audio and visuals have their own role,
but collectively, they create the unique audiovisual experience to really captivate the
consumer’s eye.
3.1 Audio Portion
Melodic Dubstep is a genre that combines the pretty and the ugly together. It also
tends to be quite emotional, euphoric, and more musically pleasing compared to pure
Dubstep.10 The musical goal for this EP was to create beautiful sounding introductions
and breakdowns with rich melodies and lush pads, while the drops and choruses have
harsh and gritty elements but still maintaining some sort of melodic sequence. Thus, three
tracks were born with this concept in mind, blending together calm and chaos to sound as
a cohesive musical piece.
3.2 Visual Portion
No stock imagery is used in the creation of this audiovisual EP. Everything has
been created from scratch, designed to fit within the confinements of the mapped
sculpture. The projected visuals will be of abstract style with complicated patterns and
will be animated to create dynamic movement within the piece. Although the colours will
be increasingly saturated, it is still necessary because it will guarantee a bright and vibrant
output from the projector. The visuals will be then tied to the music. For example, a
pumping animation will be connected to a pulsing kick drum in the song, and gradual
10
Phyllis Feng, “8 Emotion-Filled Melodic Dubstep Songs that will Suck You into the Genre,” Affinity, October
24, 2018, http://culture.affinitymagazine.us/8-emotion-filled-melodic-dubstep-songs-that-will-suck-you-intothe-genre/.
4
fades for slower sections to emphasize a more euphoric ambience. All these visuals will
take place on the paper structure constructed using simple Kirigami techniques. 11 As the
sculpture will be made simple due to size constraints, creating three versions of the same
design but at different scales would make it more interesting. Replicating the sculpture
three times at different scales will create three layers of different projections. This creates
some opportunities for some unique animated visuals. Furthermore, it’s a unique
experience opening up a CD case and have something totally unexpected fan out before
the person when the three layers rise up in front of the audience. The different layers will
interact with each other using different colours and animations to create a unique
audiovisual performance which can be seen below in figure 1.
Figure 1. Example of the abstract visuals on the different layers of Kirigami art.
11
“Simple Kirigami Tutorial,” May 12, 2016, YouTube video, 3:55,
https://www.youtube.com/watch?time_continue=213&v=7VoP-Q6UESw.
5
4. Innovative Aspects
This is innovative in two ways. Firstly, it can be an interactive way for people to
participate and enjoy the EP, as opposed to just putting the CD into a player and pressing
play. It is a different way to consume music, a treat for the eyes and ears. The enticing visuals
will complement the music, creating an audiovisual experience which is so much more than
just a listening activity. Secondly, it can revive the purchase of albums again as opposed to
solely using digital downloads or streaming methods to consume music and can possibly
reopen an outlet of revenue for artists. The EP will not only contain original music work, but
original artwork too, combining to art mediums into one cohesive art piece.
5. New Skills Acquired
A variety of new skills had to be acquired in order to create this EP. These skills have
been learnt during classes, extra office hours inquiries, independent research, and through
many mistakes made along the way.
5.1 Music Creation Skills
This was the first time creating a musical body of work from the very bottom.
Not using any loops, not doing a remix, not collaborating; not using someone else’s
idea as a basis has definitely been challenging. All three tracks were created solely in
Ableton Live and consists of many original sound design elements from soft synths
like Xfer’s Serum and Native Instrument’s Massive. These skills were acquired in
both of Ben Cantil’s classes. Other skills like mixing and composition skills were also
acquired in those classes, but was most beneficial outside of class as implementation
of the techniques learnt in class were then applied to the compositions. Other
technical musical skills learnt was also using iZotope’s Insight 2 for checking the
6
LUFS (Loudness Unit Full Scale)12, and Vocal Synth for altering the timbre of the
vocals in the last track.
Taking the DJ class with Nacho Marco was more helpful than anticipated. The
exploration of the Melodic Dubstep genre was done mostly trying to execute DJ sets
in the genre. Discovering how the intros and outros were so alluring and mystical,
dissecting how thay transitions into hard and heavy drops, and just balancing the
clean and dirty so it didn’t sound bipolar. It was also important that there were some
emotional elements to evoke the listener with some high impact feelings. All the
assigned coursework for that class became research when writing the tracks for the
EP.
5.2 Visual Creation Skills
Most of the visuals were made in a variety of Adobe software. They start as a
single photo, then manipulated in Adobe Photoshop to create a beautiful piece of
symmetrical abstract work. This is then brought into Adobe After Effects by blending
two layers of the same abstract pattern but rotating one to the left, and the other layer
rotated to the right at different speeds. The aesthetic results will vary depending on
the colour, blend mode, scale, and frequency of rotation. This results in a piece of
trippy animation that can be used for the projection mapping part. However, these are
skills have been acquired over the years prior to coming to Berklee and are not new
skills acquired.
12
Craig Anderton, “What is LUFS, and Why Should I Care?”, Sweetwater Sound, November 7, 2018,
https://www.sweetwater.com/insync/what-is-lufs-and-why-should-i-care/.
7
New skills learnt in Adobe After Effects were using learning to create line
traces. This type of animation creates outlines that progress over a series of time and
can be done by controlling any of the stroke offset parameters on a shape layer. But
another technique to create more exhilarating line traces was to use a plug-in called
Saber by Video Copilot. This plug-in is able to simulate a variety of different designs
like fire, electricity, and many more, to outline the desired shape. This plug-in was
used heavily as it was able to generate a wide array of different designs by just
duplicating and varying by altering the parameters like colour, glow, etc.
Another new visual skill was learning to use Cycling 74’s Max to generate
visual content, taught by Pierce Warnecke in the Non-linear Structures for Real Time
Media class. A few different types of abstract visual animations were generated from
using V-Synth and Syphon Recorder. These visuals are more random and less
symmetrical, but are still able to achieve desirable results of charismatic movement
and colour.
A mockup of the different layers was needed in order to be able to visualize
how the different layers of projections will interact with each other. This was done in
Adobe After Effects, by having one main composition with the three different layers,
and three sub-compositions representing each layer. As seen in figure 2, the
composition labeled ‘Demo Comp’ was the main composition to see all three layers in
action. Highlighted in red are then divided and animated individually in the layers
‘Big’, ‘Mid’, and ‘Small’ each representing a layer in ‘Demo Comp’. Any changes
made in the individual compositions will be reflected in the main composition as the
animation process progresses.
8
Figure 2. Demonstrating the visual animation process on the different layers of the sculpture.
5.3 Projection Mapping
To composite the layers together, the individual were exported as individual
files with a .DXV format. Then brought into Resolume Arena 6 which is also divided
into three individual layers. The Advanced Output section is the most crucial part to
the projection mapping process. With the projector connected and in the Advanced
Output using the Polygon Tool, the sculpture is then mapped out, each layer will have
it’s own mapping. Each layer/ mapping in the Advanced Output section will
correspond to a layer in the Resolume session. This will result into the different layers
acting on their own, but play together in sync with the music. This is a new skill
acquired in Devon Bryant’s Live Video class.
9
5.4 Kirigami
Building the sculpture was a heavy trial and error journey. Testing out
different shapes, size, distance between the layers, and many other factors that had to
be considered. The most important one was the stability of the whole sculpture, and
how to compress it into the CD case. It had to be tested multiple times with different
dimensions with different paper densities.
Figure 3. The amount of experimentation needed to test effectiveness of the sculpture.
6. Challenges
Most of the expected challenges were mostly of the technical sort. It was expected
that the first encounter with projection mapping was going to be difficult and unfamiliar.
Deciding what kind of sculpture to squeeze into the CD case was also a challenge due to size
10
constraints. Even during the mapping process, if the projector moved ever so slightly, the
entire mapping would shift and would become inaccurate. Having to do the mapping in a
short amount of time made presentation time rather stressful as well. But it was discovered
that even though it can be easily destructible, it is actually a very forgiving art due to the
bright, vibrant colours and vigorous movements of the animations. Abstract patterns were
also chosen to create more flexible imagery so it still manages to captivate the audience even
if the projections escape the confinements of the mapping. Most of all, each component
required several trials in order to figure out the best part that worked the most efficiently. For
example, learning and practicing mapping with Resolume, creating multiple versions of the
visuals to figure out which type of colour combination, animation movement, and music
interaction worked best. Patience was also tested during the learning process when a million
mistakes were made and waiting for the animations to render.
The unexpected challenges were mainly on a mental and emotional level. Balancing
too many classes at once, saying yes to too many peer projects, maintaining a healthy
lifestyle, uncertainty of the project’s outcome, doubting the success of the project, and
paranoia that the project would receive negative feedback, etc. These obstacles affected
presentation time with a lot of nerves and anxiety even with a lot of preparation. The best
way to overcome this is just to be confident personally and with the project as that helps to
eliminate the doubt and uncertainties.
7. Future Ramifications
This first stage of the project was to mainly focus on the content and exploring the art,
musically and visually, the next steps would be how to distribute the EP as a final product. As
11
one of the current issues is that projection mapping is not a user-friendly process, two aspects
are pivotal in order to achieve feasibility for commercial consumption.
7.1 Projection Mapping for Dummies
Projection mapping can be a tedious process and requires a certain level of
technical skills in order for it to be executed effectively.13 The goal would be to make the
projection mapping process simple enough for any average consumer to accomplish the
mapping process. One possibility to tackle this is to use Augmented Reality technology.
By using AR technology to map the sculpture, the user can use their mobile devices to
hover over the sculpture via a mobile application and it will then trigger the music and
project the visuals onto the sculpture when the target image has been detected. For the
detection of the sculpture, a mobile application can be developed with platforms like
Vuforia and Unity 3D.14 But the actual mapping after detection within the application will
still need to be further researched. This method will make the EP more accessible to
consumers and would not require any advanced technical skills to enjoy the audiovisual
experience. Furthermore, this will be an interactive component for the consumers which
can be educational about the art of projection mapping.
Another alternative would be to have the EP be presented as an art installation.
The consumer would purchase the EP as mini installation installed at a location of their
choice. The installation would be pre-mapped, so the user just has to turn the installation
on in order to enjoy the EP. This would make the EP more exclusive and could have a
higher price value due to the installation of the installation. Furthermore, the user would
13
“Projection Mapping Series Part Three: Pros, Cons, and Alternatives to Projection Mapping,” Ascend, 9
October, 2018, https://ascendstudios.com/projection-mapping-pros-cons-alternatives/.
14
“How To: Augmented Reality App Tutorial for Beginners with Vuforia and Unity 3D,” Posted by Matthew
Hallberg, July 3, 2016, YouTube video, 19:22, https://www.youtube.com/watch?v=uXNjNcqW4kY.
12
not need to worry about the mapping process at all which could be appealing to certain
audiences.
7.2 Marketing and Distribution
One approach will be to partner with a projector company that manufactures miniprojectors and sell the EP with a mini-projector included as a set. This could work as a
gimmick as it is unusual to purchase music with video projectors. Other promotional
gimmicks could also be to include a vinyl version to offer a bigger sculpture as an option.
The vinyl version will not only include a larger sculpture, but also the mini-projector as
well as a copy of a pressed vinyl.
Getting signed with a label could also help with the funding and budget of the EP
and would alleviate all the projector costs or installation costs. It could also help with the
awareness of the EP if marketed through the label. Crowdfunding could also be another
alternative that can be explored. The benefits of this is that there will be more interaction
with the interested audiences and also that there will be more freedom as there will not be
any contractual obligations. 15
Even though tangibility plays a huge role in this project, having the music
available on all streaming platforms is also vital as streaming is still one of the most
common methods of music consumption.16 This will be helpful in marketing and
15
Haakon André Søttar Hagen, “Crowdfunding Music: A Look at Crowdfunding as an Alternative Source of
Funding for Independent Creators Within the Recorded Music Industry,” Master’s thesis, University of Agder,
2017, https://pdfs.semanticscholar.org/a590/d114f9f6cb5d4775e1672349850f42365537.pdf.
16
Chris Smith, “Download Music vs Streaming Music: Which is best for you?” BT, 28 February, 2019,
https://home.bt.com/tech-gadgets/phones-tablets/downloading-music-vs-streaming-music-which-is-best-foryou-11363974732804.
13
promoting the EP as it could even encourage audiences to purchase the physical EP if
they enjoyed the music first.
8. Conclusion
This unusual EP may look like an outdated piece of media on the outside, but once
opened, the user is greeted with a pleasant, unexpected surprise. A different spin on the old
CD case can bring back the love we once had for the CD but in an innovative way with a
different purpose. The art of projection mapping should also be embraced as it has such a big
impact and has a very strong ‘wow’ effect. These initial stages focuses on the content itself,
the next step would be how to make projection mapping simple enough for an average
consumer and mass produce the final deliverable. Even though there is emphasis on the
physical product, streaming is still a popular and convenient method of distributing music
and is still the preferred way of music consumption. Therefore, having the online platforms
could create awareness of the existence of the physical EP and encourage people to purchase
it after listening to the EP online, opening an extra outlet for revenue for artists. But most
importantly, the audience will have an interesting audiovisual experience whilst enjoying this
activity of consuming music.
14
Appendix A. Scaled-down Nett of the Sculpture
15
Appendix B. Google Searches for the State of Art Review
16
17
Bibliography
André Søttar Hagen, Haakon. “Crowdfunding Music: A Look at Crowdfunding as an
Alternative Source of Funding for Independent Creators Within the Recorded Music
Industry.” Master’s thesis, University of Agder, 2017.
https://pdfs.semanticscholar.org/a590/d114f9f6cb5d4775e1672349850f42365537.pdf.
Anderton, Craig. “What is LUFS, and Why Should I Care?”, Sweetwater Sound, November
7, 2018. https://www.sweetwater.com/insync/what-is-lufs-and-why-should-i-care/.
Feng, Phyllis. “8 Emotion-Filled Melodic Dubstep Songs that will Suck You into the Genre.”
Affinity, October 24, 2018. http://culture.affinitymagazine.us/8-emotion-filled-melodicdubstep-songs-that-will-suck-you-into-the-genre/.
Gebel, Meira. “The Walkman Just Turned 40 – Here’s How Listening to Music Has Changed
Over the Years.” Business Insider, June 1, 2019. https://www.businessinsider.com/historylistening-to-music-recorded-walkman-2019-6?IR=T.
Hoffman, Tony. “The Best Portable Projectors for 2019.” PC Mag, 17 June, 2019.
https://uk.pcmag.com/projectors/65529/the-best-portable-projectors.
“How To: Augmented Reality App Tutorial for Beginners with Vuforia and Unity 3D.”
Posted by Matthew Hallberg, July 3, 2016, YouTube video, 19:22,
https://www.youtube.com/watch?v=uXNjNcqW4kY.
Ingham, Tim. “The Album is in Deep Trouble – and the Music Business Probably Can't Save
it.” Rolling Stone, November 9, 2018. https://www.rollingstone.com/music/musicfeatures/the-album-is-in-deep-trouble-and-the-music-business-probably-cant-save-it-753795/.
“Michael Jackson - Dangerous Collector's Edition First Printing 1991 Unboxing 4K | MJ
Unboxing.” October 8, 2017, YouTube video, 2:33.
https://www.youtube.com/watch?v=fCeQqwDh0qE.
“Projection Mapping Series Part Three: Pros, Cons, and Alternatives to Projection Mapping.”
Ascend, 9 October, 2018. https://ascendstudios.com/projection-mapping-pros-consalternatives/.
Richman-Abdou, Kelly. “Kirigami: The Ancient Art of Paper Cutting and How Artists Are
Keeping It Alive.” My Modern Met, April 22, 2019. https://mymodernmet.com/kirigami/.
Smith, Chris. “Download Music vs Streaming Music: Which is best for you?” BT, 28
February, 2019. https://home.bt.com/tech-gadgets/phones-tablets/downloading-music-vsstreaming-music-which-is-best-for-you-11363974732804.
“Simple Kirigami Tutorial.” May 12, 2016, YouTube video, 3:55.
https://www.youtube.com/watch?time_continue=213&v=7VoP-Q6UESw.
18
SPATIAL ILLUSIONS:
AN AUDIO-VISUAL EP THAT WILL ENCOMPASS
VIVID, DYNAMIC PROJECTION MAPPED VISUALS
Submitted in Partial Fulfillment of the Degree of
Master of Music in Music Production, Technology and Innovation
Supervisor: Pierce Warnecke
By Li-Ying Ooi
Valencia Campus, Spain
July 2019
Table of Contents
Abstract
iii
Acknowledgements
iv
1. Introduction
1
2. Review of The State of Art
1
2.1 One of A Kind
1
2.2 Portable Projection Mapping
2
3. Description
3
3.1 Audio Portion
4
3.2 Visual Portion
4
4. Innovative Aspects
5
5. New Skills Acquired
6
5.1 Music Creation Skills
6
5.2 Visual Creation Skills
7
5.3 Projection Mapping
9
5.4 Kirigami
9
6. Challenges
10
7. Future Ramifications
11
7.1 Projection Mapping for Dummies
11
7.2 Marketing and Distribution
12
8. Conclusion
13
Appendix A. Scaled-down Nett of the Sculpture
14
Appendix B. Google Searches for the State of Art Review
15
Bibliography
17
ii
Abstract
The EP, Spatial Illusions, will be presented in the form of a traditional CD album, but
instead of opening up to a compact disc, a pop-up structure will emerge when opening the
CD case, and that will be the object that is to be projection mapped with some original music.
The music itself will be in a hybrid genre called Melodic Dubstep. This genre was chosen
because it embodies the beautiful and lush elements of genres like Trance and Future Bass,
but also have the grittiness and the sprightly energy of Dubstep. The music will be
accompanied with vivid, vibrant visuals to create a unique audiovisual experience which will
be executed via projection mapping. This paper will not only address the different audio and
visual aspects of the project, but also the challenges faced and next steps of the EP.
Keywords: projection mapping, video mapping, projectors, melodic dubstep, pop-up, CD
case, CD album, abstract, audiovisual.
iii
Acknowledgements
Throughout the course of this project, I have received a great amount of support and
assistance. I am truly grateful to my supervisor, Pierce Warnecke, for his guidance and
encouragement, especially during times of doubt when I was uncertain about the project. I
would also like to express my appreciation to Ben Cantil for mentoring me with the music
aspect of the project. It was the most nerve-wrecking portion of the project and his
knowledge and insight has been extremely helpful to my development as an artist. Many
thanks to Devon Bryant as well for providing me with so much new knowledge on an art
form that I have never encountered before, it was immensely eye-opening, pun intended. My
gratitude also goes to the panel, Pablo Munguía, Pierce Warnecke, and Ben Cantil, for their
time and constructive criticisms during the prototype sessions, and also Stephen Webber and
Katerina Becic for their encouraging feedback during the final defense. I am also grateful to
all my peers in and out of Berklee for their praise and support. Their compliments have given
me confidence in my artistry, visually and musically. And finally, my parents, who have
supported and provided for me all these years, for which I would not have gotten this far
without them.
iv
1. Introduction
The uprise of streaming on digital platforms has caused the decrease of the physical
means of music consumption,1 creating a dent in the music market as artists lose an extra
outlet for revenue.2 Hence the idea of this unconventional EP was born. It will utilize the
lowly CD case as the physical media, but with a non-traditional purpose. To be clear, this
project does not aim to bring back the use of CDs, but just to revive the tangibility of holding
the physical object in hand. The user will be greeted with a pleasant surprise when opening
the CD case. A paper sculpture appears before them, unexpectedly replacing the CD. Its
unorthodox usage will be used to advocate the art of projection mapping, making it an
extremely unique CD album, and at the moment, the only one of its kind. The EP is titled
Spatial Illusions due to the visual aesthetics of the projections creating light movements onto
layers of mapped space, then combined with energetic music to create a unique audiovisual
experience.
2. Review of the State of the Art
Spatial Illusions is not just another audiovisual EP, but a way of innovating old media
with new objectives. This section will assess the individuality of the EP’s physical nature and
the technology of projection mapping in a cozier setting.
1
Meira Gebel, “The Walkman Just Turned 40 – Here’s How Listening to Music Has Changed Over the Years,”
Business Insider, June 1, 2019, https://www.businessinsider.com/history-listening-to-music-recorded-walkman2019-6?IR=T.
2
Tim Ingham, “The Album is in Deep Trouble – and the Music Business Probably Can't Save it,” Rolling Stone,
November 9, 2018, https://www.rollingstone.com/music/music-features/the-album-is-in-deep-trouble-and-themusic-business-probably-cant-save-it-753795/.
1
2.1 One of A Kind
Presently, it seems like there isn’t any other body of work that is anywhere similar
to Spatial Illusions. Google searches like ‘Projection Map Album’, ‘Projection Map CD
Album’, ‘Projection Mapping CD Case’, and ‘Pop-up CD Case’ do not yield any relevant
results. Most results refer to projection mapping as an installation or an event. Some also
refer to projection mapping as a geographical item due to the word ‘map’. Google
searches about repurposing the CD case only show concepts that destruct the CD case
instead of embracing it. It is worthwhile mentioning the first pressing of Michael
Jackson’s Dangerous, which was a collector’s edition album with some pop up artwork
like a children’s pop up story book, but it does not contain any animations or moving
parts and is purely for display3. Therefore at this present time, it’s safe to call Spatial
Illusions a one of a kind projection-mapped album.
The art of Kirigami was used to construct the physical structure built into the CD
case. Unlike Origami, Kirigami was used because it involves more than just folding of the
paper. In addition to folding, Kirigami also involves more cutting of the paper that will
result with a 3D paper creation.4 But this is where most of it ends, a final product that is
just an intelligently cut and folded piece of paper in a card or book. By squeezing it into a
CD case to compress it and using it for projection mapping with music, it creates an
innovative piece of art using projection mapping to highlight its beauty and design.
3
“Michael Jackson - Dangerous Collector's Edition First Printing 1991 Unboxing 4K | MJ Unboxing,” October
8, 2017, YouTube video, 2:33, https://www.youtube.com/watch?v=fCeQqwDh0qE.
4
Kelly Richman-Abdou, “Kirigami: The Ancient Art of Paper Cutting and How Artists Are Keeping It Alive,”
My Modern Met, April 22, 2019, https://mymodernmet.com/kirigami/.
2
2.2 Portable Projection Mapping
Most of the projection mapped pieces out there are usually large in scale on stage,
in a festival, or in a live show. So in order to enjoy a projection mapped installation in
person, one would usually have to attend an event or go to a large venue to enjoy this art.
Whilst that is great, it is not common practice to have projection mapped art in a more
intimate setting. There is now an increasing number of affordable small and portable
projectors on the market today. Which means it is possible to enjoy the art of projection
mapping in a smaller, more intimate setting. Major brands like Sony and LG not only
have projectors for professional use, but they also have a wide range of projectors for
home use, big and small. The Sony MP-CD1 Mobile Projector 5 and several of the
Minibeams from LG 6 are palm-sized and can project from one meter to three meters.
Other shapes and sizes like Anker’s Nebula Capsule 7 in the shape of a soda can and
Merlin Digital’s PocketBeam Cube 2 8 in the shape of a small cube can project a similar
range which should be sufficient enough to projection map a small object. Therefore,
there are opportunities to enjoy projection mapping art on a consumer level, rather than
just using the projector to watch movies or play games. Hoffman (2019) compiled a list
of mini projectors in the market indicating that there is a demand for compact projectors.9
Therefore, with a portable projector and a portable sculpture, no fancy technology is
required for an average consumer to participate in projection mapping.
5
“MP-CD1 Mobile Projector,” Sony.com, accessed 4 July, 2019,
https://www.sony.com/electronics/projector/mp-cd1#product_details_default.
6
“LG PW 1500G”, Lg.com, accessed 4 July 2019, https://www.lg.com/pa_en/projectors/lg-PW1500G.
7
“Capsule Projector ,”Anker.com, accessed 4 July 2019, https://www.anker.com/store/capsuleprojector/D4111111.
8
“PocketBeam Cube 2”, Merlin-Digital.com, accessed 4 July, 2019, https://merlin-digital.com/pocketbeamcube2.html.
9
Tony Hoffman, “The Best Portable Projectors for 2019.” PC Mag, 17 June, 2019,
https://uk.pcmag.com/projectors/65529/the-best-portable-projectors.
3
3. Description
Spatial Illusions not only aims to create visual stimulation, but auditory invigoration
as well with the melding of the audio and visual. Both audio and visuals have their own role,
but collectively, they create the unique audiovisual experience to really captivate the
consumer’s eye.
3.1 Audio Portion
Melodic Dubstep is a genre that combines the pretty and the ugly together. It also
tends to be quite emotional, euphoric, and more musically pleasing compared to pure
Dubstep.10 The musical goal for this EP was to create beautiful sounding introductions
and breakdowns with rich melodies and lush pads, while the drops and choruses have
harsh and gritty elements but still maintaining some sort of melodic sequence. Thus, three
tracks were born with this concept in mind, blending together calm and chaos to sound as
a cohesive musical piece.
3.2 Visual Portion
No stock imagery is used in the creation of this audiovisual EP. Everything has
been created from scratch, designed to fit within the confinements of the mapped
sculpture. The projected visuals will be of abstract style with complicated patterns and
will be animated to create dynamic movement within the piece. Although the colours will
be increasingly saturated, it is still necessary because it will guarantee a bright and vibrant
output from the projector. The visuals will be then tied to the music. For example, a
pumping animation will be connected to a pulsing kick drum in the song, and gradual
10
Phyllis Feng, “8 Emotion-Filled Melodic Dubstep Songs that will Suck You into the Genre,” Affinity, October
24, 2018, http://culture.affinitymagazine.us/8-emotion-filled-melodic-dubstep-songs-that-will-suck-you-intothe-genre/.
4
fades for slower sections to emphasize a more euphoric ambience. All these visuals will
take place on the paper structure constructed using simple Kirigami techniques. 11 As the
sculpture will be made simple due to size constraints, creating three versions of the same
design but at different scales would make it more interesting. Replicating the sculpture
three times at different scales will create three layers of different projections. This creates
some opportunities for some unique animated visuals. Furthermore, it’s a unique
experience opening up a CD case and have something totally unexpected fan out before
the person when the three layers rise up in front of the audience. The different layers will
interact with each other using different colours and animations to create a unique
audiovisual performance which can be seen below in figure 1.
Figure 1. Example of the abstract visuals on the different layers of Kirigami art.
11
“Simple Kirigami Tutorial,” May 12, 2016, YouTube video, 3:55,
https://www.youtube.com/watch?time_continue=213&v=7VoP-Q6UESw.
5
4. Innovative Aspects
This is innovative in two ways. Firstly, it can be an interactive way for people to
participate and enjoy the EP, as opposed to just putting the CD into a player and pressing
play. It is a different way to consume music, a treat for the eyes and ears. The enticing visuals
will complement the music, creating an audiovisual experience which is so much more than
just a listening activity. Secondly, it can revive the purchase of albums again as opposed to
solely using digital downloads or streaming methods to consume music and can possibly
reopen an outlet of revenue for artists. The EP will not only contain original music work, but
original artwork too, combining to art mediums into one cohesive art piece.
5. New Skills Acquired
A variety of new skills had to be acquired in order to create this EP. These skills have
been learnt during classes, extra office hours inquiries, independent research, and through
many mistakes made along the way.
5.1 Music Creation Skills
This was the first time creating a musical body of work from the very bottom.
Not using any loops, not doing a remix, not collaborating; not using someone else’s
idea as a basis has definitely been challenging. All three tracks were created solely in
Ableton Live and consists of many original sound design elements from soft synths
like Xfer’s Serum and Native Instrument’s Massive. These skills were acquired in
both of Ben Cantil’s classes. Other skills like mixing and composition skills were also
acquired in those classes, but was most beneficial outside of class as implementation
of the techniques learnt in class were then applied to the compositions. Other
technical musical skills learnt was also using iZotope’s Insight 2 for checking the
6
LUFS (Loudness Unit Full Scale)12, and Vocal Synth for altering the timbre of the
vocals in the last track.
Taking the DJ class with Nacho Marco was more helpful than anticipated. The
exploration of the Melodic Dubstep genre was done mostly trying to execute DJ sets
in the genre. Discovering how the intros and outros were so alluring and mystical,
dissecting how thay transitions into hard and heavy drops, and just balancing the
clean and dirty so it didn’t sound bipolar. It was also important that there were some
emotional elements to evoke the listener with some high impact feelings. All the
assigned coursework for that class became research when writing the tracks for the
EP.
5.2 Visual Creation Skills
Most of the visuals were made in a variety of Adobe software. They start as a
single photo, then manipulated in Adobe Photoshop to create a beautiful piece of
symmetrical abstract work. This is then brought into Adobe After Effects by blending
two layers of the same abstract pattern but rotating one to the left, and the other layer
rotated to the right at different speeds. The aesthetic results will vary depending on
the colour, blend mode, scale, and frequency of rotation. This results in a piece of
trippy animation that can be used for the projection mapping part. However, these are
skills have been acquired over the years prior to coming to Berklee and are not new
skills acquired.
12
Craig Anderton, “What is LUFS, and Why Should I Care?”, Sweetwater Sound, November 7, 2018,
https://www.sweetwater.com/insync/what-is-lufs-and-why-should-i-care/.
7
New skills learnt in Adobe After Effects were using learning to create line
traces. This type of animation creates outlines that progress over a series of time and
can be done by controlling any of the stroke offset parameters on a shape layer. But
another technique to create more exhilarating line traces was to use a plug-in called
Saber by Video Copilot. This plug-in is able to simulate a variety of different designs
like fire, electricity, and many more, to outline the desired shape. This plug-in was
used heavily as it was able to generate a wide array of different designs by just
duplicating and varying by altering the parameters like colour, glow, etc.
Another new visual skill was learning to use Cycling 74’s Max to generate
visual content, taught by Pierce Warnecke in the Non-linear Structures for Real Time
Media class. A few different types of abstract visual animations were generated from
using V-Synth and Syphon Recorder. These visuals are more random and less
symmetrical, but are still able to achieve desirable results of charismatic movement
and colour.
A mockup of the different layers was needed in order to be able to visualize
how the different layers of projections will interact with each other. This was done in
Adobe After Effects, by having one main composition with the three different layers,
and three sub-compositions representing each layer. As seen in figure 2, the
composition labeled ‘Demo Comp’ was the main composition to see all three layers in
action. Highlighted in red are then divided and animated individually in the layers
‘Big’, ‘Mid’, and ‘Small’ each representing a layer in ‘Demo Comp’. Any changes
made in the individual compositions will be reflected in the main composition as the
animation process progresses.
8
Figure 2. Demonstrating the visual animation process on the different layers of the sculpture.
5.3 Projection Mapping
To composite the layers together, the individual were exported as individual
files with a .DXV format. Then brought into Resolume Arena 6 which is also divided
into three individual layers. The Advanced Output section is the most crucial part to
the projection mapping process. With the projector connected and in the Advanced
Output using the Polygon Tool, the sculpture is then mapped out, each layer will have
it’s own mapping. Each layer/ mapping in the Advanced Output section will
correspond to a layer in the Resolume session. This will result into the different layers
acting on their own, but play together in sync with the music. This is a new skill
acquired in Devon Bryant’s Live Video class.
9
5.4 Kirigami
Building the sculpture was a heavy trial and error journey. Testing out
different shapes, size, distance between the layers, and many other factors that had to
be considered. The most important one was the stability of the whole sculpture, and
how to compress it into the CD case. It had to be tested multiple times with different
dimensions with different paper densities.
Figure 3. The amount of experimentation needed to test effectiveness of the sculpture.
6. Challenges
Most of the expected challenges were mostly of the technical sort. It was expected
that the first encounter with projection mapping was going to be difficult and unfamiliar.
Deciding what kind of sculpture to squeeze into the CD case was also a challenge due to size
10
constraints. Even during the mapping process, if the projector moved ever so slightly, the
entire mapping would shift and would become inaccurate. Having to do the mapping in a
short amount of time made presentation time rather stressful as well. But it was discovered
that even though it can be easily destructible, it is actually a very forgiving art due to the
bright, vibrant colours and vigorous movements of the animations. Abstract patterns were
also chosen to create more flexible imagery so it still manages to captivate the audience even
if the projections escape the confinements of the mapping. Most of all, each component
required several trials in order to figure out the best part that worked the most efficiently. For
example, learning and practicing mapping with Resolume, creating multiple versions of the
visuals to figure out which type of colour combination, animation movement, and music
interaction worked best. Patience was also tested during the learning process when a million
mistakes were made and waiting for the animations to render.
The unexpected challenges were mainly on a mental and emotional level. Balancing
too many classes at once, saying yes to too many peer projects, maintaining a healthy
lifestyle, uncertainty of the project’s outcome, doubting the success of the project, and
paranoia that the project would receive negative feedback, etc. These obstacles affected
presentation time with a lot of nerves and anxiety even with a lot of preparation. The best
way to overcome this is just to be confident personally and with the project as that helps to
eliminate the doubt and uncertainties.
7. Future Ramifications
This first stage of the project was to mainly focus on the content and exploring the art,
musically and visually, the next steps would be how to distribute the EP as a final product. As
11
one of the current issues is that projection mapping is not a user-friendly process, two aspects
are pivotal in order to achieve feasibility for commercial consumption.
7.1 Projection Mapping for Dummies
Projection mapping can be a tedious process and requires a certain level of
technical skills in order for it to be executed effectively.13 The goal would be to make the
projection mapping process simple enough for any average consumer to accomplish the
mapping process. One possibility to tackle this is to use Augmented Reality technology.
By using AR technology to map the sculpture, the user can use their mobile devices to
hover over the sculpture via a mobile application and it will then trigger the music and
project the visuals onto the sculpture when the target image has been detected. For the
detection of the sculpture, a mobile application can be developed with platforms like
Vuforia and Unity 3D.14 But the actual mapping after detection within the application will
still need to be further researched. This method will make the EP more accessible to
consumers and would not require any advanced technical skills to enjoy the audiovisual
experience. Furthermore, this will be an interactive component for the consumers which
can be educational about the art of projection mapping.
Another alternative would be to have the EP be presented as an art installation.
The consumer would purchase the EP as mini installation installed at a location of their
choice. The installation would be pre-mapped, so the user just has to turn the installation
on in order to enjoy the EP. This would make the EP more exclusive and could have a
higher price value due to the installation of the installation. Furthermore, the user would
13
“Projection Mapping Series Part Three: Pros, Cons, and Alternatives to Projection Mapping,” Ascend, 9
October, 2018, https://ascendstudios.com/projection-mapping-pros-cons-alternatives/.
14
“How To: Augmented Reality App Tutorial for Beginners with Vuforia and Unity 3D,” Posted by Matthew
Hallberg, July 3, 2016, YouTube video, 19:22, https://www.youtube.com/watch?v=uXNjNcqW4kY.
12
not need to worry about the mapping process at all which could be appealing to certain
audiences.
7.2 Marketing and Distribution
One approach will be to partner with a projector company that manufactures miniprojectors and sell the EP with a mini-projector included as a set. This could work as a
gimmick as it is unusual to purchase music with video projectors. Other promotional
gimmicks could also be to include a vinyl version to offer a bigger sculpture as an option.
The vinyl version will not only include a larger sculpture, but also the mini-projector as
well as a copy of a pressed vinyl.
Getting signed with a label could also help with the funding and budget of the EP
and would alleviate all the projector costs or installation costs. It could also help with the
awareness of the EP if marketed through the label. Crowdfunding could also be another
alternative that can be explored. The benefits of this is that there will be more interaction
with the interested audiences and also that there will be more freedom as there will not be
any contractual obligations. 15
Even though tangibility plays a huge role in this project, having the music
available on all streaming platforms is also vital as streaming is still one of the most
common methods of music consumption.16 This will be helpful in marketing and
15
Haakon André Søttar Hagen, “Crowdfunding Music: A Look at Crowdfunding as an Alternative Source of
Funding for Independent Creators Within the Recorded Music Industry,” Master’s thesis, University of Agder,
2017, https://pdfs.semanticscholar.org/a590/d114f9f6cb5d4775e1672349850f42365537.pdf.
16
Chris Smith, “Download Music vs Streaming Music: Which is best for you?” BT, 28 February, 2019,
https://home.bt.com/tech-gadgets/phones-tablets/downloading-music-vs-streaming-music-which-is-best-foryou-11363974732804.
13
promoting the EP as it could even encourage audiences to purchase the physical EP if
they enjoyed the music first.
8. Conclusion
This unusual EP may look like an outdated piece of media on the outside, but once
opened, the user is greeted with a pleasant, unexpected surprise. A different spin on the old
CD case can bring back the love we once had for the CD but in an innovative way with a
different purpose. The art of projection mapping should also be embraced as it has such a big
impact and has a very strong ‘wow’ effect. These initial stages focuses on the content itself,
the next step would be how to make projection mapping simple enough for an average
consumer and mass produce the final deliverable. Even though there is emphasis on the
physical product, streaming is still a popular and convenient method of distributing music
and is still the preferred way of music consumption. Therefore, having the online platforms
could create awareness of the existence of the physical EP and encourage people to purchase
it after listening to the EP online, opening an extra outlet for revenue for artists. But most
importantly, the audience will have an interesting audiovisual experience whilst enjoying this
activity of consuming music.
14
Appendix A. Scaled-down Nett of the Sculpture
15
Appendix B. Google Searches for the State of Art Review
16
17
Bibliography
André Søttar Hagen, Haakon. “Crowdfunding Music: A Look at Crowdfunding as an
Alternative Source of Funding for Independent Creators Within the Recorded Music
Industry.” Master’s thesis, University of Agder, 2017.
https://pdfs.semanticscholar.org/a590/d114f9f6cb5d4775e1672349850f42365537.pdf.
Anderton, Craig. “What is LUFS, and Why Should I Care?”, Sweetwater Sound, November
7, 2018. https://www.sweetwater.com/insync/what-is-lufs-and-why-should-i-care/.
Feng, Phyllis. “8 Emotion-Filled Melodic Dubstep Songs that will Suck You into the Genre.”
Affinity, October 24, 2018. http://culture.affinitymagazine.us/8-emotion-filled-melodicdubstep-songs-that-will-suck-you-into-the-genre/.
Gebel, Meira. “The Walkman Just Turned 40 – Here’s How Listening to Music Has Changed
Over the Years.” Business Insider, June 1, 2019. https://www.businessinsider.com/historylistening-to-music-recorded-walkman-2019-6?IR=T.
Hoffman, Tony. “The Best Portable Projectors for 2019.” PC Mag, 17 June, 2019.
https://uk.pcmag.com/projectors/65529/the-best-portable-projectors.
“How To: Augmented Reality App Tutorial for Beginners with Vuforia and Unity 3D.”
Posted by Matthew Hallberg, July 3, 2016, YouTube video, 19:22,
https://www.youtube.com/watch?v=uXNjNcqW4kY.
Ingham, Tim. “The Album is in Deep Trouble – and the Music Business Probably Can't Save
it.” Rolling Stone, November 9, 2018. https://www.rollingstone.com/music/musicfeatures/the-album-is-in-deep-trouble-and-the-music-business-probably-cant-save-it-753795/.
“Michael Jackson - Dangerous Collector's Edition First Printing 1991 Unboxing 4K | MJ
Unboxing.” October 8, 2017, YouTube video, 2:33.
https://www.youtube.com/watch?v=fCeQqwDh0qE.
“Projection Mapping Series Part Three: Pros, Cons, and Alternatives to Projection Mapping.”
Ascend, 9 October, 2018. https://ascendstudios.com/projection-mapping-pros-consalternatives/.
Richman-Abdou, Kelly. “Kirigami: The Ancient Art of Paper Cutting and How Artists Are
Keeping It Alive.” My Modern Met, April 22, 2019. https://mymodernmet.com/kirigami/.
Smith, Chris. “Download Music vs Streaming Music: Which is best for you?” BT, 28
February, 2019. https://home.bt.com/tech-gadgets/phones-tablets/downloading-music-vsstreaming-music-which-is-best-for-you-11363974732804.
“Simple Kirigami Tutorial.” May 12, 2016, YouTube video, 3:55.
https://www.youtube.com/watch?time_continue=213&v=7VoP-Q6UESw.
18
Media of
