and the Media Lab is Digitopolis:

I guess the real question is who is Milo? Steve Jobs or DARPA?

Poster/custom typeface designed by nottwomu.ch

On December 15th, the NIME class at ITP presented their final projects at Park Slope’s own Southpaw, a venue with added nostalgic value for me as the first legitimate music venue to open in the Slope during my yoof. The performance had an incredible energy from our incredibly supportive crowd of friends, family, classmates and alums who trekked down to South Brooklyn to see the show. All in all, it was a truly special night.

My project, as you may have gathered from previous posts, my performance was dubbed p*t*e*r*o*p*t*y*x after the family of unison-blinking fireflies found in Malaysia (yes, I have not stopped receiving my fair share of ribbing for that overly-complicated explanation). Alas, sometimes I can be a little long-winded, and to quote a friend who saw the performance live, “man, that came out way cooler than the way you explained it.”

So, we’ve gotten the first round of video documentation, and am waiting on a high def. version coming in the next few weeks. For now, thanks to Tianwei for this video:

Gym Dance from Reggie Watts on Vimeo.

During the peak of the recession of 2008 and into early 2009, ad hoc tent cities began to pop up in cities around the country. These proto-shanty towns, a known phenomenon in other urban areas of the world, were now taking root in the richest country in the world. The tent, almost always a symbol of recreation in this country, a superficial structure designed to minimally shield us from the elements, had now become home to hundreds if not thousands of individuals who had become migrants within their own borders, no more shielded from the harsh elements of the urban experience than a camper hoping to stave off interaction with the “known unknowns” of the wilderness.

You might remember this approach with Chime, where I had an Asterisk server (a hosted voicemail server, essentially) speaking to a softphone on my computer, which was in turn relaying voice data to Max/MSP where pitch analysis was generating MIDI notes that were being imported and quantized in Ableton Live. WTF?! Anyways, it works.

Now with that out of the way, let me explain this crazy ass rig I have going here. Basically, I have serial data being generated in Max (from MIDI notes being pushed from Ableton Live) being sent to the Arduino, which is in turn controlling the LEDs and playing them in a sequence determined by the music coming from Ableton. Pretty cool! Definitely my intention.

Max Patch Sending and Receiving Serial

Ableton Live Setup

However, it’s not much of an “instrument” yet. Seeing as the control element is still not quite up to snuff, I decided to experiment with a potentiometer controlling the tempo in Ableton, so I set up a parallel patch that is receiving serial data from the Arduino that is being pushed to Ableton through the ctlout object. Sure, all of this could’ve been done using OSC but who has that kind of time?

So now, a potentiometer controls the tempo of a pre-sequenced composition in Ableton by way of Arduino and Max/MSP. Which means that theoretically, I could hook up flex sensors all over my body, mess with the tempo indefinitely and be completely content. Of course, it’s all a breadboard dream right now.

Tomorrow, provided I haven’t inhaled a copiously toxic level of non-lead solder, I will be expanding the breadboard version to a larger-scale mockup of the jars I will be using to contain the lights. Additionally, the controller may end up being kill switches for each of the MIDI channels.

In short, sometimes making it work within the allotted time requires some shortcuts. At some point I will be making a whole list on how to make this without having to jump through all of these hoops. But for now, there are hoops.

From my performance plan for NIME:

Background

Pteroptyx is a genus of the subfamily luciolinae and the family lampyridae, more commonly known as the firefly or lightning bug family. The organisms of this genus are known for one peculiar, defining characteristic: the ability to blink in unison. In the jungle in Malaysia, kelip-kelip (as the pteroptyx are known in Bahasa Malaysia, the local language) are a major draw for tourists and locals, who travel specifically to witness this phenomenon that no one seems quite able to understand or explain.

The metaphor of unison pulsing fit in perfectly with my original plans; I began with the idea of visualizing polyphasic music, heavily influenced by the compositions of Steve Reich. Initially, I wanted to make a performance tool that made Reich’s music more visual to the audience, and with p*t*e*r*o*p*t*y*x, I think I have found a display that accomplishes this goal.

The Instrument

p*t*e*r*o*p*t*y*x is made up of 18 frosted glass jars scattered across the stage, an Arduino, and a laptop running Max/MSP and Ableton Live. Each glass jar contains an array of four superbright LEDs (the fireflies), corresponding to 18 different MIDI notes (one jar per midi note), and a conductive lid.

A composition, composed of a MIDI sequence of 7 different repeating patterns of varying lengths and tempos, repeats endlessly in Ableton Live, pulsing MIDI notes to Max/MSP. Each midi note in Max/MSP triggers a serial out event (in this case, turning on each array of LEDs), so each note being sent from Ableton has a corresponding LED sent via the Arduino.

Each of the jars is wired, and will be connected to an Arduino which is in-turn connected to my laptop off-stage.

The Performance

The 18 frosted glass jars are going to be scattered throughout the entire area of the stage. Some will be hung, some will be on the floor, and some will be on surfaces scattered about at different heights from the floor. The sequence will already be running and repeating in Ableton Live.

Next to each of the jars will be the conductive lid. Atop each jar is a soft-switch, and each circuit is closed when the conductive jar lid is placed on the jar.

The performance will begin with the stage completely dark. I will come out holding a candle (most likely an LED candle given fire code/me not wanting to screw up any wires) and I will begin by closing the first jar. Because the sequence is already running on the laptop, closing the jar will trigger the first note/light combination to start repeating.

The first jar will repeat for the first 30 seconds, alone, then I will start to close the other jars around the stage as the piece builds. Eventually, all 18 jars will be phasing in and out in different patterns by about five minutes into the performance. The tempo of the piece will be controlled by a potentiometer, located near the final candle in the pattern. After all 18 jars are started, the tempo will gradually increase for another minute or so.

The denouement of the piece is the removal of the lids, until finally the last light/tone repeats until the end of the performance.

“After Goldsworthy” is a video sculpture by Marco Castro and Ari Joseph. A tent-like structure serves as a projection surface for mapped video projections. The viewer inside the tent is immersed in a synthesized natural environment, akin to being enveloped by the construction of an Andy Goldsworthy piece. 

The piece was created for Video Sculpture, a class at New York University’s Interactive Telecommunications Program (ITP). 

Music: Ted (Bibio Remix) by Clark.

Music visualization is a curious endeavor. In the last 50 years, we have developed visual music, which is the concept of taking principles of music and manifesting them visually. In the last 20 years, subsequent to the development of visual music, advancements in sound analysis software for computers led to the development of sophisticated music visualizers being built-in to everyday programs (think Windows Media Player visualizers circa 1999).

Moving snowflakes!

In many ways, music visualization is just reverse engineering a musical score, merely showing how the different parts of a sound structure can be broken down into their components and then rendered visually. And much like data visualization, there is a lingering hope that some new profundity will be achieved with novel presentation, that visualizing sound will enrich the experience.

Often the goal of these visualizations is diffuse and rather unclear. It seems like the goal of most music visualizations is to highlight advances in sound recognition software, or the programming chops of a specific programmer (I still love you, flight404):

Audio responsive, snake from flight404 on Vimeo.

But there are several more explorations of music visualization that are more compelling to me that I feel remain largely untouched. For one, the playing of acoustic instruments has an inherent visual component that I think deserves to be highlighted in and of itself. Peruse the drum corps below to see what I mean. They often do an excellent job of showing off not only the technical prowess of playing music together in sync, but the mechanics of playing synchronized music, and they achieve this by exaggerating their gestures.

This concept was a really compelling source for visualizing music, and so I have been developing a piece called Seasons that tries to visualize the mechanics of this sort of group performance by using the gestures as the source of visualization. Below is a visual, video-based prototype of the idea I have in mind, with the goal of eventually building 10 of these instruments and playing the piece with a small ensemble:

And “it’s the motherf*****’ rrrremix”: