Archive for December, 2009

Today I presented my final project (the figure-skating-inspired-movement-painting) in ICM (Introduction to Computational Media), and yesterday I presented it in Pcomp (Physical Computing).  The presentation entailed me putting on a jacket that had the accelerometer with XBee sender in a pocket and a green circle sewed onto the front.  I started the Processing sketch, clicked on the green circle (for tracking purposes) and then once the blank white “canvas” came on screen, I moved around to create an etching along with other effects.  I really enjoyed hearing the commands of “jump” and “spin” from some classmates and the sounds of glee when my line-bows, spirals and spinning arcs appeared in reaction to my movements.  Here’s a video demo:

Pcomp-ICM Final Project demo from N Z on Vimeo.

I approached this project in three main stages. Rather than create a really long post, below are the project stages with links to the posts (photos and/or videos are included in the posts):

Since the final Processing sketch depends on both camera tracking and the XBee/accelerometer to work, it is available as a zip download via the link versus a link to an online sketch.  Of course there is much more that I’d like to do to improve this project, but I must now focus on getting my freelance work done and my film completed on adult competitive figure skaters.

The final test related to my final project was to incorporate camera tracking via colors/blobs in order to map a person’s movement to the etching effect in my Processing sketch.  I spent a fair amount of time reviewing the color tracking, brightness threshold and motion tracking examples in Learning Processing, but none of them did what I needed and I couldn’t quite figure out how best to manipulate the code to get it to accurately track the color I click on and identify the center x/y coordinates.

NZ_20091206_blobsI ended up asking Elie Zananiri, sitting across from me one day, who recommended the JMyron library, as he’d worked with it before and had even created an online JMyron help post.  I downloaded and imported the library into a new Processing sketch and then added the “findGlobs” and “trackColor” constructors in “setup.”  The array variable for “globArray” was originally set to “globBoxes” so that I could see, via rectangular bounding boxes, what blobs were being detected (screen shot taken later at my office, for visual reference).

Once I was confident that this was detecting the color I wanted, then I switched the globArray to “globCenters” so that the center-point could be determined and mapped to the x and y coordinates of whatever I wanted to draw (in this photo, I’m drawing a simple ellipse at the blob’s center point). NZ_20091206_blobC

For my final sketch, I did not want to have the video image displayed, but rather only the effects that I created in Processing.  I wanted the video image to come up initially though so that I could click on the color to be tracked.  Once I clicked, I wanted it to disappear.  This was accomplished by creating a variable for “state” set to “0” initially and then “1” at the end of the “mousePressed” code.  In “draw”, I would then add this pseudocode: if (state == 0){loadPixels through updatePixels in here} else{background(255); my draw code}.  The full code is included in the final sketch on the main “documentation” post.

The original idea or context for this project was to be able to trace a skater’s movement over the ice, assuming a camera could be placed directly over a rink, and visualize jumps and spins in a standard manner that skaters and coaches can understand.  While this has utility, it’s not so interesting visually to non-skaters, which is why I wanted to create effects that had some visual appeal.

I started with a simple circle following a mouse, but that was too basic, so I transformed them into a trail of small circles that lagged slightly behind the mouse, to create an interesting movement effect (this was inspired by a sketch I saw on Open Processing).  I then added Perlin noise around the trail, which for me symbolized ice particles coming off a skating blade.  Also, when the skater or mouse is still, the noise continues, which symbolizes a sort of restlessness and should therefore encourage movement.

NZ_20091206_spiralFor the spin, I wanted to create a spiral from the center point of the mouse position (which would ultimately correspond with the person’s position in relation to the camera), and then vary the speed depending on how fast one turned.  As a test, I made a spiral where the x coordinate of a point was determined by: “float x = r * cos(theta);” and the y coordinate by: “float y = r * sin(theta);”.   The radius incremented by a set value but the theta incremented by speed, which was determined by the position of the mouse on the screen (left is slow, right is fast).  This spiral processing sketch worked fine on its own, but when I added it to the “etching” sketch, I was not happy with the results, especially since the spiral did not render fast enough, even at a high speed value.

NZ_20091206_spinsI then decided to create a different kind of effect with faster movement to correspond with a left or right spin.  I first created one that utilized arcs in a loop in this manner: “arc(mouseX, mouseY, random(50), random(50), random(PI), random(PI));”.  It created a full, spherical effect which was interesting.  When I took it to Dan for review, he modified the code so that the arcs rotate around the center, which I really liked (see the code in the “Spin” class in the sketch link below).  I created an alternate spin effect, also using arcs within a loop, in this manner: “arc(mouseX, mouseY, arcW+i, arcH+i, random(PI), random(PI));” (see the code in the “Spin2” class in the sketch link below).

NZ_20091206_jumpsFor the jumps, I wanted the effect to be happily explosive and vertically oriented, yet somewhat random.  Within a loop, I coded lines in this manner: “line(mouseX,mouseY, mouseX-(random(-15,15)), mouseY-(random(-75,75)));”.  I added “noise” within a smaller random rectangular area around the center, which I plan to modify at some point.

Finally, I combined all the classes and code together in one Processing effects-only sketch, whereby the different effects are controlled by keys, which gives the person running the sketch complete control, via the mouse and keys, of the “movement painting.”

I soldered wires from a 5-pin header to the LilyPad:

  • white: Z value on the accelerometer to d0 on the LilyPad
  • yellow: Y value on the accelerometer to d1 on the LilyPad
  • green: X value on the accelerometer to d2 on the LilyPad
  • red: 3V on the accelerometer to 3.3V on the LilyPad
  • black: ground on the accelerometer to ground on the LilyPad

I then soldered red and black wires between power and ground respectively from the LilyPad to the battery holder.  I placed the accelerometer in the 5-pin header and the XBee sender in the LilyPad.  Once I confirmed that this was working via the Processing sketch, then I sewed it into my jacket pocket (see photo below) so that the orientation of the accelerometer would be relatively constant and generate similar values each time I jump or spin.


The last thing that I needed to do was to measure the values for jumping and spinning.  I wanted to save a file with all of the readings from the X, Y and Z values for each type of movement.  I accomplished this by adding the following code to the Processing sketch:

At the top (global variable):

int counter = 0;
String[] alldata = new String[5000];

At the end of the parseData function:

alldata[counter] = adcValues[0] +” “+ adcValues[1] +” “+ adcValues[2];
if (counter < 5000) {counter ++;} else {counter = 0;}

saveStrings(“accValues1.txt”, alldata);//one quick turn to the right
I then commented the above out and changed the file name for each type of movement

After I created a bunch of text files corresponding to different movements, then I pasted the data into different sheets in an Excel workbook and looked for significant fluctuations in the X, Y and Z values.  I discovered the “normal” range for the Z value averaged at 590; the Y averaged at 453 and the X at 487.  I deduced from the readings that a jump happened when the Z value went above 800; a spin to the right happened when the Y value was between 300 to 400; and a spin to the left happened when the Y value went below 300.  These were factored into conditional statements within “draw” in the final sketch.