[Computational Choreography] Choreo3D

This project is one of the projects developed in a 5-day Choreographic Coding Lab (CCL) NYC 2015 Aug, and further refined though NYU Tisch’s GSO Interdepartmental Grant.
CCLs are an outcome of Motion Bank, a four-year research project of The Forsythe Company focused on the creation of on line digital scores with guest choreographers.”

DATE:  August, 2015

TOOLKITS:  Cinder, C++, Vicon Motion Capture System

DURATION:  5 days

COLLABORATORS:   Craig Pickard, Gene Han

SHOWCASE: CCL;  Tisch GSO Event 


For the New York edition of the CCL, we created prototype software for editing choreography. It is an exploration of possible interactions between a dancer, or multiple dancers, and a computer. There has been extensive research in the field of Human-Computer Interaction in a practical sense, but there remains room for further exploration in terms of this interaction as an artistic collaboration where the computer is not simply a tool, but rather a collaborator.

  1. Motion capture system and visualization:  Throughout the workshop, several dancers were recorded using the Vicon motion capture system.  We fed this raw data into a custom-built Cinder application in JSON format.  We created a system for visualizing the data, with the ability to apply different visual effects using the graphical user interface (GUI). The user is able to add multiple dancers, and alter the speed of the data’s playback, thus creating a new piece of choreography, or a new performance, but combining separate individual performances into a cohesive whole.
  2. Digital contact improvisation: after visualizing the data in 3D space. We attach a skeleton system we build by ourselves, trying to manipulate the data based on the existent movement. i.e. give an external push to the dancer and see how the body response while listening to the previous movement. That’s by somehow exploring the concept of “contact improvisation” which is a dance technique.


Blue lines are original data from motion capture, the front skeleton with dots are skeleton that can receive external forces. QQ20160213-0@2xQQ20150902-18@2xQQ20150902-20@2x

Receiving a force in the breast:

Screen Shot 2016-02-13 at 3.39.55 PM

Apply a force directing to the ground to the head:



I’m not familiar with any physical libraries for Cinder like box2D, that which can be used for 3D skeleton system. So in the limited 5-day time(or better say 4 days), I chose to build my own simple one.
I used tricks of spring system. Basically is Hooke’s law : F = kX, where k is a constant factor characteristic of the spring: its stiffness, and X is small compared to the total possible deformation of the spring.
I built 2 classes, one for joints and one for bones. I set the k to be very large so as to fake like stick. There were many limitations in the results, like the empty missing points from the original data. And due to the accuracy of motion capture, the distances between 2 joints are changing which is different from reality. The result proved that spring is not a good choice. But I’ll keep update for this project in the future.


303 1517 1615 1902 2123

GITHUB: https://github.com/streamgao/Choreo3D/