Creating Art Through Hand and Body Movements
Bringing Surprise and Discovery to Public Installations
Public installations should evoke a feeling of discovery and surprise. This installation seems ambiguous from a distance, but as you get closer it becomes clear that you are part of the art. Users can draw using their hands, while visualising motion with their bodies.
2 weeks (self directed)
Strategy & Planning
User Experience Design
As part of the NCAD Graduate Show, myself and Ronan Healy were asked to design and install an interactive public installation.
We wanted to experiment using computer vision technology to create an interactive display that encourages the exploration of movement throughout the space. In the display, hand and body tracking is combined to establish a visual interplay of colour and motion.
HOW IT WORKS
The physical aspects of the project are a freestanding plinth situated in the middle of the room, and a roof mounted projector which was used to project the visuals on the wall in front of the plinth.
The input devices for tracking were a Leap Motion Controller and a webcam.
There are two major parts of the code: hand tracking and motion tracking. All of the project was coded by myself and Ronan in Processing.
A webcam was mounted above the projection and pointed at the exhibition space. The pixels from the webcam video are analysed and the differences between the previous and current frames are drawn with white pixels. Trails were added to the movement by having a level of fade through opacity layering.
The pixels without motion were drawn as a transparency layer so that it would always be on top of the art.
Hands were tracked using a Leap Motion embedded in the plinth.
Hands and fingers are identified through the infrared sensor, identifying left and right hands. Right hands draw circles of varying scale and colour depending on hand orientation (pitch, roll, yaw) and the users xyz location (closer to the screen draws larger circles). Left hands act as an eraser.
Each finger is individually tracked and will only draw if it is extended, allowing for fine drawing with a single finger. The hand tracked drawing layer was drawn as a PGraphic behind the motion layer so that the motion wouldn’t be drowned out.
The initial rollout of this interactive display was slightly too ambiguous, which required us to add instructions to the plinth for the hand drawing in particular.
The Leap Motion can track hands quite accurately, but sometimes requires time to begin tracking. Some users would wave their hands very quickly over the sensor, too close to the sensor, or with a phone in their hands. This often meant that their hand wouldn’t be picked up and they would assume that it doesn’t do anything.