Facing Interaction

Facing Interaction was the research project that arose during my Fellowship at Microsoft Research, where I collaborated with Dr. Qin Cai in the creation of several prototypes of interactive artworks. In this project we explored how sensations and emotional states might be communicated non-verbally to others, to ourselves, to objects, or even to places.

Assuming that facial gestures could constitute a window onto somebody’s emotions, the prototypes aimed to support reflection on the poetics of non-verbal communication.

Facial Pentatonic

A simple musical instrument controlled by the user's face, exploring how facial gestures could be translated into musical ones.

Using the metaphor of singing, the user triggers sounds—MIDI messages sent to Ableton Live—by opening their mouth.

Head rotation selects which note within an A-minor pentatonic scale is played, freeing the performer's hands.

Look at me!

This prototype explored the projection of emotional states onto objects by embedding a simple behaviour in a plastic cup.

By tracking the user's head, the object appears to want to be looked at and reacts negatively to a lack of attention.

Several other prototypes were created, including a vest with vibrating motors indicating where the user was being looked at, and a virtual choir with eight channels of facially controlled sound.

My piece Walrus (2016), presented at ISEA 2016, also began as one of these prototypes.

Some prototypes were presented at Microsoft TechFest 2013, and were also discussed in the paper Facing Interaction, presented at ISEA 2014 (proceedings).

The projects are described in this Microsoft Research talk.