Final Deliverables
|
|
|
Face OSC Alarm Clock: face2wek_moutheyebrows.ck
Expressive Face OSC Instrument: face2wek_almosteverything.ck
Expressive Face OSC Instrument: face2wek_almosteverything.ck
Reflection
I used my 3 creations to test the capability of the Wekinator app in three different ways.
For the first concept, I wanted to create a functional alarm clock to keep me awake while studying. For this, I wanted to be able to detect when I was asleep (eyes closed or partially closing) and have the output sound wave wake me up. For all other expressions, the sound output should be silent. Using the Wekinator as a simple binary classifier in this way proved to be very successful and the detection was reliable. There was also a nice gradient of increasing noise as the eyes were partially closed.
For the second concept, I wanted to create a more expressive instrument using Face OSC. I modified the code to read 13 unique features (mouth, eye, jaw position as well as face scale, position, and orientation). This instrument turned out to be somewhat expressive, as I was able to reliably get silence and transition along a few different planes with single feature changes (e.g. moving my eyebrows or opening my mouth). However, this was a good example of overtraining the data and having too many inputs. While I was able to have some decent control when changing one feature from a neutral expression, the responsiveness of the system when overwhelmed by several non-neutral inputs became worse.
For the final concept, I was hoping to design an instrument that wasn't operated by simple static position changes but instead by movement patterns (gestures). I used the mouse drag input device and trained faster and slower siren-like sounds for faster and slower mouse movement. I also tried to train big open sounds to circular gestures. However, this type of movement classification was certainly beyond the capabilities of the Wekinator as the instrument completely failed to distinguish between different mouse speeds or any gestures. By moving the mouse around during training, it left the majority of the field trained on one monotonous sound.
For the first concept, I wanted to create a functional alarm clock to keep me awake while studying. For this, I wanted to be able to detect when I was asleep (eyes closed or partially closing) and have the output sound wave wake me up. For all other expressions, the sound output should be silent. Using the Wekinator as a simple binary classifier in this way proved to be very successful and the detection was reliable. There was also a nice gradient of increasing noise as the eyes were partially closed.
For the second concept, I wanted to create a more expressive instrument using Face OSC. I modified the code to read 13 unique features (mouth, eye, jaw position as well as face scale, position, and orientation). This instrument turned out to be somewhat expressive, as I was able to reliably get silence and transition along a few different planes with single feature changes (e.g. moving my eyebrows or opening my mouth). However, this was a good example of overtraining the data and having too many inputs. While I was able to have some decent control when changing one feature from a neutral expression, the responsiveness of the system when overwhelmed by several non-neutral inputs became worse.
For the final concept, I was hoping to design an instrument that wasn't operated by simple static position changes but instead by movement patterns (gestures). I used the mouse drag input device and trained faster and slower siren-like sounds for faster and slower mouse movement. I also tried to train big open sounds to circular gestures. However, this type of movement classification was certainly beyond the capabilities of the Wekinator as the instrument completely failed to distinguish between different mouse speeds or any gestures. By moving the mouse around during training, it left the majority of the field trained on one monotonous sound.