The Unknown Land
The Unknown Land
InstruMe
An interactive AR filter with Spark AR
Date
Jan.2022
Global Game Jam
48 Hours
My Role
UI/UX Designer
Game Designer
Artist
Link
Playtest Link
Tools
Maya
Figma
Spark AR
Demo Reel
InstruMe is an interactive music composer AR filter built with Facebook’s Spark AR Studio in 48 hours. Simply follow the filter test link and open up the Facebook app, you can try it out and compose your own rhythm with just facial expressions and head movement. Enjoy the duality of Oriental and Western instruments and festival vibes!
Design Process
As we are building a product in a relatively short period, we followed the structure of 'Design Sprint':
Understand
Ideate
Decide
Prototype
Testing
Understand Needs
Given the situation that Global Game Jam was on the weekend right before Chinese New Year, we wanted to create an experience for audience to enjoy the festival vibe; at the same time, we want to use the game format to have more audiences appreciate the culture in a way of 'duality'. Politically, there is a duality happening in around the world; however, for the festivals, we are sharing the same mood.
Main goals of the piece will be: interactive, fun to play, easy to execute, culture-oriented.
Based on our observation of social platforms, we concluded that filters on social platforms are a fascinating way for audiences to share their experience and emotions with their friends and family; Spark AR is a powerful tool to develop a piece with executable scope within 48 hours. So we narrowed down and aimed our final design product as a piece of AR filter (game).
Ideate and Decide
We came up with three design ideas:
Considering music is one of the most powerful media to arouse emotional resonance, we decided to further develop the second idea 'Music + Oriental / Western'.
Prototype
Interaction Built in Spark AR
Prototype from Sketch to High Fidelity
We made a list of possible facial motions (which can be recognized in Spark AR), combined each of them with a music note respectively. Once the user trigger one or more motions, the filter will play the music note(s) and display the notes according to recognization. UI for 'Oriental' vibe is designed with traditional Chinese name of the notes (different from western ones, Chinese only have five as standard).
Testing and Iteration
We conducted a series of testing, got feedback from both team members and 'naive' users, and iterated to finalize the design. One of the major changes is fix the recognition process with programing method, distinguish closing eyes from unintentional 'twinkle' (twinkle would not trigger any interaction)
Art & UI Assets
UI and graphics are done with Adobe Illustrator; 3D models are designed and animated with Maya, textured with Substance Painter (co-work with Eiko).