Source: ACCD IMMERSION LAB
Live Sketch
Time Line
5 months
Project Type
Scope
About
Team of Two
(John Ma, Ian Sterling)
Machine Learning
Augmented Reality
User Scenario
UX / UI Design
Lens Prototyping
Snap Inc. Sponsored. The Future of Storytelling with Augmented Reality is a transdisciplinary studio that will explore the next phase of narrative design using cameras, machine vision, machine learning, augmented reality, and game development software.
What is Live Sketch?
Explores how the machine vision models interpret a simple human input and context to author unexpected AR content.
Users can share their sketch with friends for them to add to and view in the network.
Generates unique 3D objects based on the sketch inference, as well as, the objects, spaces, and people recognized in the scene.
Stories
An account of imaginary and events told for entertainment.
An account of past events in someone's life or in the evolution of something.
Inspiration
Evolution of the English Alphabet
Simple letterforms evolve over time to have a deeper meaning for humans
How would something like this work for the future of machines? what meaning would they derive from simple drawings?
"What if the machine could create deeper meaning out of simple gestural illustrations which humans created? How could this evolve over time?"
Research
Machine Learning:
Generating 3D Assets from pre-existing 2D Data Sets
We were able to signify the technology in which machine learning can identify a 2D image into a whole new 3D Assets.
System Diagram
Click to Enlarge the image
Interaction Mockups
Asset changes based on context and environment: Sketch becomes a surfboard on the sand and when users moves the camera to the ocean it transforms into a shark.
Animation exploration of how assets can morph between one another in real time.
Making it Social: Sketches can be shared but will always be different based on what’s in the environment