Style Transfer Gallery
See through the eyes of an artist
Role: Experience Designer
Capital One is a co-sponsor of Tom Tom Festival’s Applied Machine Learning Summit, and they wanted to do more than just talk about machine learning — so they asked us to create an interactive experience showcasing a realtime example of the technology in an engaging, participatory way, while also highlighting some of the area’s amazing local artists. A central theme of Tom Tom is “learn to see your world in a new way.” We took inspiration from that ethos to create a pop-up art gallery that would further explore the idea of viewing the world from a new perspective, powered by a machine learning technique called style transfer.
Meet the artists
Concept sketch →
Install & launch!
Imagery from Princess Mononoke by Studio Ghibli for testing purposes only
The Experience
Festival participants were invited to have their picture taken, which was then superimposed into the style and technique of a featured artist, and projected among an array of digital screens. This created a real-time, visual tapestry of participants while also preserving a history of previous festival-goers and providing a unique lens into the community.
Under the hood, we used a machine learning technique called “neural style transfer.” This model takes inspiration from what we know about how the human brain processes visual information in order to teach the algorithm how to create images in the style of a particular work of art. Training for a single painting’s style takes hours, but once complete, the model can transform a new image into that style in a fraction of a second. This allowed us to process video collected from a camera in real-time, so participants could see themselves moving and interacting as if they were in the painting, before snapping a self-portrait to be added to the Gallery.
What is Style Transfer?
Style transfer is taking an image and recomposing that image in the style of another. In this experience, we employed neural networks that learn the artist’s style, trains the models on that data, and applies the artistic style to the portraits to create a new, unique art piece. While quaint by today’s AI standards, in 2018 this was cutting edge machine learning tech.
Process
We needed to create a turnkey experience prioritizing flexibility and portability. To maximize engagement we designed a series of “artist easels” or monitor stands that are lightweight, stable, and easy to break down for shipping. The “spine” of the stand is the largest element so it’s size determined the overall dimensions of each three-unit crate. The TV support brackets can be repositioned to accomodate a 65″ TV horizontally and a 36″ TV vertically.
CAD model →
Day 1 — Victoria Crown Theater (indoors)
Outcomes & Lessons
The experience generated 26,000+ social impressions over the 2 day festival, garnering trust with stakeholders and leading to additional work on a much bigger project.
Creating portable experiences can be challenging, especially in designing for a range of environments. It requires a thoughtful, meticulous approach that forefronts both adaptability and innovation. Equally important is the need for transparent and equitable compensation for artists in the use of their creative work in machine learning experiences.
Day 2 — Emancipation Park (outdoors)
Fabrication: manualabor.com