Royal Caribbean Now Sets Your Vacation Photos to Music Using AI

Berklee College of Music helps cruise line train a song bot

Royal Caribbean AI project now sets photos to music. Royal Caribbean

Everyone loves posting a good vacation photo to Instagram—but what if each one could have its own unique soundtrack, too?

Royal Caribbean is experimenting with that possibility, on Tuesday launching an online tool that turns user images into kaleidoscopic mini-videos, complete with original music inspired by the visuals — and assembled by artificial intelligence (AI).

A picture from a botanical garden, of red flowers and green leaves, generates two bars of smooth jazz. An elaborate piece of graffiti on a brick wall renders into a crunching hip-hop beat. In a quick snap of the Bensonhurst Statue House, the cruise line’s technology recognizes a dour likeness of a face peeking over a fence, and delivers a funky nu-disco snippet, with pumping guitars and horn swells.

Titled SoundSeeker, the marketer’s new website chops and swirls the three photos into one spinning abstract visual sequence — weaving in ample shots of crystal blue waters — and strings together the music, for a total of six bars at about 100 beats per minute. Some 15 seconds later, it’s all over, and the user is left with a personalized video ad — one Royal Caribbean is hoping he or she will be eager to show friends.

“People of all ages crave new ways to share their best experiences on social media,” says Jim Berra, chief marketing officer for Royal Caribbean International. “This tool allows you to put a completely unique, multisensory spin on sharing those memories — now friends and followers can see and hear your life’s adventures.”

For help creating SoundSeeker, agency MullenLowe turned to Berklee College of Music’s Institute for Creative Entrepreneurship — which focuses on emerging technologies — as well as London-based production company Unit9 and Stockholm-based music tech company Plan8. The team started by interpreting hundreds of music tracks and 10,000 photos to distill some 2.5 million combinations into a map of just ten possible moods like “Cozy, Warm and Safe” or “Energetic and Intense.” Those moods and interpretations form the basis for teaching the AI how to analyze photos and combined with parameters like facial and color recognition (powered by Google Cloud Vision) allowed it to crank out a fitting clip of music from one of a million possible unique combinations. It is created by layering instrumental parts composed in advance for the cruise line in genres from rock to hip-hop to electronic dance music.

In other words, it’s not precisely robot magic — but it’s not entirely prescriptive either.

“The key here is that Soundseeker doesn’t have a pre-defined set of rules that it follows to map specific things in the photos to specific musical elements,” says Christian Madden, svp creative director at MullenLowe. “It uses machine learning to make those decisions based on everything it knows from past experience. That past experience, its knowledge of how visual information is linked to musical information, was taught to the AI by humans when we trained it, transferring our own opinions and taste to the AI.”

How does that actually play out? “When the AI sees a person doing a backflip off a cliff into the ocean on a sunny day, it decides that this is a high-energy, adventurous scene that should get high-energy adventurous music,” explains Madden. “It knows this because it was trained on images that have similar elements. The AI takes into consideration everything it was taught about sunny days, cliffs, ocean scenes, bright blue colors, excited facial expressions and arrives at the conclusion that this is a high-energy adventurous photo.”

In fact, the point was, in a way, to get AI thinking more like a human — not in conceptual or analytic terms, but emotionally. “It works closer to the way our instincts do as the selection is the product of the training against the hundreds of photos that we all did,” says Alex Horton, creative director at Unit9. “The fundamental here is that it’s not theoretically driven in the way a composer or creative works when they build an arrangement. Instead, we created many ‘colors’ of arrangements, and it learned what the best fits are based on many criteria that were determined by what people like.”

“While this sounds mechanical, in a way it transcends that,” he adds. “A lot of us have tried to think of this process in a component level way, but it’s higher level than that. As composers, we all employed the techniques and tropes of the music [but] we wanted to provide the system with many shades and moods of music to give it the most nuance to choose from.”

For Royal Caribbean’s part, it represents an ongoing effort to position the brand as an innovator — building on marketing efforts like underwater Snapchat Spectacles and onboard attractions like a VR-enhanced trampoline park.

And while it’s a fun and intriguing application of technology, human musicians probably don’t need to panic about computers replacing them, at least not yet. Because there’s still more of a learning curve to teach the robot composers how to feel.

CREDITS:

Client: Royal Caribbean
Campaign: SoundSeeker (August 2018)
President and CEO: Michael Bayley
CMO: Jim Berra
VP, North America Marketing: Kara Wallace
Director, Brand Marketing: Marilyn Jimenez
Manager, Brand Marketing: Ana Rebbapragada
Senior Account Executive, Brand Marketing: Paul Duma

Music Consultancy: Berklee College of Music/Berklee Institute for Creative Entrepreneurship

Chair, Electronic Production and Design: Michael Bierylo
Assistant Professor, Electronic Production and Design: Jean-Luc Sinclair
Associate Professor, Electronic Production and Design: Ben Houge
Vice President, Innovation & Strategy/Managing Director for BerkleeICE: Panos Panay
Artistic Director of Video Game Scoring: Michael Sweet
Director of Operations and Programs, BerkleeICE: Nicole d’Avis
Publicist: Tori Donahue
Executive Assistant and Project Manager: Ty-Juana Flores
Director of Communications and Strategy, Office of the President: Maria Goldberg

Agency: MullenLowe
Chief Creative Officer: Mark Wenneker
Executive Creative Director: Tim Vaccarino
Executive Creative Director: Dave Weist
SVP, Group Creative Director: John Kearse
SVP, Group Creative Director: Rob Kottkamp
SVP, Creative Director: Christian Madden
Associate Creative Director: Erek Vinluan
Associate Creative Director: Danny Streadbeck
Associate Creative Director: Mark Philip
Creative Director, Design: Sean O’Brien,
Sr. Digital Designer: Lauren Schroeter
SVP, Head of Integrated Production: Zeke Bowman
Senior Digital Producer: Annie Carney
SVP, Director of Account Management: Bruce Gold
SVP, Group Account Director: Kay Pancheri
Account Director: Courtney Calvert
Senior Account Executive: Danielle Dawkins
SVP, Account Director, PR: Jaclyn Ruelle
Account Supervisor, PR: Emily Schmitt
Project Management Supervisor: Jessica McLeod
Project Manager: Ellen Cizek
VP, Business Affairs Manager: Maria Rougvie
DP/Editor, Yeti Productions: Jake Stafford
Content Producer, Yeti Productions: Eric Skvirsky

Production: Unit9
Production Team
Executive Producer: Richard Rowe
Producer: Nina Marien,
Producer: Shehryar Riaz
Producer: Martina Sogni
Project Manager: Maja Gawecka
Project Manager: Feodar Voloshin
Project Manager: Alexandra Hook

Tech Team
Technical Director: Maciej Zasada
Tech Lead: Michal Przyszczypkowski
AI specialist: Grzegorz Rucki
Machine Learning Developer: Tony Holdroyd
Backend Developer: Krzysztof Kokoszka
Backend Developer: Krzysztof Skoracki
Backend Developer: Abirafdi Raditya Putra
Frontend Developer: Edgard Zavarezzi
Frontend Developer: Tobiasz Dziubinski
Frontend Developer: Luis Bizarro

Creative Team
Creative Director: Alex Horton
Music Composition and Production: Plan8
Creative Lead: Stephan Bischof
UX Lead: Silvia Massotti
Art Director: Karol Goreczny
Art Director: Kacper Chlebowicz
Designer: Lukasz Nosek
Motion Designer: Lukasz Schaefer
Motion Designer: Simone Nunziato
Motion Designer: Norbert Mikolajczyk

PMO & QA Team
Head of PMO & QA: Dominic Berzins
QA Manager: Tom Watson
QA Lead: Adam Mingay
QA Senior Tester: Ayesha Evans
QA Tester: Evi Nathanaili
QA Tester: Rozzie McCann
QA Tester: Jemma Stafford


@GabrielBeltrone gabriel.beltrone@gmail.com Gabriel Beltrone is a frequent contributor to Adweek.
{"taxonomy":"","sortby":"","label":"","shouldShow":""}