Playlist
August 14th, 2019
In the first-of-its-kind undertaking for The Mill, we collaborated with Norwegian musician Cashmere Cat and renowned Director Jake Schreier on the groundbreaking music video ‘Emotions’.

We had a chance to catch up with some of the project’s lead artists to learn all about how the intricate work came together. Scroll to read all about it!

Cashmere Gif 2

THE KICK-OFF

Jake Schreier, Director, Park Pictures
When Magnus first described what he was looking for to me, he said he “wanted to disappear.” He also was making a record that was very influenced by video games, so the thought of building him a video game avatar seemed fun. We started with older references, but his music isn’t a retro version of video game music at all, so ultimately a more modern approach felt right. Magnus and I had been playing a ton of Fortnite together (we are both terrible), so when The Mill proposed working in the Unreal Engine it was a natural fit.

Lisha Tan,  Creative Director, Character Development
For the character development, our design team worked closely with Magnus Høiberg aka Cashmere Cat and Jake Schreier on some fascinating references they provided, ranging from Japanese anime to early ‘00s games. Ultimately we wanted to craft a cat character that incorporated Nordic influences, as Magnus is Norwegian. Designers Sasha Vinogradova and Sidney Tan worked on a couple of rounds of sketch explorations and Magnus refined what he liked from what was presented. Sasha then sculpted the final character in ZBrush, which was later translated into Unreal engine and helped the director visualize the form and dimension of the character for approval.
 
Troy Barsness, Technical Artist
It was great to engage in the development of the character and story from the very beginning. Ultimately, the character is a fictional feline Princess dancing in this magic world to Magnus’ song! Designer Ed Laag played a big role in taking Sidney’s initial environment concepts and fleshing out a detailed version of the 2D concept as well as some initial 3D models. Ed’s understanding of spatial 3D, as well as the concept, helped bridge the gap from 2D to 3D smoothly. From there, Dave Witters and I were off to the races refining our way in "unreal engine" prepping for the virtual set mocap shoot.

Bts Cashmere Gif 3

CREATIVE PROCESS

Glyn Tebbutt, Creative Director
The concept of the music video is to show the cat dancing alone in her fantasy world, before pulling out to a final scene that reveals that Magnus (Cashmere Cat) and our performer, Margaret, are actually sharing a dance. Jake Schreier’s work is all about impressive single takes and single camera moves. It was crucial for us to ensure this technique remained possible for Jake, even when we cut from the augmented world to live action. Therefore, our creative approach was to shoot a motion-capture performance and to implement a virtual production pipeline to enable as much creative control on set as possible.

Bts Cashmere Gif 1

VIRTUAL PRODUCTION

Troy Barsness, Technical Artist
We knew an ambitious virtual production pipeline would be required to deliver a seamless relationship between the real world and the simulated, for everyone on set. A display screen rendering the CG character, landscape, props and structures in real-time, allowed Jake to, in effect, direct the CG character as opposed to the live-action dancer, as she moved through the stylized world while being shot with a handheld camera. This allowed everyone to adjust the characters actions, the lighting, and even environmental textures, all whilst still on set, which meant we could try out different iterations without lengthy waits for rendering.  The process required a two step approach. The first step was to get the general layout and as many assets in place to create the look. The environment was then lit, textured and elevated to create something as close to finished as possible. Once everyone was happy, we would test the interactivity and refine it. Finally, we were ready to shoot!

Cashmere Gif 4

Dave Witters, Technical Artist
The process for me started with customizing the design of the virtual camera system Jake wanted to shoot the music video with as well as creating a 3D model of the environment, mocap stage and the practical rigging set pieces that the dancer would interact with. This included a set of parallel high bars, using readily available 1-1/4" hardware, that became the ‘swinging tree’. I then replicated this in 3D, modeled the rigging and accurately positioned the tree-branch rigs and apple-boxes on the stage to correspond with the rehearsed choreography, designing and developing the 3D environment to match. The virtual camera system was connected via wireless to a huge 24 foot 3D-Live LED screen showing the real time Unreal Engine rendering. I planned the environment layout so the ‘sunlight’ shining over the lake and lighting the cat emanated from the LED screen onto set creating an immersive ambiance similar to the 3D scene, to help elevate the performance and inform Jake's decisions. We transitioned from Jake’s handheld virtual camera move to a live action feed from an ARRI Alexa to capture the live action piece all while switching the stage lighting via DMX for the big reveal - that it’s Magnus dancing with the cat!

Bts Cashmere Gif 4

Tawfeeq Martin, Technical Innovations Manager
I’d like to imagine our account equals the excitement of early film pioneers when they first produced movie magic. To the sound of Magnus' drum (song still stuck in my head!) our team moved virtual boulders, trees, and castles in the far off sky. With a snap of the finger, a sun set into place, all in-service of illuminating Jake's vision.
 
Alongside development duo Hiro Miyoshi and Mars Wong, The Mill's Creative Technology team dove head first into architecting an elegant technology flow leveraging the best in class tools available. Epic Games' Unreal Engine served as our hub -- all queued by our mill smart+virtual timecode slate we 'live linked' talent performance via the Vicon motion capture stage and displayed on 3D LIVE LED display. Practical lighting included custom-built scene aware ARRI SkyPanels. There’s a moving moment when both cats finally embrace, triggering a game event that transforms both the virtual and real world from day to night —like magic!

Glyn Tebbutt, Creative Director
The huge benefit to this way of shooting is that it gets you immediately in the environment, rather than trying to imagine the end result in your head.

Bts Cashmere Gif 5

CHALLENGES

Troy Barsness, Technical Artist
Given that this project is a music video, timing was everything! The dancer’s movements had to match the beats of music. Every step had to be synched up and match perfectly. In terms of the virtual pipeline, an area we had to troubleshoot was that we were at the mercy of how the engine resolved the simulation of clothes and fabric. It was changing with each iteration so some trial and error was required to get to a point where the engine was consistently replicating iterations correctly, pushing it to its limits.

Jake Schreier, Director
The challenge was to retain some of the spontaneity of that kind of filmmaking and I feel like the Unreal Engine and a motion capture approach allowed us to keep some of that in there. It’s hard to explain how fun it is to have a camera in your hands that can see live into a virtual space. Also, Margaret gave a great performance worth capturing, which is the point of all of this stuff in the first place.

Cashmere Gif 5

GROUNDBREAKING

Tawfeeq Martin, Technical Innovations Manager
This first of its kind music video is a study in what’s possible through creative exploration on set leveraging motion capture, realtime interactive visualization and interactive set extension. It is a fun and refreshing pursuit in production efficiency empowered by technology in the service of the creative. We hope this inspires a larger paradigm shift fueled by virtual production, in which case we will be ready!

Troy Barsness, Technical Artist
What is captured on set is near-final. The final stage is clean up, additional character modelling and iterations, such as adding volumetric mist and fog. They were all very minor adjustments.

Cashmere Gif 7

Magnus Høiberg aka Cashmere Cat
I’m so thankful for The Mill, we couldn’t have seen our vision through without them ! We came to them with a strange idea of a beautiful virtual cat, and I had so much fun working with them to bring her to life. I would also like to give a special shout out to Millton, the cat who lives in the parking lot of The Mill studio. I love him.

Glyn Tebbutt, Creative Director
It was important to keep the emotion and intricacy of the piece whilst translating the nuances of animation within mocap into a CG character. While working with Unreal had its challenges, Troy and Dave did amazing work in terms of bringing their own creative ideas and elements to heighten, embellish and amplify the final result. Jake has a powerful legacy of movement in his storytelling, where even the simplest idea becomes elevated by everything he brings to the table. This process was such a new format for all of us, and was the perfect combination of technology, Jake’s vision and creativity.

Cashmere Gif 10

Jake Schreier, Director
It was interesting for me to try and meld some of the principles from the minimalist live performance videos I usually work on, with a more traditionally maximalist, animated world.