The quest to explore new avenues of storytelling has inspired a wave of innovation; both in developing tools that allow viewers to experience film in an immersive way, but also that allow Directors and DPs to immerse themselves in the virtual reality stories they are creating live on-set.
Challenged with developing a 360-degree immersive film of blockbuster proportions for Google ATAP’s Spotlight Stories, we set about developing the world’s first ever live action cinema grade production tool; Mill Stitch.
Mill Stitch is an award winning tool that enabled director Justin Lin to view the 360-degree action of ‘HELP’ in real-time, as the images fed through a series of cameras.
In creating ‘HELP’, we came across a shortfall of hardware and technology, which prompted us to start developing our own solution.
There was no software that could pull VR footage through from a 360-degree camera rig, allowing it to be reviewed on-set, in real-time, as a director normally would shooting a linear film.
We used a 360-degree camera rig that enabled us to capture the full field of view we needed in cinema quality, utilizing wide-angle lenses. But how could the team review the content being filmed live on-set? This is where Mill Stitch comes into play.
After extensive testing and R&D, our team developed custom-built propriety software that takes the images from multiple cameras and ‘stitches’ them together. The output is a 360-degree view that can be viewed in real-time via a tablet or touch screen desktop, giving director Justin Lin the ability to navigate the images in a similar way to the ATAP user.
What we’ve developed is a tool that can be used in future projects, allowing our clients and creative partners to push the boundaries of on-set experiences and immersive storytelling further.
Not only this, but the in-built recording system within Mill Stitch, Live Stitch, enables us to supply edit houses with offline selects, effectively speeding up the entire process.
We set out to create a new way of experiencing cinema, but have created a future proof tool that enables not only the viewer, but the production team to immerse themselves live, in real time, on set.
The final product of our collaboration with Google and development of Mill Stitch is ‘HELP’; a groundbreaking fully immersive film that mixes live action and CG in a 360-degree environment, creating an action-driven story deploying cinematic quality visual effects.
Since its development Mill Stitch has been awarded in the VR & AR category at the 19th annual SXSW Interactive Innovation Awards.
Insights into Mill Stitch:
· Live video ingest, lens de-warp and stitch across multiple cameras
· Projection of stitch onto interactive sphere
· Ability to scale future camera and lensing configurations
· Interactive controls allowing for Joystick/ Virtual Camera / Touchscreen latitude-longitude user input
· Ability to accept external rotational and positional data to translate and composite on-set pre-visualization
· Latency: 8-14 frame delay from actual performance to stitched interactive output
· Storage: Recorded four day rehearsal and performance video capture and automation
· System uptime: 100% at 96 hrs runtime
· Real-time interactive output and automation replay saved at the very least double the amount of runtime worth of processing per shoot day