In the 1st Half, Mill 2D Lead Artist Daniel Morris and 3D Supervisor Wyatt Savarese shared their tips from the set for crafting the perfect crowd and stadium. For the 2nd half, they’ll reveal how they build, blend and control a realistic crowd and stadium.
Once the data, high-res stills and footage from the set have been gathered by the VFX Shoot Supervisor, the work begins - or in some cases - continues back at The Mill. From replicating a stadium's architecture to controlling a crowd's every move, Daniel and Wyatt talk through the process below...
Wrangling the Footage
When we get back from the shoot, the first thing we do is data wrangle all the footage. This is why supervising the job is so important, as you'll be familiar with the footage and know exactly what to transfer, saving a huge amount of time. It’s always beneficial to look through all the rushes from all the cameras. This can involve footage from multiple cameras (34 or 16 mm film, Alexa, RED, 5D) and there could also be some good crowd or sideline moments that are at the beginning or end of takes.
All the footage shot needs to be edited into an organized and easy-to-use manner. Depending on the job, you will have specific angles (facing camera, side angle, top view), specific actions (clapping, cheering, staring, standing), and people in different lighting (sunlight or shade). There are also sideline people to take into account which can differ for every sport.
Building the Stadium
The stadium is built from a combination of available reference photos and documents for the location, and the footage or high-res stills that a VFX supervisor shoots of the existing stadium. We can also begin to build the structure ahead of the shoot by using using reference photos. The entire stadium will be built when time allows, adding in as much detail as possible, but it's not always necessary to build the parts that are not viewable in camera.
After the stadium structure is built, we begin texture work. Texture work for stadiums are all the parts that we need to model to make the stadium: the actual color of the seats on the stadium, the railings, signboards, ad banners, exit signs, etc. This is where the photos we took for reference really come into play.
Building the Crowd
A crowd will always be assembled from both CG and real people. Real people are used for close-ups, when the crowd is in focus and the sidelines. For wide shots, CG Massive
is used for the shear scale. As soon as the edit is locked, we will decide which shots are CG Massive and which can be built using the footage and left in 2D. With a good 3D track, we can replicate the camera move and place locked off crowd plates into any section of the crowd.
The 3D team will begin by breaking the crowd work into two parts, texture work and agent work. Textures are the crowd's clothing like outfitting them in team colors and fan accessories. Agents are what the simulated crowd is called in Massive. It gives you a huge amount of freedom to manipulate and time the crowd's actions, ranging from the usual crowd cheering to the more specific Mexican wave (featured in the video below).
Placing the Crowd
Using reference photos of what was shot on set, we will start laying out the placement of the crowd. The crowd's placement and activities will depend on the type of crowd and event. If it’s a stadium, we will place them as if they were seated in rows. If it’s a park or an outdoor concert, we will place them as if they are mulling about.
If there is a need for single people to be put into seats, then they need to be prepared individually and a matte made for each person. A script can then be written to place people into each seat according to the geometry of the CG stadium.
The front sections of the crowd will generally be real people composited on the CG Massive. Some of the architecture of the stadium can be pretty specific, such as people in front of glass or in awkward corners. It helps if to film people in these positions on set.
Lights, Flares, Action: Bringing It All Together
The CG crowd and stadium elements are then combined with the live action shots. Each shot will be put together differently depending on how much has been captured in camera and how much is CG. You generally start with the the live action footage and add in the CG stadium and then the CG crowd. If we are just extending the stadium, we find the best points to blend between the existing stadium and the new CG, always keeping in mind the flow of the architecture. The camera movement across the CG stadium plates will be tracked to match the live action. Once we have the camera track, we will work on determining the correct scale of the stadium and CG crowd. Then using this as a base, we comp in the live action plates like the people in focus, on the sidelines, etc.
We then move onto the simulation stadium lighting process. Perfect lighting is essential for a dynamic crowd that draws the eye in. All the answers can be found in the live action plates. The 3D and 2D teams will work very closely to ensure that the CG crowd matches the movement and lighting of the real crowd, lighting everything according to the geometry of the stadium and using reference stills from the set. CG will supply certain lighting passes (ID, normal, occlusion) that can then be manipulated in the composite to match the real in camera lighting.
We then add in atmospheric elements like dust, flags and confetti. The final touches are lights, lens flares and camera flares. These bring the whole shot together and make it feel like it was all captured through the same lens.
For more insights from Mill artists, explore our Behind the Work series.