Massive Crowds in Hugo

Hugo has been nominated for 11 Oscars including Best Achievement in Visual Effects, and has received BAFTA's and a Golden Globes award. Adam Watkins, CG Supervisor on Hugo tells us how Massive was used to create the crowd shots. Mouse-over the images below to compare the original plates with the final composites.

"Martin Scorsese's Hugo is set in 1930s Paris, with much of the story occurring in and around a huge train station located in the heart of the city.

Much of the train station was painstakingly built on huge stages located at Shepperton, Pinewood and Longcross Studios. Where the practical sets ended, digital set extensions were added. These ranged from straightforward tasks like "a view through a doorway" to the extreme "complete 360 degree environment of the whole of Paris." One example - the Longcross set - contained a train station platform with a rail track and nothing else but green screens. This meant that the views had to be digitally created in every direction, detailed to such a level that they even included CG pigeons and pigeon poop.

However, just recreating the views wasn't enough. We were trying to recreate a busy and crowded train station, meaning that in most cases the extensions had to be packed with CG extras that looked realistic enough to match the real extras on set. From the outset we knew we needed an efficient and versatile CG crowd system to meet our requirements, something that we could plug into our existing pipeline and customize. It had to be cost-effective and let us do really big tasks on a small budget with a tiny team.

When it came down to it, there were fundamental reasons that Massive was the only realistic choice:

Massive AI

The high visibility and density of train station crowds meant that we couldn't just place animated CG extras and have them walk around at random. They'd quickly walk through other people, columns, walls and trains. Also, so much movement in large public spaces is based on small reactions and interactions between individuals. If that basic foundation is missing then a crowd can quickly begin to feel unnatural, expecially when seen directly alongside a crowd of real people. Having the ability to give the crowd "intelligence" to react/interact with their environment and other agents meant we could create CG extras that moved with real purpose around the train station, and began to approximate the behavior of the real extras from the set. This would've been almost impossible to achieve any other way.

Flexibility in Layout

The layout of the train station was vaguely established but was never, ever locked. We quickly learned that to Marty, continuity is not important so long as the shot looks good. This meant shifting walls/columns/props around to improve framing, sometimes on a shot-by-shot basis, meaning that we never really had locked locations for our crowds at all.

So instead our crowd had to adapt dynamically to the changes we made as we made them. We used Massive's layout tools in a way that we could just re-route the direction a crowd should take in an area, and then leave the agents to automatically avoid obstacles all on their own.

Integrated Cloth Solution

Hugo is set in the 1930s Paris during wintertime, which meant that everyone is bundled up in big warm jackets, scarves or really long overcoats. We had to match these, and did so using the built-in cloth functionality in Massive. The result was CG extras in costumes that constantly reacted naturally to movement, greatly increasing the realism of the crowd.

Ease of Pipeline Integration

Integrating Massive into a pipeline is fairly straightforward. So much of the data going in or out of the program can be tweaked using simple scripting or just a text editor, it really is a very open architecture. Additionally the data formats output by Massive can be easily integrated into standard CG applications. Running a single sim would generate all the data needed by both our 3ds Max or Maya pipelines. With minimal processing we'd quickly have CG extras in our scenes ready to submit to the farm.

Ready-to-Run agents

For Hugo there simply wasn't the budget to develop all brand-new agents or do extensive custom mo-cap sessions. So the Massive Ready-to-Run agents like "locomotion agent" provided the perfect building block for creating agents that we could adapt for the show. We were able to prototype CG extras very quickly and start demonstrating them in shots pretty much right away.

When I saw sequence 73 for the first time (the train crash sequence), this particular shot jumped out as one of the most challenging. The people on the left-hand platform were not only CG, but they had to match photography of real people seen on the same stretch of platform in the surrounding shots. In addition, the main train station concourse is constantly in the background, which had to have one huge consistent crowd throughout the entire sequence.

So we dipped into our library of CG extras and found ones that looked the closest to the real people we had to match to. It took about a half day of texture/shader tweaking to get them looking the same. We adjusted the agent locators to match orientation versus the scene. Then we adjusted variables on each agent that let us trigger the actions performed so they more closely matched the activities of the plate people (walking, chatting to neighbor, etc.) and also synchronize when they happened. The end result was that we had agents that could perform actions on cue and matched the real extras actions close enough so there were no continuity breaks between shots. This approach was very successful and we used it on a number of other shots after that.

I will say though - it didn't work 100% of the time. Occasionally the agents stopped behaving and went into a "zombie mode." They fell out of their actions, hunched their shoulders and stood swaying on the spot like they just had a bad concussion. Drove me nuts trying to fix it but it was damn funny when it came up in dailies.

This shot had so much attention focused on it. It's not only a full CG shot, with CG train station, tracks and one of the closest CG crowds to camera in the show, but it also featured one of Marty's homage shots to the Hitchcock Vertigo shot and it's in s3d. Needless to say, it was under the most intense scrutiny at all times.

Suprisingly, it went smoother than expected. After working out the process for the v73b-02 shot above, we had the entire platform mapped with agents acting like the real extras, so we adjusted lighting a little and had a result after a couple iterations. In all honesty, the placement of Hugo sitting on the CG ground while adapting to the Vertigo camera shift in stereo was way more challenging - the crowd component just worked well from the start.

This shot was our first tentpole shot for testing many techniques used on Hugo. It was a quick action shot, with the camera following the station inspector through a crowd before the camera tilts up to see green screen. Since it was short, it was ideal for testing CG crowds, train platform look-dev, steam trains, environmental fx, stereo matchmoving, you name it.

Our first crowd test was awesome. We imported a crowd into the scene, just lit it with a simple lighting rig and an hdri, and just like that we had lots of CG extras people holding up quite nicely alongside the real crowd. High fives all round. That was in May. The edit moved around, and sadly the shot got cut. C'est la vie. Skip to several months later with the show dangerously close to delivery, the shot came back. Except now the shot had been shifted in time and extended so it was no longer a quick action shot but instead lingered lovingly on the crowd, the entire set extension area and all those CG extras walking around the platforms. Marty just loves trains - who knew?

It's one thing if the camera is moving, you can really get away with a lot (even in stereo). But with the camera just hanging there in space, it meant dialing the CG extras performances to the most exacting degree possible. One thing that really helped around that time was that we added a control that let agents carry luggage/suitcases/newspapers in either hand and apply the appropriate dynamics values depending on the object carried. It was simple but very effective in kicking up the realism a notch. I think we got away with that one ... just.

This was one of the largest crowd shots in the whole of 'Hugo'. Around the beginning of the show we'd set a camera up where the 'hanging clock' should be in the train station. It was very high up in the rafters of the station and we were very comfortable that the crowd numbers wouldn't be that great and the shot really was easily manageable.

Then at the very last moment, we received a camera revision. The clock was positioned way, way lower. Now, the entire ground floor and platforms of the train station was visible, requiring every single area to be packed with CG crowd. We went through our usual process of adjusting a crowd layout and lanes that the agents could follow. We placed props on the platforms so agents would flow around them. We added geo in areas where the doorways of trains should be so agents could get on and off them. In some areas (like around columns or props), natural eddies would form where the agents were avoiding getting stuck. This was the perfect place to add groups of 2 or 3 ambient agents with luggage, standing chatting like they were waiting for trains.

We did a bunch of quick iterations to dial in the overall crowd (sim, adjust, sim, adjust). Then we'd fire off a high quality sim that would usually complete within a couple hours (even with all the cloth being used!) For this specific shot, there were a few crowd versions but we were done way quicker than expected.

Considering the exceptional challenges "Hugo" presented, Massive endured everything we threw at it and delivered amazing results back. There really was no other way we could have achieved what was required for the show, without using Massive."