How MR. X mobilised Massive for Unfortunate Events

Over the years, visual effects house MR. X Inc. has regularly had to realize digital crowds. On the recent Netflix release, A Series of Unfortunate Events, the studio employed its Massive pipeline to deliver a number of complex crow shots. These appeared in the 'Vile Village' episodes of season 2 of the Netflix show, and had to reveal the crows flocking as well as landing and taking off from trees and buildings.

Simon Milner, who was crowd supervisor at MR. X on that project, breaks down how Massive is used internally at the studio via an in-house management tool, and how it helped craft crowds of crows for Unfortunate Events.

Massive at MR. X

In general, MR. X carries out its base crowd layout work either inside Massive Prime or in Massive for Maya. Then, once the shot is set up with agents having been loaded and placed, the studio's in-house management tool gets to work. Milner called this utility 'MassiveX', and it has been used on every crowd-related show at the studio for the past five years.

"MassiveX takes care of all the housekeeping needed on a shot or sequence, including directory creations, sim settings and agent overrides," explains Milner. "It's able to publish to dailies and also integrates with Shotgun." due to intentionally hackable ASCII text files.

"It is also designed to be autonomous as possible," adds Milner, "allowing the artist to quickly layout the shot and get preview renders of simulations for blocking purposes or density checks right through to final submissions as quickly and easily as possible. In some cases, a single button is used to simulate, render and submit to dailies."

MR. X also has a bespoke lighting and rendering workflow for Massive simulations. This involves a Houdini throughput, with rendering done in either RenderMan or Mantra. "Because there is no direct Mantra support in Massive," details Milner, "all the simulation files - apf, ribs and cdls - are converted into Bgeo files and point clouds which can then be read into Houdini via an OTL/Houdini Digital Asset. This cache evaluator can then be used to modify the original simulation, making changes to shaders and geometry variations. It also allows for minor changes to the layout of agents such as translation and rotation."

"The benefit of this setup," continues Milner, "is that changes and fixes can be applied far quicker in the pipeline than having to return back to the crowd team to fix, re-simulate and then submit to dailies again. All shading, lighting and geometry variation is managed inside of Houdini."

Countless crows

When it came to the crow crowds in A Series of Unfortunate Events, MR. X looked to Massive immediately. "These shots would have been extremely difficult, if not impossible, to do in other crowd software at the time," states Milner. "Massive gave us the power and flexibility to approach each shot technically and still see continuity with the other shots. Massive's layout options were critical for all these shots, be it generating agent positions based on splines for the branches of trees, or polygon face placement for buildings."

"I chose Massive because it gives me the confidence that I can create any kind of agent I need on a show. From simple underwater reeds to complex multi-legged creatures, flying insects and hero humans. Massive is flexible and powerful enough for me to create all the agents I need."

The 'Vile Village' crow scenes required, first, an analysis of how real crows behave. "The crows needed to take off on cue, but in a natural way," says Milner. "This meant not all on the same frame, or as the director put it, 'like a shotgun has gone off'. Some leaving the center of the tree would trigger others to take off. Then the crows needed to find the quickest way out of the tree. You generally don't see birds taking off when startled and flying through the tree to get out, they tend to head for the nearest exit."

Once the crows were free of the tree, they would then be able to change direction and altitude to head to their target, which was a nearby village. The reverse was also true, in terms of heading from buildings in the village back to the tree. Milner notes that the crows' flight path was "artistically driven, with murmurations and twists and turns depending on the client direction for the shot. This was managed by using a combination of 3D lanes, animated sound targets and a follow agent, with the brain changing the weighting of each influencer and triggering the murmuring."

Crow performance

Milner devised a number of brain behaviors for crow agents, including: take-off, land, fly, flock, avoid other crows, avoid tree branches, head towards a target, and perform murmuration. To get to a suitable simulation stage, MR. X began with a Massive agent building process. An important part of selling their behavior was ensuring they kept to their own spots when perched on buildings or in trees, with only minimal walking about (otherwise, notes Milner, "they'd fall off branches or buildings or crash through other crows.")

"All the actions were keyframed by the animators, and we had to give strict instructions not to over-animate," says Milner. "Once you add an animation which has a specific little bit of performance, say, a hop or a turn of the head and squawk when landing, that suddenly becomes very obviously repeated when applied to 10,000 supposedly different individual birds. So all the animation sources had to be 'clean' and straightforward. Once the agent had these base animations, Massive was used to enhance the individual performances by procedurally adding to the crows activity such as opening beaks, turning heads and fluttering feathers."

MR. X employed a variety of placement methods for the shots, with tree placements predominantly handled as splines on specific branches, or, a duplicate of the tree geometry would be created and processed through Houdini. This generated single polygons at set intervals across all the branches.

"These polygons were then used to generate the bird placements in the tree," outlines Milner. "This worked very well to add large numbers of crows to the tree without having to place manually, although shots closer to camera utilized manual placement. The director's brief was to make the tree look like a Christmas tree, with crows covering all branches. Every time we thought we'd placed enough crows in the tree, the note came back, 'Fill up the tree more,' so in the end each iteration was adding thousands more crows until the right density was reached."

Simulating in reverse: a major save for MR. X

Creating the crow crowds in motion was not a simple task; Milner says it was Massive's ability to flip the animations and simulate shots in reverse that even made them possible. This is because simulating a shot in the 'normal' way proved tricky based on what the crows had to do. For example, in one of the opening shots, the crows fly past camera, hurtling towards the ground, before landing on buildings in the village. A closing shot then has them as only tiny dots on the horizon, before they fly towards camera in swirling and twisting murmurations en route to the tree, with some crows landing in the branches inches from the camera.

"In the case of the first shot," explains Milner, "the camera move was so extreme, the birds had to fly vertically downwards at over 400 miles per hour. Once they reach a certain point above the village they would peel off in one of four directions and land gently on buildings and wires. Then for the closing shot, the trick was getting such extreme shots to work perfectly, that is, having the agents land exactly on their correct spot. We could not have birds landing in the tree on top of other birds, they all have their own space."

Milner says that after running multiple tests it became clear that a normal simulation was not going to meet the demands of the shot. "Similar effects have been achieved in the past where agents need to walk/run into shot and form a logo. But there always seems to be some hesitation and shuffling about as they settle in their final spots. We couldn't afford to see this, so in the end I decided to reverse all the animations and run the simulation backwards for both of these shots."

"I kept the motion tree as it was, which basically included: Stand -> Stand_to_TakeOff -> Fly -> Fly_to_Land -> Stand. But I reversed the actions in the action editor, so that they played in reverse. So when the trigger to fly was activated, the agent was actually playing reversed Stand -> Fly_To_Land -> Fly. I would then simulate the shot with the birds flying backwards, and then afterwards reverse the sim files so that when rendered, frame 0 of the sim was really frame 1000, 1 was 999, 2 was 998 et cetera."

The result worked, and Milner was able to place all the crows on their final 'landing' spots at the start of the simulation and then have them take off backwards into the scene and perform the correct paths and behaviors. Once reversed at render time, it appeared as though all the thousands of crows fly in and land perfectly on their appropriate spots.

"I was extremely pleased with this approach and I struggle to think of another way I would have manage the shots so effectively without employing this method," says Milner. "And here's a tip: if reversing actions in Massive, then either do it before applying the agent curves, or else remember to edit the transition and latch curves, because they will be reversed as well and won't work."