Eternals directed by Chloé Zhao is the story of the Eternals, a race of immortal beings who live on Earth and have shaped its history and civilizations. Weta Digital was the lead visual effects house on the film, and rather than cataloging all their shots, environments and visual effects we sat down with Matt Aitken and Sidney Kombo-Kintombo to discuss the character work involved around the lead Eternals. The film has a rich set of new heroes and these all required unique new approaches to bring their individual personas to the screen. Matt and Sidney are ideally placed to discuss what it took to build versions of each of the characters so that they could fly, build incredible machines, shoot fire and fight at the foot of a volcano to save the earth from being ripped apart by the ‘Emergence’.
Matt Aitken is one of Weta Digital’s most experienced Visual Effects Supervisors, having worked at Weta for much of its history. He most recently supervised Black Widow, his other projects as Visual Effects Supervisor include Avengers: Infinity War, Avengers: Endgame, District 9, The Adventures of Tintin, Iron Man 3, and all three films in The Hobbit trilogy. He is also known to be one of the most generous and supportive names in the business.
Sidney Kombo-Kintombo was the lead Animation Supervisor at Weta Digital on Eternals. He has previously been Animation supervisor at Weta on Avengers: Endgame (2019), driving the animation of large end battle scene, focused on Thanos, and the Women of Marvel scene, and he was one of Weta Animation Supervisors on Avengers: Infinity War (2018). On that film, his contribution was on the nuanced facial animation of Thanos and the fast-paced third act battle sequence. Sidney joined Weta Digital for The Jungle Book in 2016. Outside his work at Weta Digital, Sidney is establishing an education program in digital animation, offering young Africans opportunities to develop the skills to tell their own stories.
The film presents a fresh set of 10 superheroes in a world post-Thanos snap and Weta needed to build a visual language that not only separated the various new heroes but did not visually confuse them with the distinctive styles of the main MCU Avengers. For example, in Eternals Ikaris (Richard Madden) can fly, but unlike Iron Man, he has no jet propulsion, he flies by controlling gravity which means his body language and animation needs to be very different. As Ikaris is effectively the ‘perfect’ superman, his body motions when flying are all extremely symmetrical. Sidney places his body animation on a continuum with Spiderman at one end, with limbs bent, asymmetrical and swinging, Iron Man in the middle, more rigid but with a definite sense of propulsion and Ikaris at the other end, perfectly symmetrical and legs always together. “He is the total opposite of that spectrum, he flies most of the time really close and closed up, legs close together. I am not sure I may be lying here, but I can’t recall a single shot of him where he doesn’t have the legs really close to each other, arms controlled – which is something that you would not imagine on a lot of other characters (in the MCU)”.
To stage the vast fight in the third act, Weta needed detailed body doubles of all the primary cast, and in the case of Thena (Angelina Jolie) not only was her body double used for the extremes of fighting, but in every shot in the third act location, Thena is digital from the neck down. The actress’s wardrobe was changed so her body, clothes, and weapons, everything but her face was always digital, and for extreme fight sequences, she was entirely 100% digital. “Not her normal costume, but her superhero costume was redesigned in post and it is a fully digital asset,” explains Matt. “On the island, she’s CG from the neck down in every shot. It is painstaking work that you’d rather not do all the time, but you do get the opportunity to do it from time to time.”
While Weta needed to also create the deviant creatures the Eternals battle including Kro, the demonic CG characters did not intercut with live-action actors the way the Eternals characters did. In any shot, such as Ikaris throwing Thena, the start of a throw or fight move might be the actual actors, but this would then be taken over by a fully digital version of the actors to complete the move or to show Thena violently reacting and fighting back with matching superpowers. One sequence that was an interesting exploration of the deviants for Weta was the Ice cave accelerated evolution that comes from Ajak’s (Salma Hayek) death and absorption of her powers.
Makkari (Lauren Ridloff) locates the place of the Emergence at the foot of an active volcano in the Indian Ocean. Lauren Ridloff had to be fully digital for the speed-running sequences. “For Makkari running, we had to go full tilt and all out on the way she was running, because there was no way we could make that work any differently.” But even with Makkari fully digital, the team tried to embed her in the shots, “we added tiny impacts of her feet on the ground,” he adds. “We were trying to make sure that we had something that would make sense once you see that in the action shots.”
The location for filming the Emergence took place in the Canary Islands. Weta’s Matt and Sidney both traveled to the remote islands off the coast of Africa, for the shoot, which was prior to the COVID-19 outbreak. Principle photography wrapped around Jan 2020, but unlike other films that could assemble actors for reshoot and additional photography, once COVID happened the production was limited in its ability to reshoot or even travel. The Eternals shot more on location and utilized natural light than some prior Marvel films and Mat also did his own second unit shoot there.”
“We filmed in the Canary Islands on a wild beach for three weeks for end fight, but they didn’t fully choreograph the battle,” explains Mat “Eventually when it was edited, there were about three big holes in that sequence. But we couldn’t do any meaningful pickups shoot because of COVID, so the production came to us and asked us to pre-visualize those missing chunks of the battle.” The task for the Weta team was not about augmenting plate photography, it was about creating full CG shots that felt like they were shot on location. “And I think I did a count and about 42% of that beach fight are either fully CG or predominantly CG with maybe a face that’s been tracked and comped into place.” Sidney, in particular, worked very closely with the director to both block the correct action and make sure that the performances were as genuine and as authentic as possible to the acting choices that were made by the actual actors. That being said the incredible keyframe character animation team at Weta produced an entire sequence that looks photographic and is almost half the time fully CG. They did this without the luxury of being able to MoCap all the staff or even meet in person with the creative team or actors. While there was some MoCap done, the team mainly did hybrid work “we may take the beginning of a motion, and we might capture that on stage, but all the reaction and all the anticipation to that action, is all keyframe animation. So there is a little section that is actually MoCap but it is super tiny in the middle of all of the shot,” Sidney explains. “Nobody can move that fast. Obviously, nobody can do the kind of crazy stuff the Eternals do.”
“Chloe knew how she wanted to shoot that sequence and she had a very specific and pretty unique type of framing and lenses choices, which didn’t give us anywhere to hide,” Sidney explains. The director likes the actor’s centered in the frame and she disliked the lens from being dutched (a dutch angle is a camera shot in which the camera has been rotated relative to the horizon or vertical lines in the shot). “She didn’t want me to dutch the cameras, which is, as you can imagine in a Marvel movie, it’s pretty hard to get away from,” Sidney jokes. “Taking that into consideration, we have to come up with a way to visually tell that they are superheroic, but more with the dynamics of the camera when they are fighting while the, rather than grabbing big rocks and throwing things or doing that type of action.” If you review the end fight sequences, Weta made the lenses and central framing of the heroes are consistent whether one is watching an action scene or something outside the big action sequences.
The actors were all scanned by the production effects team during principal photography. As there was not a lot of complex facial animation or dialogue the production did Photo Booth (multiple cameras) and Medusa Scans (4D facial scans developed by Disney Research Zurich). While Matt agrees that the Light Stage remains the gold standard for capturing separate diffuse and specular facial detail, it was not an option, and building on the work of Thanos and then Gemini Man, Weta has extremely complex and accurate subsurface skin shaders that they knew could deliver even closeup shots of the hero characters. Sidney also felt strongly that while it is always best to plan and shoot for complex facial solutions, they did know that if things had played out differently, that Weta did have the in-house talent and experience to do even closeup talking CG character, “it’s not like it’s something that would scare us in any way, as we have done that on other shows where we have had to go super realistic on a real actress. So we can definitely do that if needed,” Sidney explains.
“Then we did the ‘Pepsi challenge’ thing with both Thena and Icarus where we worked up at Digi doubles for both on a good plate background shot,” Matt recalls. “That was really instrumental in us getting, particularly the facial performance side of things, right.” The team did the digital characters in two stages. First, the facial rigging, modeling + texturing, and then the full body/costume work to support the level of work shots that the team needed to fill out the third act story and drama.
One option that Weta also had as a possible path, if needed was neural rendering with deep learning. While various parts of Weta’s pipeline now use machine learning, Sidney has a healthy caution in advocating it for hero facial animation. Personally, his thinking is, for his trained eye, “when working with facial performances and machine learning (ML) it tends to chase the shape of the mouth of the nose, of the eyes, or of the eyebrows, but it doesn’t chase the emotion.” Rather than pointing to any fault in the technology, he feels that the ML tends to never be as good since it is unaware of the purpose of the shot, the sub-text of the scene. It may be accurate, but it is failing in “what the emotion of the character should be at that the specific moment, and making sure that even if the shapes are right, that the emotion is right.” he adds. “What is coming out of the machine learning a lot of time is fantastic in terms of shape. It’s just totally clueless in terms of emotion.” For Matt, the rapidly developing world of ML is “just another tool to put into the hands of the animator, even if it is just to validate a particular pose on a frame.”
What’s important for any animator is the back story of their character and especially in a film like Eternals when the director and creative team want each character to have their own animation style and unique silhouette. For some characters, this is fairly obvious. Makkari is the runner, Gilgamesh is the strongman, Phastos is the builder, but there was great subtlety as well. Thena could summon weapons, but she could never throw them, so all her fighting had to be at close range. For Weta that meant thinking about how she would summon the energy and for the animation team how she would move. Given Thena was fully animated from the neck down the team had flexibility, but they completely modeled the cadence of Thena’s movements and the style of her elegant movements on the detailed performance Angelina Jolie brought to the part. She conceived of Thena as almost a ballerina. “When Thena is fighting it looks like she’s dancing,” Matt comments. “Actually, there is one shot where we were worried that we might have gone too far, where she leaps over some lava right down the far end of the cave.” The shot is completely a CG shot but based and inspired by Angelina Jolie’s very deliberate and considered performances on set. “In the end, it works, but we actually referenced lots of different footage of ballet dancers for Thena.”
On location, the team had operating witness cameras on the beach and the Weta camera department, who are responsible for tracking and match moving would start by match moving off her body performance with their tools. “But rather than taking that directly, we took a pass with our motion editors. They are great, and they’re traditionally working with captured motion off the (MoCap) stage,” Matt outlines. “That’s their role, they understand performance and they understand the body mechanics, – the performance aspect of the way our body moves.” Without fail every one of Thena’s shots went through an ’emotion edit’ process which moved the data from being a large technical match move to being a true performance that was crafted. The difference may be quite subtle, but it is important.” In this respect, Sidney feels this is a perfect reflection of his feeling towards ML. “It’s the same subtle conversation we are talking about when we say chasing the shapes on the facial and chasing the emotional, it’s exactly the same thing. It is one thing to technically fit what is on the plate (photography)… it is another to improve the match move to a point where you get the sense that it is the body of Angelina moving naturally, and you cannot notice the difference – because the motion editors got the nuances of her performance through their technique.”
Phastos (Brian Tyree Henry) as the intelligent weapons and technology inventor Eternal also had complex hand motions that like Thena had to connect visually with later visual effects. Here most of the complex hand motions are those of Brian Tyree Henry’s but in some shots, as with other characters summoning ‘magic’, the team digitally replaced the actor’s hands and blended back into the main actor’s live-action body.
One of the interesting things about working on digital doubles is how the repetitive viewing of a face changes one’s ability to see it objectively. For this reason, Sidney limited Matt’s viewing of the digital versions of the Eternals. “By design, in our reviews, we try not to show Matt the characters on a daily basis,” he commented. “Actually they were reviewed twice a week at maximum because I rely on his ‘fresh eyes’ when viewing the characters.” Similarly, if at all possible he rotates the animators on which character they are working on, and only if there is a particular look or sentiment that one animator seems to be particularly good at will be keeping an animator on the same character over a large number of shots. What he seeks from all animators is that they identify with the character and that they give the digital character ‘appeal’. “The most important thing is that there is appeal. It’s still true from traditional 2D animation there has to be something appealing in terms of shapes when animating characters.”
The earlier narrative explanatory version of the Emergence with Arishem was done by Scanline but Weta had to tackle the final attempted Emergence and trying to depict its scale relative to the Island and the film’s heroes. In addition to the character work, there were extremely complex fluid sims for the start of the Emergence, large destruction sequences for the Domo ship crashing, and a lot of Lava work with the Volcano. Initially, there was even more lava than in the final film and it was wound back in post. “We had a big simulation team on the show,” says Matt. “The really giant water simulation stuff was the scariest simulation work on the show, especially when the hand was coming up out of the ocean with the giant whirlpools, – that was something I lost sleep over!” he exclaims.
Finally, Weta also did the post-credit character animation sequence with Eros, Thanos’ brother (Harry styles) and the Patton Oswald character, Pip the Troll. Only time will tell how they may play out in the MCU.