Meet the VFX Team of Avengers: Endgame

Much like the heroes of the Marvel Cinematic Universe assemble en masse to tackle problems of intergalactic proportions, the visual effects artists behind Avengers: Endgame also gather the best and brightest among Hollywood's top effects houses to create everything from never-before-seen spectacles like the various superhero-led legions emerging from Doctor Strange's interdimensional portals to the subtleties of Professor Hulk showing off for some admiring fans. ComicBook.com takes a deep dive behind the scenes of the epic films visual effects leads.

During our chat, we spoke with visual effects producer at Marvel Studios Jen Underdahl, visual effects supervisor at Marvel Studios Dan DeLeeuw, visual effects supervisor at Industrial Light & Magic Russell Earl, and visual effects supervisor at Weta Digital Matt Aitken.

ComicBook.com: There may be a different answer for each of you, but when you got a sense of what was going to be required for this film, what was the thing that you said, "Okay, if we don't do this, we're not going to get anywhere,"? The thing you knew you absolutely had to pull off, and pull off magnificently?

Jen Underdahl: Thanos and Smart Hulk. Thanos, 100%. We knew that for [Avengers:] Infinity War, going back a little bit, but since Thanos ties into Hulk, if he couldn't carry two-thirds of the film that he did in Infinity War, have the emotional impact and be able to pull Josh Brolin's performance through, that movie wasn't going to get made the way it got made. We started development on that character about a year before we even started, really. So you go to for that kind of thing: it was so very important to get Thanos and Smart Hulk for the second one right, because they carried so much emotional weight.

Russell Earl: I think that for us, Smart Hulk we knew, that was the one thing. The films had all built up, with [Captain America: The] Winter Soldier, [Captain America:] Civil War, Infinity War, so I felt like we were in a very good place in terms of the battle and the end battle stuff and all of that.

But we knew that if we didn't get Smart Hulk and capture Mark [Ruffalo's] performance, that to me was the one thing that sort of kept me up at night, because we were rebuilding the pipeline in order to make sure that we captured the fidelity of his performance and just the subtleties of it and translating that into Smart Hulk or something where if he wasn't in the scene with everybody else we just wanted to make sure that we got that right. So for me, that was the biggest thing.

Was there a "Eureka!" kind of experience with that process where you knew, "Okay, we got it,"?

Earl: As we were trying to find the character, we had great artwork from the Marvel [visual development] team and we had thousands of photos of Mark, and we did a Medusa capture session, which is a multiple camera photo capture session. We did back session with him, but we had his performance and we looked at all the previous Hulk models. I mean, we're fortunate in that we've worked on a lot of prior Hulk films, so we just were going through all of that and … wanted to find [footage of] just Mark being Mark.

So we found this scene: it was an interview that he had done after the movie Spotlight, it was a serious thing, and so he animated it. At that point we were pretty far along with Smart Hulk, so he could make this just Smart Hulk – or Professor Hulk – with just normal Ruffalo delivering this interview and we saw, we're like, "Oh my God, this is great", and we sent it down to these guys.

Dan DeLeeuw: And then we showed it to Mark and we were in Scotland. And so it's basically Smart Hulk, really enjoying the movie Spotlight. Mark's like, "Yeah, that's great".

You get it where you love it, and the design came in and it turned out, what it's like, and, Smart Hulk looked really handsome. And then you show the studio and the directors and they like it. But I think you're not really done until you get the confidence of the actor to go along with it because it's something that the same thing we did with Josh shot of Thanos, where he really enjoyed the show.

You get to the point where they trust you with what their performance is going to be and they know that you can reproduce their performance, so they can just not have to overact or push through the motion capture or anything like that. You can just let them be them in their role and actually get it on the screen.

Earl: Prior to that we had shot a test when they were shooting [Thor:] Ragnarok, and the [Endgame directors Joe and Anthony Russo] called, had them shoot a test and he actually was playing it a bit more on the Hulk side than the Banner side. So we got that, but the nice thing about the Spotlight test was it allowed them to see Mark didn't have to play the role bigger. He could play it more subtly and we could capture that.

Underdahl: And it happened to be, when we showed him this stuff, his first day of shooting as a Smart Hulk character in that New Asgard environment. And you could see throughout how well he didn't overact.

Matt Aitken: It was Ruffalo as Hulk.

Underdahl: It was Ruffalo as Hulk, yeah.

The flip side of that: was there something that you thought going in, "Oh, that'll be a piece of cake," that turned out to be a little bit more labor-intensive than you anticipated?

Underdahl: It was a lot. All of it was hard! Yeah, we never had a moment like, "Oh we got this." It's more like, "Wah!"

Earl: The nice thing is that we've all worked together on a number of these films. So it's like we had gone through training and by the time we got to Endgame it was like we sort of knew what to expect. Like, things are going to change, the expectations are going to be higher, and so I felt like we were really well prepared.

Underdahl: That is a great point. I mean we have been together since Winter Soldier, and so with Winter Soldier and Civil War and then Infinity War. And by the time we got to Endgame and we had shortcuts, we had built-in efficiencies: we knew the animation that we were going to take care of with Spider-Man because we had introduced him to the MCU in Civil War, and we had Giant-Man nailed and how to sell all the scale stuff, so everything built on itself gradually over the seven years to the point where we really could handle all the technology to the creative task that was before us.

Earl: And just that and just having a good shorthand and knowing like, "You know what? I'm pretty sure this is what Dan's after and what the [Russo brothers] are going to like." And we had a lot of systems in place where we could share assets back and forth to Dan and Jen keeping everything consistent and to a really high level. You're jumping from one [VFX] vendor to the next, and you wouldn't know it seeing the film.

Every time you guys go to work, you're trying to push things a little bit further forward, and this movie had an extra special dimension because it was the capstone of 10 years of Marvel films, and you've been involved in a lot of them. So what was that pushing forward task like on this one? Smart Hulk was obviously a big part of that, but tell me about knowing that you want to take it to the next level and make this great capstone for the last decade of films.

Aitken: Just the scope of the show is so huge that it's going to be special because of that, so we worked on Thanos for both films, and for Endgame I think he's more complex emotionally. Like, he actually allows his emotions to rule what he does. He says, "I'm going to enjoy doing this." So we pushed the system that we used to create Thanos so the head had a range of emotional complexity that it supported. We were very aware that we had to take him to the next level for the head's character to come through the right way. He carries a chunk of the film, he is the antagonist so he has to work really well.

We have this new system that debuted on Endgame where we were able to do complex simulation work, which has a particular team, and the pipeline that that does explosions and pyrotechnics and destruction and all that sort thing. The portals that open up that all our heroes come through are handled by that same team. We want that to be animation driven. We want our animators to be able to control the timing and the way that those elements play, because they are kind of dramatic elements in their own rights.

So we set up a system whereby the animators can block all that stuff out, and then the timing and the shape of those events can pass through automatically to our simulation team so that we can show a path of that from our animation team to Dan and the team at Marvel and get Mark on it. And then know that we did the simulation work and it's not going to change. It's going to change in terms of the detail and the visual complexity, but the timing beats are going to be what they approved.

Underdahl: For me it was just making sure that we had the right team size and we had the right companies, and we cast not only the vendors that were going to do the work with their enormously talented supervisors and teams below them, but even our second unit supervisor and our main unit supervisor, who is really a master at performance capture and he's done several movies doing it and we knew he was the right guy. He knew where the pitfalls were.

So just making sure that our review systems, our pipelines, everything was as set up and as well-built and robust as it could possibly be, because whatever was going to come, we had to make sure we were protecting the imagery, and it couldn't be because of some sort of mechanical failure or some sort of personnel failure. We had an army.

DeLeeuw: I don't think there's anything that we didn't do in this movie. We like, aged someone up, we made them younger and then fire, smoke, water explosions, you know, it's a character animation. Did you deliver? You know, if it's in the visual effects book, we did it.

Aitken: And did it to a really high level.

DeLeeuw: Yeah, if we want to have an army come out, we're going to have tens of thousands of army come out. If we're going to have our heroes in the scene--

Underdahl: --make them wet.

DeLeeuw: We're going to put them all in. Yeah, it's like, "There's Rocket, we all know Rocket. We're going to flood a room with Rocket in it. Did Rocket have to be in there? No, but it was in the script so we're going to flood the area and drown Rocket."

Was something that you saw in a Marvel comic that you really want to make visually translate into the film. Was there any little inspiration that you found on the pages of the source material?

DeLeeuw: Yeah, I mean it's, the shots got cut up a little bit but you know, definitely Cap lifting Mjolnir, we pulled some of that from the panels in that, and there was Smart Hulk holding up the building, we pulled some panels from that. Definitely with the "Women of Marvel" and everybody lined up, we always called them the "splash panel shots" from the big two-page panels, two pages that you'll put that together. So it's always something we go back to.

With Doctor Strange, we started it on Infinity War, but what's great is it's the beauty of working at Marvel and it's the production company that is Marvel, you've got people you can say to, "Okay, I need a list of every single Doctor Strange spell ever." And so you wait about a day and a PDF shows up of all the comics with every single Doctor Strange spell ever. And it's like, "Yeah, we're going to use the Winds of Watoom, and then we're going to use this, and we're going to banish them to another dimension." And so it's like it's all just at your fingertips. Anything you want, you're just like, "Yeah, I need to know this," and, "There you go."

Earl: And then there's the fun in you get those comic pages and it's like, "Okay, this is what it's this," and it's like, "Okay I will now translate that into something that can be [cinematic]," because like the film are all grounded into some sort of physical truth and to help it feel more realistic. For the real world. How do you translate that comic page?

Aitken: That was the case of the Scarlet Witch energy: she takes her to the next level, but you gave us great panel art reference for her for that, which was pretty on target but still needed to go through that process of being realized in a photoreal content to work.

What's creatively exciting about working with the Russos. What do they kind of bring to the table in terms of what you're doing for the visual effects?

DeLeeuw: Well, I think it's something that since Winter Soldier they've got their key phrases for things, like "Keep it grounded." And then, with this, it's one of the things they said was, "The best idea wins." So whether it be in script or in stunt [visualization] or in pre-visualization or animation from the visual effects houses, they're really collaborative. So if you've got a way to try to beat an idea then they take it. The film's always open to suggestion from all the departments and that collaborative atmosphere is imbued in everyone. So if your idea gets beat, you're kind of like, "Oh, okay, bummer. But it's definitely better than what we had and just makes the movie better."

That's the end of the secret to the Russos, and the secrets to what was great about these films is that was the right people at the right time. I go back and reflect on it and I think it's without the team that was put together by Jen and Victoria [Alonso] and the brothers, these films couldn't happen the way they did. It was just exactly the right group at the right time.

Underdahl: Who had grown together too, starting with what we achieved on Winter Soldier up through what we thought at the time the Civil War was like the biggest thing. "Holy cow – how are we going to get that one done?" And then here we all get back together for these two. And what we learned along the way definitely really helped, I think, ultimately get the best out of all of us.

Earl: It was definitely sad to see that sort arc of films capped. But yeah, the Russos were great as the whole team was. Just super collaborative: they know what they want, but not afraid to say, "Hey, what do you guys think?" Is there a way we could do this differently?" And just really a pleasure to work with.

Underdahl: Pretty darn cool under fire too, these guys. The enormous stress they must have been under. We never saw it.

Were there little Easter eggs that you guys put a lot of effort into simply because it meant something to you to kind of sneak this in or get this one little two-second shot of somebody?

Aitken: Howard the Duck!

Underdahl: Howard the Duck. That was one of my favorites.

DeLeeuw: There was an old version of Infinity War where Quill stole Howard's ship so we knew that's where Howard was and so it was kind of the like, well, "Let's have Howard come out," and then it was like quick hustle to get the asset to Weta and have them drop it into the shot.

Underdahl: But I mean very, very last minute, too, you guys were absolutely buried, and to ask you to do yet one more thing--

Aiken: Ah, we really wanted to do it! It was a cool idea. And he's in it for about 17 or 18 frames of the movie, so I think in terms of effort per time on screen, it was up there, but it was worth it to have him. And for the fans, you know, when they get to see him there's a huge cheer. So that was great.

*****

Avengers; Endgame is nominated for three Visual Effects Society awards, and has also made the shortlist for the forthcoming Academy Awards visual effects nominations.