HBO's adaptation of the critically acclaimed video game, The Last of Us, has been one of the most highly anticipated TV shows of the year. To create the show's immersive world, the team at DNEG faced the daunting task of creating massive environments that spanned locations in Austin, Boston, Kansas City, Salt Lake City, and Jackson, Wyoming. With tireless efforts from an in-house team and multiple location shoots, they were able to capture all the necessary data to bring the world of The Last of Us to life on screen.
We'll dive into the behind-the-scenes with Juan Carlos Barquet (DNEG Environments Lead), Melaina Mace (DNEG CG Supervisor), Nick Marshall (DNEG DFX Supervisor) and learn about how they brought this highly anticipated show to life and how MattePaint was utilized to achieve their creative vision.
How did you approach the creation of the vast amount of environments on The Last Of Us?
Melaina: Having our team be physically on set was incredibly important for us on this project since we needed to capture LiDAR for all the locations that we were building 3D environments for. This also included roundshot capture for all of the Boston locations. Having all this information available was incredibly useful later on, especially as we were able to capture some of the key buildings that we used for Episode 2 and onwards.
Another key benefit of being on set was the ability to collaborate with Alex Wang, HBO VFX Supervisor, on the fly to ensure everything was set up correctly and that our shoot teams were able to capture what they needed. This was especially important when it came to the lighting, as our in-house shoot team at DNEG accompanied us on set for multiple location shoots.
Nick: We also did a bunch of splinter shoots in various cities, such as Boston and Salt Lake City. Knowing what we were capturing on set and having that in post-production was invaluable. Of course we didn't use all of the captured data, but having coverage of everything was incredibly useful later on. It allowed us to capture light and specific street/building references that we knew would be needed, which saved us a lot of time.
How did you manage the decision to switch between 2D and 3D methodology?
JC: We took a case by case approach to the visual effects work, using a combination of 2D and 3D techniques for many of the shots. For example, in some shots we would use 3D tools to dress the environment with vegetation and then push that through the CG pipeline but then finesse the shot at the end with the Matte Painting team to make it look more photographic.
There were a few shots where we decided upfront that a 2D or 2.5D approach would be better, especially if the plates had real buildings in them and didn’t require complex destruction. The overpass sequence, for example, showed many buildings that existed in real life but needed to be damaged. Since there wasn't much parallax in those shots, we opted for a 2D or 2.5D approach. Although, shot-to-shot work was often needed, because the lighting conditions and perspectives were changing.
Interestingly, the slanted skyscrapers in downtown Boston were initially going to be treated in 2D to damage the structures in a more subtle way. However, in the end the client decided to make it a more intense destruction shot so we went for a 3D approach. Since the leaning tower was such an iconic shape from the videogame, we're glad they decided to go that route for the show.
Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.
Melaina: The initial plan for this particular angle was to add broken windows, weathering, and some overgrowth, and you can see the two buildings on the left were actually in the original plate. There was a point midway through production where they wanted to go bigger, and we got a new concept which had much more intense destruction. Fortunately, the left and far right buildings had been planned as full CG builds, so we could adjust them to have more destruction.
Nick: It was one of our biggest sequences, and a significant effort was put into it. Two or three people worked on specific building destruction simulation, and we had a team constructing the assets. Building them, dressing all the interiors with furniture, and constructing the internal structure of the buildings all took a lot of time and effort.
Vegetation grown over everything looks great in theory, but it's much harder to achieve in practice due to the changes in perspective and angles.
Overall the team did a fantastic job, and the shots turned out great. Working on post-apocalyptic, destruction, and zombie apocalypse scenes is really interesting, but not without its challenges. Nonetheless, it's moments like these that make working in VFX rewarding. Being able to bring fantastical worlds to life and make them feel real is an incredible experience.
What was the biggest challenge in the creation of the environments on The Last Of Us?
JC: One of the things that makes these types of environments so challenging is ensuring that they don’t become too noisy or overwhelming, as this could easily distract from what was happening with the characters. We needed to find ways to add detail and clarity without going overboard.
Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.
In order to achieve this, we used light to break up some of the bigger shapes and make sure the viewer's eye had somewhere to rest. We paid close attention to the details, even in places where people might not necessarily notice. For example, we made sure that the plate building reflections changed to reflect the changes in the background.
After the characters see the skyscrapers in Episode 2, they keep walking through downtown Boston and encounter a lot of destruction. This included a collapsed building and a crater, which our department was responsible for creating and these were some of the biggest challenges we faced.
One shot, where the group are looking down a street at the collapsed building, required a mix of different techniques, with the background building being 3D with some matte painting and CG vegetation on top. Despite the time constraints, we were able to match everything pretty closely to the plates.
Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.
Finding the right balance between destruction and leaving some areas untouched is always a fine line that can make it look believable or not, and it can be really easy to make it too busy and distracting. It was quite challenging but we were able to pull it off and create a sequence that was both impactful and visually stunning.
What led you to want to utilise MattePaint content on the show?
Melaina: Accessing high-quality HDRIs and visual references is imperative for us to achieve a realistic and convincing final product in any production. We have been fortunate enough to use the HDRI sequences provided by MattePaint on multiple shows over the past few years and, on shows that require extensive environmental work, we find utilizing MattePaint Assets and HDRIs to be really valuable as we can be so much more efficient.
As an example, on The Last Of Us, we had a few sequences that required full sky replacements and precise reflections on buildings. The HDRIs captured by MattePaint helped us to achieve these effects effortlessly. I remember one night we were working on a nighttime shot with lightning, and MattePaint had recently shared a striking lightning shot that they had captured earlier that day! Having access to such content was truly invaluable.
While we try to capture HDRIs on-set as much as possible, there are often times where this is not possible - usually due to time constraints - so having a library of images and visual references readily available is essential.
Was there a specific shot or sequence that particularly benefited from that content?
JC: Yes, this shot below serves as an excellent example of a MattePaint HDRI, which required minimal editing. The scene includes a combination of live-action and 3D elements with matte painting in the quarantine zone and beyond. This is one of the first big VFX shots for the whole show, making it really important to convey a snapshot of the world we were entering in a single shot. We had to eliminate unnecessary details and focus on storytelling and the sky plays a key role in that.
Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.
Nick: One of my favourite sequences in the show is where Ellie is walking across a plank between two buildings. The lighting setup for this was quite challenging. The scene was relatively long and involved moving across different locations, from exiting the window to crossing the plank and finally looking out at the statehouse. To accurately depict the environment, we had to be very selective about the buildings we used and how they looked. However, we faced a significant challenge in dealing with the varying lighting conditions of the plates captured on set, because it changed so much depending on the time of day they were shooting.
Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.
When we first received the shot, it was inconsistent, with slight differences in the temperature of the lights and shadow quality compared to the real sun. Naturally, you have to avoid sudden shifts in lighting conditions, so we had to set it up so that we maintain lighting consistency with the earlier parts of the sequence. We addressed much of this through the MattePaint HDRI’s, as we were able to find several selections which matched the lighting throughout the sequence and allowed us to produce a memorable and effective shot for the show.
Have any of your workflows changed since incorporating the MattePaint libraries into your pipeline?
Nick: Yes, definitely. Before we had access to MattePaint we had a time consuming process of finding HDR’s from onset, online or having a matte painter create a sky which can take weeks. This often results in a disparity between the sky that the audience sees and the sky that is used for lighting. This is no longer an issue for us with access to MattePaint as the HDRIs offer both high-resolution and high-fidelity so they’re suitable for lighting as well as to use in-shot.
MattePaint is doing some great work for the industry as we are now able to easily find a sky from thousands of options and, even if it’s not perfect, we can paint it a little bit, or blend 2 frames together very easily.
For example, the first visual effects shot of the show has the Austin skyline in the background as we drive away. We found a MattePaint sky that almost exactly matched our plate, and we were easily able to blend it in.
Another challenge we used to face was that often you’ll find the perfect sky which the client approves but has requested changes. Maybe some clouds are distracting and need to be removed, or they want the sunlight to be an hour later. Augmenting a sky like this can be very time consuming and can go wrong quickly. It can require tricky gradient matching or trying to find other clouds that fit the same style. Clouds are so specific, they're all unique and work as a system, so making a sky is never as good as real-life.
Having access to full sequences across an afternoon is a game-changer too. If we find the perfect sky but the client feedback asks for it to be later or earlier in the day, we can go back to the website to grab it and drop it in. This saves so much time as artists no longer have to spend weeks doing the perfect skydome simply to get notes on it. We can now iterate through multiple options to clients and effects and have a much better starting point.
Most of the skies that we have done in the last few years have been based on MattePaint HDRIs, and we have had to augment so little in those, which ends up being a huge time saver, allowing us to focus on other elements of the shot.
Do you have any tips or advice that would be helpful to our community of aspiring matte painters and generalists?
Nick: It's essential to remember the fundamentals as an artist. It's all too easy to become lost in the latest technology and software releases, bombarded with claims of lightning speeds and AI capabilities. While it's good to remain curious about industry developments, ultimately, the quality of your work depends on your artistic vision and skill level. Training your eye to understand composition, shape design, perspective, and lighting takes a lifetime of practice. Delicately handling these elements injects an authenticity that sets your work apart. So, don't get too distracted by technology. Instead, focus on the fundamentals and develop a keen understanding of what makes an image truly work. By honing your craft, you'll be able to create work that is both authentic and exceptional.
JC: I believe that learning the concepts behind representational painting, whether it be digital or traditional, is essential. It provides a foundation in composition, perspective, color theory, and lighting that guides artistic decisions, regardless of the software used. While software can be learned later, this foundation always helps. I also highly recommend the book Color and Light: A Guide for the Realist Painter by James Gurney, which I read as a teenager and continue to reference. Additionally, learning photography and how to shoot good photos with a DSLR is also helpful for artists.
Melaina: My number one tip would be to always use reference(s) when creating your work. When reviewing a shot, I can usually tell if an artist has referenced real photography or not. Using reference(s) allows you to capture details that might otherwise be missed and can help to ensure that your work has a level of realism that might be missing otherwise. Even if you're creating something that looks quite surreal, including reference can help your audience to better understand your creative choices and see how they relate to reality.
Conrad: My primary advice is to start with simple tasks. Instead of creating complex post-apocalyptic matte paintings, for example, start with something like a sky replacement and focus on making it as photo realistic as possible. To get a job in the industry, you don't need to know everything about composition and lighting. As a junior artist, your first tasks might involve cutting out elements or replacing text on signs in a scene. These are basic tasks that help you build a foundation and prove to your supervisor that you can handle more complex work. Once you have proven your skills, you can start focusing on more advanced skills like composition and perspective, which are essential for career advancement.
