“Star Trek: Strange New Worlds'' is one of the many new productions embracing the benefits of Virtual Production. By utilising Pixomondo’s world-record-sized Virtual Production stage powered by MattePaint Assets, the show was able to achieve a totally new visual experience for television audiences.
Dieter Magallan
Virtual Art Department SupervisorDieter has been with Pixomondo since 2016 and recently moved into the role as Matte Painting Supervisor in their Toronto location. He has worked on over 20 high-profile VFX productions.
Mujia Liao
Head of Virtual Art Department / Art DirectorFor the past decade, Mujia has worked mainly as a concept artist / matte painter. She has both supervised and worked with the team at Pixomondo since 2016, delivering over 40 titles.
How did you approach the Virtual Production work for Star Trek?
We have two phases in the Virtual Production pipeline. The first one being visualisation. This is where we block out the scene with proxy geo and lay out all the parts that we need. We have concept artists working on it, sketching out the idea and we start to loosely paint things in. Because there’s a Virtual Production stage, we start selecting a sky right at the start because it will impact so much of the look of both the physical stage environment and the digital one.
"...as soon as we sort of have a blocking of a layout and we go into the lighting options, that's when we introduce the different HDRI, like right away...."
After that, we will block out the scene with either Kit Bash elements or marketplace elements to get us started, or we'll model something very proxy or sometimes we will receive those assets from the client. But as soon as we sort of have a blocking of a layout and we go into the lighting options, that's when we introduce the different HDRI, like right away.
"...the MattePaint HDRIs are just ready to go, it’s really easy to play with different looks..."
How is Virtual Production reshaping the way you work compared to VFX?
Everything is becoming more fluid and clients want the option to change things at any stage. This has been happening for some time now, but Virtual Production has accelerated because it’s real-time, so when the client realises they can change the lighting in real-time, they ask to see options. We’ll present five or six different lighting options to the client in Virtual Production, and because all the MattePaint HDRIs are just ready to go, it’s really easy to play with different looks.
While working on Star Trek, we had a client meeting where we presented options of a cloudy sky, sunny sky and sunset sky all for the same environment during one meeting and we were able to make a call very, very quickly based on what they liked.
If there's any specific requests based on that first presentation, we could mock it up on top of the HDRI very quickly as well. Most of the time those requests were to open a pocket of sky to reveal a planet, or removing a particular cloud because it distracts from the story. Once we have the buyoff from the client, we go into our next phase, which we call “realisation”, this is where we take everything to the final look. So that’s where we do the DMP for the sky, environment and comp. You know, the typical VFX pipeline.
"...the HDRIs are 32-bit it’s really easy to adjust them live..."
Did you run into any challenges that the HDRIs helped to solve?
With virtual production, you have to pretty much cover the full 360 even though you don't end up seeing the whole thing most of the time. So it means a lot more 360 skydomes, and environment work. This is very tricky because you have to think about the scene as a whole, and you can’t get away with the same stuff that you can with a normal VFX shot.
For this production, the client wanted a very specific romantic look for the Vulcan Planet Sky and sent over specific references they wanted to match. It was really lucky timing because by this time there were so many MattePaint HDRIs to choose from. We were able to find something that worked as a base really quickly, and then we painted over it to add more of the specific romantic look the client wanted.
Colour management is also a challenge because the LED stage is very sensitive to colours and treats the Red channel differently to normal screens. So what you see in photoshop often looks really different in Unreal, and then looks different again on the LED stage. So it can become a nightmare to have consistent colours. Oftentimes the only way for us to see the true correct colour is through the final camera, through the actual lens.
We use that as our final buy-off on colour so it’s really important that we can easily change colours on-the-fly in Unreal while at the stage, and not be tied to a traditional workflow where you have to adjust everything offline. Because the HDRIs are 32-bit it’s really easy to adjust them live, and we’re really embracing this workflow this year as it’s a game-changer for our efficiency.
"...exactly what we need for this workflow because they are “drag and drop” ready..."
How is the Asset workflow changing with Virtual Production?
I think the industry in general is moving towards an “Assetized” workflow where you purchase “building block” assets to use in your workflow. The MattePaint HDRIs are exactly what we need for this workflow because they are “drag and drop” ready.
Obviously it's not the only thing that we have to do for the environment, you have to do the mountains, cities, etc, but MattePaint has the assets for the natural environment and we pick up 3D asset kits too. It’s important because we are dealing with very tight deadlines for productions.
"...it’s become really fun to explore the options and experiment with different lighting..."
What is the best part about using the MattePaint HDRIs?
There just isn’t a comparison to the HDRIs. The way everything is shot as a time-lapse means we can really art direct the way the sky looks. If the client likes the sky but wants the sun obscured, or the sun revealed, we can just skip a few frames forward or backward.
Often we will pick two or three HDRIs and then we select different portions of each that works well and combine them. Because you have different sequences shot at the same elevation of the sun, it makes it really easy to just line it up in our scene and it’s so easy to quickly reveal the areas we like from one HDRI and combine it with another.
Historically the sky was a bit of a chore because it takes so long to put together something that works and you never know if the client will decline it. Now, it’s become really fun to explore the options and experiment with different lighting and we always know there is a high-res 50k that is production ready.
If you would like to check out the MattePaint HDRI library, you can download the entirely library at 2k completely free! Just sign up for an account with the link below.
MattePaint Pty Ltd
