What you should know: Video game software is making the leap from gaming to film and TV production. From previs to real-time production on set, tools like Unreal Engine are becoming indispensable for studios.
Why it matters: VFX studios can get ahead of the curve and offer cutting-edge resources to clients—with the right tools.
- The History of Unreal Engine
- How Unreal Engine is currently being used in film and TV
- How VFX studios can further leverage Unreal Engine to completely transform their production pipeline
The late ‘90s saw monumental development in personal computers and the games you could play on them. Graphics were rapidly improving, and though laughable by today’s standards, the development of first-person, 3D graphics made video games feel far more immersive. The internet was quickly expanding to consumers, making it easier to participate in online gameplay, join discussions in gaming forums, and even learn how to program games yourself. And the games that were being released in that period are titles we still say with reverence—Doom, Sim City, Duke Nukem, Half-Life, StarCraft, Myst, and so many more.
However, Unreal Tournament was in a league of its own: the fast pace, expansive maps, and computer AI opponents that—for the first time—actually seemed smart. A community of players began to grow around the game, who were modifying the game, making it their own, and sharing their creations with the community. It was all thanks to a game development company with an extremely bright future, and a piece of software they created called “Unreal Engine”.
Epic Games and Unreal Engine
Epic Games was founded in 1991 and is one of a handful of software company success stories that have earned it a rightful place in gaming lore. Their first few games were commercial successes, but what really put them on the map came at the end of the millennium when they released Unreal in 1998, quickly followed by the sequel, Unreal Tournament, the following year.
The gameplay, graphics, and capability of the game set it apart from the myriad other first-person shooters on the market. But beyond the game itself, Epic Games made a decision that sprung the game into legendary status. They opted to ship the game with the very tool – the Unreal Editor - used to build levels and gameplay, allowing players to customize the game and make it their own. It’s one of many game engines designed over the years, but it is far and away the most successful and widely used.
“If you think about technology that has stood the test of time, Unreal Engine is one of the biggest,” says Jacob Feldman, software and rendering solutions engineer at CoreWeave, who moonlights as an Unreal Engine authorized instructor. “It has over 20 years of code in there, so it’s a significant, sophisticated product.”
Realizing they had a winner with Unreal Engine, the team at Epic began to license the software to other developers to build their games off of it. Fast forward to 2017 when the studio released Fortnite, a cultural sensation that would further cement the legacy of Epic and Unreal Engine.
More recently, Unreal Engine is playing a role in another cultural moment that propelled the tool into a whole new industry—filmmaking.
Image credit: Lucasfilm Ltd. / ILM.
From Computer Games to Filmmaking
In 2019, Disney released the award-winning series The Mandalorian as part of their marquee lineup for their new streaming service, Disney+. The show was an instant hit and continued with the long history of Star Wars productions pushing the technologic boundaries of the film industry. The Mandalorian displayed a huge leap forward in set design through its use of real-time rendering of sets using Unreal Engine—effectively making the green screen obsolete.
“Unreal made the leap to broader industries like film after Fortnite was released,” says Feldman. “It became very clear that a game engine like Unreal which is designed for flexibility, performance, and visual quality had more applications than just games. This was especially clear as more powerful hardware began to facilitate things like real-time ray tracing and other visual effects which were just not possible 5-10 years ago. The VFX industry has some of the most stringent requirements of any visual medium, so Epic made a serious commitment to support the sorts of tooling that VFX studios require.”
Set Extension v3.0 – Real-Time Graphics
Filmmakers have always found ways to trick the eyes of the audience into thinking they’re seeing something bigger than what is really there—what’s called “set extension” in the industry. In the early days of Hollywood, this involved literal painting of scenes that were then seamlessly integrated into sets to provide the effect of an endless expanse.
Later, green screen technology came to be the preferred method for creating expansive worlds. However, this method provided some significant issues. For the actors and crew on set, it can be difficult to deliver convincing performances in front of a blank, bright green backdrop. Actors are not seeing the world around them, and thus not responding to it in a way that convinces an audience.
But what if there were a way to create expansive worlds on command that actors could see right in front of them? It would almost be like dropping these actors into a video game and having them perform in that world, with all the sights available to their own eyes.
Enter Unreal Engine.
The Mandalorian creator Jon Favreau went to the team at Epic Games to see if they could help him do essentially that—dropping their actors into a video game. By using massive high definition LED screens surrounding their set, the team for The Mandalorian took the old Hollywood concept of painted set extensions and brought it into the twenty-first century. Now, those painted backgrounds were living, breathing environments with dynamic lighting and movement. The actors could see them and respond to them. The physical elements in the foreground blend seamlessly into the virtual backdrop.
But it gets better—using the game engine’s technology, they can tie the movement of the camera to the background, allowing it to change with the camera to perfectly match the movements and provide smooth parallax movement that is indistinguishable from a real-world shot.
Essentially, they created real-time graphics at the heart of film production. There’s no cleaning up in post-production, these effects are done in camera in real time.
“It's an enormous help for the cast and crew to be able to see their environments in the moment,” says Feldman. “An actor today has to imagine on a green screen set what’s happening around them. This advancement let’s them actually see it.”
Beyond making life easier for actors, the real-time environments also make the jobs of the post-production team infinitely less stressful. On a green screen set, reflections are a constant battle. Any sort of shiny material will reflect the green environment and ruin the effect of the VFX. Those reflections must be removed and then the virtual world added into the tiny reflections in their glasses, helmets, or pieces of metal in the environment. The real-time environment takes all that out of the equation, since the reflections caught in camera reflect the world as it is intended to be in the story.
The Next Generation - Unreal Engine 5
More exciting developments are on the way for VFX studios looking to use Unreal Engine. In May 2021, Epic Games opened early access to the fifth generation of the tool, Unreal Engine 5. With the full launch slated for next year, Unreal Engine 5 is poised to shake up not just the video game industry, but the film industry as well. With new features that vastly improve the rendering of highly detailed geometry and how lighting responds in real time to the environment, Epic is making a strong push to create a tool that can produce computer graphics indistinguishable from many of the legacy tools commonly used in the VFX space today. As LED screens continue to develop more capabilities which improve final frame photography (and prices continue to drop), setups like The Mandalorian’s Unreal Engine-powered stage will become more accessible—and more powerful.
Making Unreal Part of The VFX Pipeline
Even for productions of a much smaller scale than a Disney television series, Unreal Engine is changing the way filmmakers work. While not every production will have the budget (or even need) for graphics on the scale of The Mandalorian, many in the industry view the use of game engines in filmmaking as critical to the future of movie making. As the software’s tool set is widely available, with extensive learning resources, amateur and indie filmmakers are using it to create scenes that would otherwise be unavailable to them without a Hollywood budget. These filmmakers are essentially creating video game cutscenes that rival any cinematic production, opening a new world of possibilities.
“When you look at what a VFX pipeline looks like today, a great deal of effort and infrastructure goes into producing the intermediary artifacts required for production,” Feldman explains. “The longer it takes to visualize revisions and map out scenes, the more likely it is that you will interrupt the artistic process or jeopardize the quality of the final output. Unreal is poised to become a one-stop shop for set planning, previs, layout, final frame rendering, and background replacement. That's with the benefit of having a highly immersive and realistic environment for actors to act in.”
As a certified Unreal Engine authorized trainer, Feldman is well-versed in introducing studios to the power and potential of this tool. While its use in Hollywood productions is still seeing early adoption, he sees Unreal Engine becoming as common as the green screen on Hollywood sets in the not-so-distant future.
Even for the film purist, the use of real-time rendering is a massive improvement on the ubiquitous green screens and post-production set extension seen on everything from The Avengers to The Wolf of Wall Street. Environments on the LED screen provide real reflections and luminance that is captured on-camera and harkens back to the Golden Age of Hollywood, but with a modern twist.
“What productions like The Mandalorian are doing isn’t really anything new,” says Feldman. “Unreal Engine for final frame photography is essentially taking what old Hollywood productions used to use—static paintings, rear projection, or miniature—and turning it into a responsive environment that is no less artistic or handcrafted, but is infinitely more capable at producing immersive worlds.”
For future-focused VFX studios, this represents an opportunity to offer a service that Hollywood studios will be eager to take advantage of. Afterall, a production need not fall into the sci-fi or superhero genres to benefit from cutting-edge VFX.
How VFX Studios Can Get Started
Unlike switching from one 3D tool to another, integrating Unreal Engine fills a completely new role for studios. This makes it a bit difficult to begin to integrate into your pipeline. “A lot of studios we talk to are certainly interested in trying these tools out, but they need help getting started and preparing their teams for the push,” says Feldman. “They need someone to sit down with them and provide real-time feedback and instruction, and that’s where the training team at Epic comes in.”
As a certified trainer for Epic, Feldman helps introduce Unreal Engine to VFX studios and give them the tools they need to get started. He helps them understand how they might use the tool in various applications and what it takes to harness the full power of the engine. His knowledge, combined with CoreWeave’s specialized cloud solutions, gives clients the support they need today and the option to grow in the future with new tools like Unreal Engine.
“One of the number one uses of Unreal Engine and a great entry point for VFX studios is what we call previs,” explains Feldman. “For projects where you’re creating these massive 3D worlds, Unreal can let you get a sense of what a shot will look like before you spend hours and hours rendering it.”
Looking Ahead
As with most technologies in the VFX world, getting acquainted with Unreal Engine is all about staying ahead of the curve.
“Unreal Engine is going to open the doors for a huge amount of content, especially from small to medium sized studios that simply could not have been produced in the past without multimillion dollar budgets.”
Want to know what CoreWeave’s specialized cloud solution for VFX can do for your studio? Request a consultation today to get started.
.png)
.png)