Section 1

INTRODUCTION

Over a 10-year time horizon, many things in our fast-changing world will evolve – new technologies will be created, new processes defined, new partnerships formed – but in the year 2030, we can be sure that people will still enjoy watching audiovisual content. We can debate the form of that experience – it could still be enjoyed on a 16 x 9 screen, or maybe inside a headset, or projected on the environment surrounding the viewer. But one thing will be certain: audiences will still enjoy stories well told with technology that stimulates the senses and creates magic.

With this 10-Year Vision paper, MovieLabs draws a picture of that future, describing how media experiences will be created and prepared for distribution. But this is not an academic exercise; this future vision becomes a north star to guide the next 10 years of innovation. Articulating what the studios collectively want to enable technologically will help identify gaps in current technology, standards and processes and guide creation of a roadmap for how the industry will innovate to deliver on that vision. This paper should be read by technical leaders at vendors, software and hardware producers, and other studio suppliers to understand where studios want to go in enabling new technical workflows.

We describe a 10-year horizon to ensure that we can move technologies from early experimentation and prototyping through scale phases and into maturity.

We should be clear that the various technical innovations we expect are all developing at different time scales and many are available in early forms today. We describe a 10-year horizon to ensure that we can move technologies from early experimentation and prototyping through scale phases and into maturity. Once mature, these technologies can be made available as standardized processes and workflows for creators of productions on all scales on movies and TV shows. As we move through the next decade, we fully expect to adapt our Vision to accommodate new technologies that may accelerate beyond our expectations, and to threats or opportunities that may emerge.

This document and the underlying principles were developed by MovieLabs with contributions from technology leaders at major studios. We extrapolated current technology trends and expectations for future growth and innovation. Our scope focuses on scripted, non-interactive entertainment TV and movie content and is therefore not aimed toward video games, live events, game shows and other non-scripted experiences that have different content creation requirements and workflows – although we recognize that many of the key principles carry over into those use cases.

Today’s scripted entertainment is characterized by five distinct workflow processes that have evolved over the last 100 years. It’s worthwhile to benchmark these processes now because they will very likely change fundamentally over the next 10 years in ways that they have not previously.

Preparation and R&D – Pre-greenlight activities including developing a script, packaging talent, budgeting production scenarios and developing visual presentations required to pitch and greenlight the project. It also includes R&D of new technologies that might be used in the project.

Preproduction – Covers the steps after greenlighting involved in defining detailed plans and processes for production. It includes virtual production and previsualization (“previz”), which are used to plan more efficient principal photography and ensure the seamless combination of physically and digitally produced elements.

Production – The steps in capturing and creating content on set, on location, in animation, in VFX, etc. It includes lights, cameras, sets, talent, grips, green screens and huge media files.

Postproduction and Mastering – Often the lengthiest part of the creation process, this includes steps such as editing, adding visual effects (VFX), mixing/editing audio, color grading and creating dozens of international masters.

Distribution – Preparing and delivering numerous variants of the content to the owner’s distribution partners for onward delivery to consumers. Delivery includes theatrical distribution, physical media (optical disc), pay-TV services, broadcasters and over-the-top (OTT) internet services.

In 10 years’ time, entertainment productions could be produced in very different ways and today’s workflow steps could be rearranged or inverted dramatically.

WHAT COULD MEDIA CREATION LOOK LIKE IN 2030?

By 2030, new technologies, especially real-time photorealistic graphics, will create a more iterative creative process where the phases of early production merge together into

Pre-Photography – Script development; production design, including setting up much of the visual FX assets in full fidelity; CG scenes; previz of some or all of the scenes; and creation of everything except the live action, with an ongoing process of R&D and refinement of tools and workflows.
Principal Photography – Similar to today, but with the option of physical actors interacting in real time with digital assets and virtual characters. Photography on some shows will expand to include volumetric capture of entire scenes, use of light field cameras or capture of multiple simultaneous camera angles.
Final Postproduction and Mastering – Color grading, final sound, VFX finishing and editing.

Starting at the beginning of a production, major movies will be previsualized with photorealistic real-time engine (RTE) pipelines with such quality that a director or cinematographer can make lighting, shooting, performance and even editing decisions in an environment that seamlessly blends the physical and digital worlds. These changes unlock many more real-time iteration opportunities for creative decision-making at any point before, during, and after production. The artist will have flexibility to try new ideas and experiment throughout the creative process, seeing her/his work in close to final render quality generated by a real-time ray-tracing engine.

We can also envision principal photography where directors are freed from the constraints of limited sets and a select number of cameras, to a stage filled with hundreds of small cameras and sensors capturing entire light fields, scenes and performances simultaneously from every angle. These “volumetric capture stages” have emerged in the past few years for immersive media projects and experimentation, but have not yet become commonplace in studio productions. However, in the future, more innovative and elaborate setups that capture whole light and space volumes will be more prevalent. A new step in post-production would enable directors, DPs and editors to select which of these many camera angles should be presented to the audience.

We can expect that media creation workflows will be cloud based with every file (from first script, to camera captures, VFX assets and audio tracks) stored in the cloud. Applications that process the media will be cloud native, meaning the application directly communicates with cloud storage and runs on a virtual workstation hosted in the cloud rather than a workstation sitting beside the users. These cloud-based workflows can be inherently more secure than on-premise workflows since cloud-native production offers the opportunity to use a security architecture tailored for cloud workflows. Applications running on virtual workstations eliminate the need to duplicate and move files between different vendors, store assets on vendor sites and permit the content creators to maintain sovereignty over their assets.

We can expect that media creation workflows will be cloud based with every file (from first script, to camera captures, VFX assets and audio tracks) stored in the cloud.

These changes in content creation workflows can also be expected to impact consumer delivery and experiences of that media. By 2030, we expect the media elements that comprise finished movies and shows to increasingly remain separate as they are prepared and distributed, then be bonded together into consumer experiences by linked metadata. We can expect future consumers to enjoy media experiences that adapt the playback experience to them by rendering out a uniquely optimized version of the content package for the specific display (brightness, color, contrast, resolution), audio (speakers performance, proximity, placement) and environment (ambient light and reflections, noise levels) that the consumer is in. By using edge computation and rendering of targeted media elements, the industry can avoid creating thousands of delivery masters for every discrete consumer device, bandwidth and playback capability, and instead deliver uniquely what is best for that consumer at that time.

With audio experiences, we also expect innovation. In the past 10 years, we have moved from channel-based audio (5.1, 7.1) to object-based audio, in which individual audio objects can be placed and tracked in 3D space around the audience. During the next 10 years, we expect these trends to continue to include potentially millions of audio objects with a granularity down to audio for individual particles. This flexibility will require new AI-based tools to handle the complex modeling and interactions of objects. The result will be rich sound fields calculated by the interaction of sound elements on the screen as they are realistically rendered out based on the echo, reverberation and distortion of their location and their interaction with other sounds in the field. This level of fidelity is similar to the particle physics used in today’s game engines and the ray-tracing used to computationally calculate the individual light rays in a scene and how they move around, interacting, refracting and reflecting across objects in the scene. In fact, the audio and video technologies may combine to calculate both light rays and sound fields simultaneously based on specific environments. A new wave of creative tools will need to be developed to allow the production team to then adapt and manipulate this calculated reality to tell the stories they want to tell.

We can also expect many mundane or repetitive production tasks to be accomplished through artificial intelligence–enhanced tools, automation, bots and processes.

We can also expect many mundane or repetitive production tasks to be accomplished through artificial intelligence–enhanced tools, automation, bots and processes. While it’s hard to predict every possible use case for such tools, we expect advances to speed creatives’ work by animating background VFX characters, doing first-pass color grades with various looks for directors to pick from, and automating rotoscoping, background removal, and “world-building” that generates realistic 3D environments for directors to walk through and edit. Our objective with this vision is to enable a world where AI-based improvements can be efficiently developed and deployed at scale.

While not all titles will benefit from all of these new capabilities by 2030, these advances will be increasingly accessible to creative teams of any scope, budget or schedule. As always, there will be financial considerations, tightening schedules and high production expectations that are common to all forms of content. The tools, processes and technologies that enable a high-budget movie to do amazing things today will eventually trickle down and enable lower cost productions to benefit from the same innovations in the future. We expect our 2030 future Vision to apply to everything from blockbuster movie franchises down through episodic productions, documentaries, independent films and even student films.

Today, some of these developments may seem like fantasy, but examples and early prototypes of all these technologies are already appearing. To unleash their full potential and enable their value to be commonplace by 2030, the industry stakeholders (studios, storytellers, independent production companies, post-production and VFX companies, application/technology providers and infrastructure providers) need to start working on these innovations now. They must establish new processes, procedures and standards. There is much to be done to create these new media workflows and realize the benefits of reduced production costs, shortened production schedules and productions freed from the constraints of today’s workflows.