Prime Focus Technologies Creates a Channel 4 Cloud Supply Chain
1.65Million
library assets (7.5Petabytes of data) moved to the cloud
300 PARTNERS
around the world
17,OOO HOURS
of new media every year
5,OOO DAILY REQUESTS
for files across linear and non-linear platforms
10,OOO HOURS
of monthly VOD output to 15+ destinations
Summary
Prime Focus Technologies (PFT) demonstrated the value of the cloud management principles of the MovieLabs® 2030 Vision by implementing it’s CLEAR® Software-as-a-Service (SaaS) platform for Channel 4 (UK). The service was designed from the ground up as a multi-cloud environment storing all media assets in Amazon Web Services (AWS) S3 buckets while managing a multi-vendor supply chain network and publishing to downstream distributors from services running in Microsoft’s Azure cloud.
Channel 4 is a UK “publisher-broadcaster,” meaning that it commissions or acquires all its programming from independent producers, the first UK broadcaster to do so on a significant scale. As part of its next-generation business architecture Channel 4 chose to transition from a single vendor handling all finished show masters to a multi-vendor media supply chain. In doing so, it sought a platform that could orchestrate workflows and associated business processes across multiple suppliers and vendors for each asset, while providing Channel 4 with complete visibility into the process. PFT leveraged its CLEAR Media ERP (Enterprise Resource Planning) platform and adapted it to serve as a multi-vendor supply chain network to meet Channel 4’s objectives.
Background
Channel 4, like many major broadcasters, sources content from around the UK and around the world with a complex network of producers and licensors providing material. All of that content needs to be aggregated and processed for both linear playout (to broadcast streams), video-on-demand via cable and satellite partnerships, and online distribution with consumer streaming. The process to source and aggregate all the files, confirm appropriate formats with correct metadata and essence media, transcode, process and then store millions of files for delivery is a daunting logistical challenge. Channel 4 had relied on a single outsourced vendor to manually manage the entire process but wanted a more contemporary cloud-based solution with multiple vendors to provide better business resiliency. Channel 4 previously had no visibility into content processing and delivery and wanted a full view of the entire media supply chain across multiple vendors.
Instead of duplicating and sending files to and from vendors and licensors, Channel 4 wanted a ‘single source of truth’ for each asset, enabling them to keep track of work performed by all vendors and manage versions and changes throughout the process. The system also had to manage deliveries to downstream distribution partners from the same single source of truth, both for new media assets and existing content in the library. In addition, Channel 4 needed the scale to migrate 7.5 Petabytes of library content while expanding steadily to manage new content as it arrives from production partners.
Channel 4 also required a system that would integrate cleanly with its existing broadcast management and rights management systems, bring automation to operations, scale dynamically to business needs, and embrace modern supply chain principles and standards. To enable the transition to a multi-vendor supply chain, CLEAR also needed to support a single workflow defined by Channel 4 for all external vendors.
Solution
PFT configured its CLEAR SaaS application to enable a unique multi-vendor, multi-cloud supply chain network for Channel 4. PFT created a new virtual repository consisting of AWS S3 buckets for all incoming and outgoing media files, designed for use with content creators, media processing vendors, and downstream distributors that were sending media to Channel 4 or receiving files for processing from Channel 4. The CLEAR application itself is hosted in Microsoft Azure and manages all traffic to and from the virtual repository, as well as a set of PFT applications that run on AWS virtual machines to perform certain media processing steps (auto Quality Control (QC), content identification, initial validation, etc.) in the S3 buckets.
CLEAR automates communications with all content providers and ingests content, e. g, commercials, directly into S3 buckets, keeping all providers aware of scheduling deadlines and requirements. CLEAR then sends automated work orders to creative servicing vendors along with links to cloud-hosted content to initiate jobs for work that needs to be done. Status notices are exchanged automatically with vendors and progress is tracked until work orders are completed and the updated content is returned to cloud storage. Clear also publishes final mastered media to all downstream distribution points, including for linear play-out and non-linear digital distribution.
As part of the solution, PFT created a cloud-based archive system for Channel 4 and ingested 1.65 million files (comprising 700,000+ video assets and 450,000+ audio/subtitle files) into the archive as an initial migration. CLEAR automates storage and movement of media from working storage to the cloud archive tier to optimize for restoration time and costs.
CLEAR has a Schedule feature that helps manage the complexity caused by changing media priorities that can be hard to manage across a multiple vendor ecosystem. Channel 4 uses the Schedule feature to assign priorities to jobs which are then automated. If the business decides priorities need to change, the Schedule feature can automatically adjust the queue, reprioritize work and communicate the changes to all impacted vendors automatically.
CLEAR receives broadcast and Video on Demand schedules from Channel 4 broadcast and rights systems and creates an advanced forecast for Channel 4 vendors for the content and material needed. That forecast is updated by Channel 4 many times each day, with Channel 4’s evolving business priorities communicated to vendors through automated work orders. CLEAR then drives the workflow by tracking task priority, job completion status, vendor SLAs, and late deliveries, with a dashboard for Channel 4 to oversee the processes in real-time.
Using the CLEAR advanced forecast function, service provider gets view of upcoming work with expected due date.
Architecture
CLEAR is a cloud-native platform that was built for hybrid deployment models. Though CLEAR, in this case, is hosted on a public cloud in Microsoft Azure, the content is stored in storage pools, including AWS S3 cloud storage and on-prem infrastructure. CLEAR’s media processing functions operate where the content is stored, utilizing adjacent resources on-demand, including micro-services for auto Quality Control (QC), transcode engines, and packaging solutions. CLEAR has multiple microservices that can be configured for each workflow, including a cataloging tool for time code capture, a content identification tool, and quality check tool.
The CLEAR implementation also has an API that Channel 4 uses to integrate directly with its business operations and rights management systems, allowing Channel 4 to communicate and update business priorities to the CLEAR system many times per day. Those API calls then configure workflow prioritization to send automated updates and schedule changes to vendors, suppliers, and distributors.
CLEAR manages and automates Channel 4 workflows across three phases of content management:
Inbound Process of Content Acquisition
All producers of commissioned content are onboarded by Channel 4, which creates logins in CLEAR to allow upload via the user interface or API directly to cloud buckets. Once the content provider uploads its content, CLEAR initiates a series of automated checks based on business rules and previously established supplier trust levels. Those include virus checks, initial runs against automated QC parameters, and confirmation of content identification metadata. Follow-up manual QC and fixing of metadata or other identified issues is managed by CLEAR with automated work orders either to vendors or Channel 4 program managers, depending on the error. As a further step, segmentation timecodes and audio track layouts are populated automatically by CLEAR or a Prime Focus Technologies Operator to ensure accurate automated QC and loudness assessment.
Media Processing – Transform/Enrich/Collaborate
If Channel 4 requires editorial changes or versioning to the ingested media, program managers communicate via API calls to the CLEAR system, which then sends automated work orders to post vendors and manages the workflow until the updated assets are ingested back into the correct S3 bucket. Validation and QC checks on the new versions are again performed according to business rules. A “super trusted” ecosystem has been created to eliminate repeated processes. This means that items sent from PFT, and new items from qualified vendors arriving at PFT, are validated with additional mandatory QC metadata delivered with media content from trusted suppliers. Apart from a virus check and validation, including a checksum confirmation, the item is not QC’d again, and the receiving party does not need to recheck the metadata. If the required QC metadata is not provided, however, the system automatically triggers a QC task.
Outbound Distribution – Linear and Non-Linear
The outbound transcode and delivery is entirely automated. Upon receiving automated messages from Channel 4 or other vendors via API or user interface, CLEAR restores content from the archive (if required), performs clipping or stitching if requested for the partner profile, transcodes, packages, and publishes the final media to its destinations. Rendition Rationalization, i.e., transcode once and reuse for multiple deliveries, is done across ten renditions to repurpose files for redelivery to other destinations within specific timelines. In the case of Channel 4, multiple All 4 platforms use the same content format, so CLEAR content publishing optimizes the number of transcodes across different requests, thereby reducing unnecessary transcoding and saving transcode time, storage and costs.
The high volumes of files with multiple renditions are prepared for both linear broadcast and loaded to the playout systems but also a myriad of different streaming platforms, including All 4, Amazon Prime, YouTube, Virgin Media Ireland, Virgin Media UK, Sky On-Demand, SkyGo, Kantar Media, Sound Mouse, and Freewheel.
BENEFITS
The CLEAR multi-vendor supply chain network orchestrates ingestion, archive, media processing, and distribution for Channel 4 across both linear broadcast and streaming business segments, providing automated tracking and task management for multiple suppliers and vendors. The result is a highly flexible, transformational, and modern supply chain that integrates with Channel 4 business systems and gives Channel 4 the agility to achieve its vision by keeping its supply and distribution chain in sync with constantly changing business requirements.
CLEAR also has enabled Channel 4 to have complete visibility into operations across vendors and manage diverse processes through real-time dashboards. The platform also simplifies work for vendors working with Channel 4 and the programmers providing content to them as they can all now use one platform to manage all media.
The new SaaS platform manages collaboration with both cloud and on-prem media processing, obscuring away the infrastructure complexity and enabling interaction with a common virtual repository accessed by all participants. In addition, both the operations and storage capacity of the system are scalable with the ability to add new cloud resources as needed in the future.
Alignment with MovieLabs 2030 Vision Principles
PRINCIPLE 1
The core of the CLEAR solution for Channel 4 is centralized content management that maintains all new and archive Channel 4 assets in cloud storage buckets. The system demonstrates Principle 1 of the 2030 Vision, showing why a single cloud location minimizes data movement and copying, reducing the risk of data loss and minimizing time waiting for file transfers. URLs for cloud media can be immediately shared with vendors so they can directly access the media associated with tasks sent to them. Today, some media is still copied from the cloud to vendors local infrastructure for creative tasks that require local processing, but that is expected to diminish over time as more microservices and applications are built to run in the cloud adjacent to cloud storage.
PRINCIPLE 2
The CLEAR application is hosted in Microsoft Azure and manages media assets stored in AWS S3 buckets as well as automated media processing jobs on that media. The CLEAR task management app in Azure manages microservices running on AWS virtual machines to process the content (for example, QC, encode, transcode or tagging). These services demonstrate the value of hosting the processing tasks where the media is stored, instead of moving that media to Azure or another compute service. By moving applications to the media Channel 4 minimizes data movement, speeds up tasks and eliminates egress costs incurred when media chases tasks and applications.
PRINCIPLE 3
When media must be processed by creative service vendors, CLEAR sends automated work orders to the vendors and can “publish” media to them in the form of a link to the media essence files in an object storage bucket. (Files are made available via a permissioned link to the S3 bucket or delivered via Aspera/Signiant depending on the partner.) Publication for final distribution can also occur in the same way with some distribution partners receiving a link for download of the Channel 4 media. As more media companies move to cloud-enabled services, PFT expects the number of partners receiving permissioned shares (instead of files pushed over point to point links) to increase, since the publish function is immediate, can contain much more nuanced rights and privileges, and comes with asset tracking, logging and automated revocation of permissions.
NEXT STEPS
PFT plans to continue enhancing the features and capabilities of CLEAR. Additional features already planned will:
- Enhance live workflows with a Live-to-VOD solution using automatic cloud scale-out technologies and Artificial Intelligence(AI)/Machine Learning to:
- Bring AI segmentation and compliance into ingest workflow,
- Explore deep metadata creation, automated clipping, and promo creation for marketing and media distribution workflows,
- Use Machine Learning for managing error prediction and monitoring of the system and workflows;
- Enable Docker for containerizing components on Kubernetes for seamless and downtime-free deployments and scale-out;
- Enable self-healing systems and advanced system monitoring and for better resiliency and support;
- Integrate with advanced linear playout systems.
MOVIELABS PERSPECTIVE
Prime Focus Technologies’ approach to automating a multi-vendor, cloud-based supply chain solution for Channel 4 illustrates the power and capabilities of a next generation services facility aligned with 2030 Vision principles. Going well beyond the manual and ‘digital’ workflow operations that are typical of today’s post and distribution services facilities, PFT has begun to demonstrate how 2030 Vision principles truly drive efficiency and automation of previously manual and long lead-time tasks. The ability to meet Principles 1 and 2 where all assets are ingested into the cloud and where applications come to the media, is in fact, a game changer in these workflows, where excessive and time-consuming file movements have been reduced or eliminated. MovieLabs sees the fact that files are in one cloud ecosystem and applications in another as an important step in realizing the time-saving efficiency of the 2030 Vision. The ability to deliver work in progress or finished assets through a link, which demonstrates MovieLabs Principle 3, additionally makes this Prime Focus Technologies case study one in which these key MovieLabs 2030 Vision aspirational principles, are, in fact, being implemented effectively today.
Get the Case Study
Download a free PDF of this case study