Content Wizard: demo of a trans-vector digital video publication tool

In order to optimise the distribution of video assets online, media organizations need tailor their offerings for specific digital channels and better understand the interests of their audiences at particular points in time, which are often influenced by contemporary new stories and trends on social media. For this purpose, the research project ReTV has developed a Web-based tool termed ’Content Wizard’ which demonstrates an end-to-end, semi-automated workflow for video content creation, adaptation and distribution across digital channels. Digital assets can be selected based on predicted future trending topics, re-purposed according to the different digital channels they will be published upon and scheduled for the optimal future publication date. The result is an innovative video publication workflow that meets the marketing needs of media organisations in this age of transient online media spread across multiple channels.


INTRODUCTION
Digital video content marketing refers to the digital marketing activities related to the promotion of digital video assets of an organisation. Media organisations promote their media assets on alternative digital channels (social media, Websites, apps) with the goal to attract viewers to their own channels and build stronger relationships with their audiences. They are forced to compete with a huge and growing scale of other video content on these channels. Digital video content marketing therefore has to address the challenge to get the attention of an audience already overwhelmed by content choice and availability. In particular, broadcasters and media archives need to stay relevant in this digital landscape in order to provide the public with contextualized and trustworthy content that they hold in their collection. This requires significant efforts in monitoring the wide range of digital platforms to understand what type of content would appeal to the audiences at that particular moment. We present a demo of our Web based tool "Content Wizard", which supports the concept of "trans-vector publishing" -where the same digital content can be semi-automatically published in different forms on different digital channels at an optimal publication time according to its topics. It presents a novel solution to distribute media assets across digital channels in a form that will attract audience attention.

CONTENT WIZARD
Content Wizard is a professional grade tool whose speciality is trans-vector publishing of video content in one seamless, semiautomated workflow. As a first step in the trans-vector publishing workflow, millions of data points -Web pages, social media posts, blog entries etc. -are monitored, annotated and analyzed to identify the topics of discourse among audiences on different digital channels. Predictive analytics is used with the collected data to  identify trending topics in the future. Videos are retrieved from a media collection which are relevant to the topic and re-purposed for the selected digital channels. A social media publication is prepared, where also the accompanying text can be summarized, and scheduled. It is a Web-based tool built on top of the social media management tool Levuro Engage 1 , a modern web application using React 2 for the frontend. Processing of uploaded videos is done using AWS encoding servers and the processed videos are stored in S3 buckets. This distributed approach scales well with an increasing number of videos. The actual posting to the different social media sites is done through their respective APIs. Below we describe the four primary steps of trans-vector publication when working with the Content Wizard.

Step 1. Prediction
The prediction step uses both event knowledge and collected metrics about past topics of online discussion to predict the next trending topics. These predictions guide both the recommendation of which content to publish (Step 2) and the scheduling when it should be published (Step 4). During user tests, we discovered that social media managers focus on posts around anniversaries and political, cultural and sporting events. We created a Knowledge Base of different types of events and deployed an API which supports a query template that provides a personalised view on future events and anniversaries in Content Wizard's Planning Calendar. For example, a user creating a campaign around sports in Europe would see contemporary sports events across European countries and anniversaries related to sportspersons. Alternatively, the user can use the Trending Topics feature which visualises when a topic of 1 https://levuro.com/ 2 https://reactjs.org/ interest is predicted to peak in popularity, or simply see which topics are predicted to be of interest to audiences on a specific future date. Here we use past content metrics such as frequency of mentions in digital channels for time series forecasting as well as the clustering of documents which reference a selected future date [7]. For instance, figure. 1 left shows the predicted popularity of the topic 'James Bond' with the initial peak on 2 April 2021 -the originally planned release date of the next film "No Time To Die".

Step 2. Discovery
Topics of audience interest are matched to topics in digital video assets using an a text-to-video matching module that is based on a dual-encoding attention network [5]. This is a network that has been trained with video-caption pairs, with the objective to make their representations in a shared embedding space as similar as possible. Consequently, the trained network, given either an input video or an input text, can represent it in this shared embedding space. To enable content discovery in the Content Wizard, we temporally segment each of the videos of a given media collection using an efficient fully-automatic method, pre-compute the embeddings for each temporal segment using the aforementioned dual-encoding attention network, and make them available in a database that allows for fast querying. When a user chooses an event or a trending topic in the Content Wizard, the text-to-video search uses the same dual-encoding attention network to turn the input query, i.e. the event's title, into a embedding vector; and, compares this embedding vector to all of the embedding vectors of the video segments that are available in the collection. The videos that are the best matches, following a segment-to-video aggregation of the matching results (max-pooling), are returned to the user. For example, see Fig.1 right for the archival videos matching the topic "James Bond" -all relate to film or cinema despite the search input not mentioning either term.

Step 3. Repurposing
The selected video often needs to be optimised for publication on different digital vectors (e.g. social media channels have different requirements for video length and aspect ratio). The repurposing step takes a video asset and produces a modified copy according to the target publication channel while aiming to retain as much as possible the original content of the video asset. When the user selects a video for editing in the Content Wizard, they are given the option to have it automatically summarised. For this, an adaptation of the Actor-Critic -Generative Adversarial Network architecture (AC-SUM-GAN) of [1] is used. The video analysis process necessary for the video summarization has been run in advance for all videos in the collection so the extracted features are already available to the summarization component, therefore the main summarization process is almost instant. Figure 2 left illustrates how the video editor cuts the video into the segments of the original video that the component proposed. The user can still make manual adjustments if so desired.

Step 4. Publication
Finally, the user can publish to multiple social networks connected to the Levuro publication module, which currently supports Twitter, Instagram, YouTube, Facebook, and LinkedIn. When publishing a video or any other type of content to social media, the Content Wizard allows the user to schedule the post (see Fig. 2 right). The Content Wizard now calls a prediction API which uses our forecasting of future popularity of topics to get the ideal date of publication. Specifically, the prediction model is run against the text of the posting (using keyword extraction) and the date is selected where the aggregated frequency of mentions of those keywords is highest (within the given publication date range, e.g. from 1 to 14 days in the future).

NOVELTY AND CONTRIBUTION TO STATE OF THE ART
In the last years, the market for trans-vector publishing has expanded and at least a dozen tools have come to market which cover at least some parts of this workflow. Typically they allow users to prepare and schedule content (text and image) to post to social networks. Facelift, Tailwind, Buffer, hootsuite, Sprout Social and Falcon.io are examples of well established tools in this area. However, they lack integration with Web and social media monitoring (to inform users of the topics their audiences are currently attentive to) as well as tools for video management (access to collections in archives or on a MAMS; browsing or search via available or extracted metadata; rapid video re-purposing particularly to focus on pre-selected topics and to target social media channels). Content Wizard goes beyond this state of the art in current tools, both by predicting the future topics of interest to the audience (as opposed to showing the topics from past content) and by automatically retrieving and re-purposing videos according to these topics and the target publication channel. Prediction of audience has been linked to social media metrics before in combination with TV program audiences [6,10] but not with digital video viewing. Over the last few years, many AI-based technologies were introduced to help retrieving [4] and curating media content in general [9] and have found their way in supporting/automating tasks in applications that offer optimised publication of audiovisual content across digital media vectors. Example of such applications implemented in the ReTV project include domain specific video analysis [11], crossmedia retrieval [5], online video summarisation tools for publishing in different media vectors [1,3], content adaptation for archival media [2] and TV content adaptation [8].

DEMO AND CONCLUSION
In an online demo, we will demonstrate the Content Wizard's capabilities using concrete examples, highlighting its novel functionalities: trending topic prediction, cross-modal video retrieval, video summarisation, publication with optimised text on an optimal date. Beside some pre-prepared examples, we will also welcome topic ideas from participants. We believe this shows the Content Wizard's unique selling proposition for media organizations which need to adapt their digital video content marketing to the new age of transient online media across multiple channels. Post-project, the Content Wizard functionalities will be available as an extension to the Levuro Engage social media management tool.