Artificial Intelligence for Multimedia

The importance of AudioVisual (AV) content in our lives is rapidly increasing with the increase of smartphones usage. We may send and receive audiovisual contents over wireless networks, exchange data on community sites, and receive geographically tagged information using smartphones, tablets, and new devices with more sophisticated cameras and displays.

On the other hand, Artificial Intelligence (AI) has become a game changer in our lives. AI is gaining popularity, more and more, day after day. Actually, it has altered all of the key areas of the business, whether we are talking about e-commerce, logistics, and transportation, or even the healthcare sector. So, where is the media sector from catching up?

In this article of my newsletter at C2M, we will show you how artificial intelligence is revolutionizing the media and entertainment sector, as well as what to expect in the future.

First, what is Artificial Intelligence?

Artificial intelligence is a branch of computer science. It studies and develops computer or machine systems which can think and do activities that would ordinarily need human intelligence. This is where the 'artificial' nomenclature comes from. Besides, it should perform? as well or better than the human computational accuracy, speed, and capacity.

Many aspects of human intellect are possible with AI such as speech recognition, decision-making, and visual perception.?But how?! Similarly to humans who have the ability to 'learn as we go'; or to put it another way, learn from their experience, AI-enabled machines can accomplish the same thing. It is known as Machine Learning (ML).

As mentioned earlier, nearly every business is using AI technologies, but the media and entertainment industry is emerging as a notable adopter.

AI in the media and entertainment industry

The growth of AI in entertainment is nothing new; rather, it has become the most talked-about topic among corporations and marketers. In fact, AI in entertainment has brought numerous significant changes to the media and entertainment sector, from generating beautiful compositions to automating monotonous chores and even delivering Virtual Reality (VR) and Augmented Reality (AR) capabilities in movies.

Where AI/ML is being used today in the streaming workflow?

AI showed recently the capacity of completely changing the media and entertainment industry, affecting everything from content creation to customer experience.

1.Content creation and edition

According to Rainer Kellerhals, Microsoft’s Media and Entertainment Industry lead?for EMEA,?AI can influence all parts of the media value chain, helping content creators be more creative, content editors be more productive, and content consumers find content that matches their interests and current situation.?

Warner Bros, the multinational media and entertainment giant, is just one example of using AI technology to manage films and budgets. Warners is using recently extensive data and predictive analytics to guide decision-making at the greenlight stage. The integrated platform can determine a star's market worth in every country and the anticipated box office and other revenue from a movie. By providing better dollar-figure parameters for packaging, marketing, and distribution decisions, including release dates, it will reduce the time executives spend on low-value repetitive operations. The tool is especially useful in festival setting, where studios frequently engage in bidding wars and make significant investments after only a few hours of consideration (as seen with New Line's 15 million $ purchase of Blinded by the Light from the Sundance Film Festival). The Kitchen, Shaft, and Godzilla: King of the Monsters, are three 2019 Warners flops that may have been saved by the AI-based system. When it comes to general film package analysis or a star's worth, the algorithm can determine in a matter of seconds what it used to take days to examine by a human.?

2.In Compression/Encoding

In history, artificial neural network has been used quite early in the image/video compression domain. From 1980s to 1990s, a number of research was conducted on neural network-based image coding. The exploration of using Deep Learning for image/video coding is worthy of reconsideration, and indeed has been an actively developing research area since 2015. Deep Learning?!

Deep Learning (DL) belongs to machine learning technology, but has the distinction of its computational models, known as Deep artificial Neural Networks (DNN). These multilayered DNNs give DL the capacity for processing data with multiple levels of abstraction and converting data into different kinds of representations.?

At present, research has shown promising results and confirmed the feasibility of Deep Learning based image/video coding. For more detailed state-of-the art study about different works already conducted, you can check my PhD thesis manuscript (link below). As well, you can find in the manuscript, two proposed techniques during this PhD to integrate AI into video compression.

Moreover, you can check our article, in which we discussed about a Real-World Use of AI for Better Video Compression by MediaKind. In this work, AI is used to improve the balance between video compression efficiency and processing power (CPU load).

3.In distribution

Intelligent data streaming is made possible by AI-driven technologies, which allows sending data instantly. Real-time content transmission and personalization are made possible using artificial intelligence.?

In fact, according to Statista, in 2018, 85% of all internet users in the United States watched online video content monthly on any of their devices. 83.3%?of US internet users in 2020 watched digital video content.

AI has proved its potential to be a game-changer for the streaming industry by offering effective ways to encode, distribute, and organize data. For example, we will begin to see how fluid this entire process can become when contracts become smarter and we migrate from what was formerly considered a 'one-button transcode' to an AI-enabled 'no-button' transcode.?Smarter data-driven systems will further eliminate redundancy in our supply chains, and AI will eventually drive the globalisation of our businesses, allowing production and distribution to satisfy a rising, but still localized, demand for our content.

When it comes to distribution, AI also helps consumers to have a more personalized experience by recommending titles and curating content based on their tastes (For example the recommendation system used by Netflix). The AI technologies first take into account the user's preferences and then recommend the most appropriate kind of video contents that they may be interested in. This is in line with the move from a 'one-to-many' to a 'one-to-one' approach.?Although media firms are already using analytics tools to understand their operations and audiences, they are just now beginning to dig into the capability of more advanced technologies like DL algorithms.

"Media companies can employ AI throughout their content supply chains to automate processes, drive decision-making, and personalize the customer experience," says Lorenzo Zanni, IABM's chief research analyst. He adds "Metadata tagging is the most prevalent use of AI so far, via techniques such as image recognition and speech-to-text transcription. The information generated automatically by AI algorithms may subsequently be used to guide content monetization strategies".

The Microsoft’s Media and Entertainment Industry lead?for EMEA explains "For example, Azure Video Indexer uses media AI technology to extract easily metadata from video, such as timecoded transcripts, faces, voices, objects, actions, brands, keywords, and emotions". He adds "Our audience insights function uses the Microsoft Azure Data Platform to collect data about user interactions with online media and create user profiles (including anonymous users) that are then used to power recommendation engines, personalization, ad targeting, and content investments".

4. At content consumer side (Social Media and Filtering fake news)

Artificial intelligence in social media has a wide range of applications and plays a critical role in how social media works.

It is no secret that Facebook employs powerful ML in almost every feature of its platform in order to improve your user experience and profits. Facebook may recognize people in photos and use automated retargeting to show you advertisements for products you have recently looked at. In addition, it uses a program called DeepText. Facebook can better interpret conversations using this technology. It may be used to automatically translate postings across languages.

Similar technology is used by LinkedIn to provide relevant job recommendations.

While for Snapchat and Instagram, computer vision and AI are utilized to overlay real-time filters that move with your face. On Instagram, like for others, AI takes into consideration your interests (likes, comments, ...) and the accounts you follow to select which posts will apear in your Explore tab.

Twitter AI is utilized for fraud detection, propaganda removal, and hateful content removal. Twitter also use AI to suggest tweets to users depending on the kind of tweets they engage with.

The internet is flooded with "fake news", making it more difficult for consumers to distinguish reality from fiction.?To identify 'fake news', DL AI techniques may now be used to source and fact check a story.

For example, Google's Search Algorithm update aims to combat the spread of fake news and hate speech. In addition, the University of Michigan has created an AI technique that correctly detects fake news 76% of the time. Websites are fed into a smart algorithm that scans the sources and predicts the most accurate and reliable versions of news.

The takeaways

This is absolutely not an exhaustive list of AI applications in the media industry. Many other interesting implementations can be cited; for instance, the human percepted image quality assesment (without reference) modeled by AI. Content Aware Encoding (known also as per-title encoding) is a big subject and a potential field to profit from AI advantages. This topic will be most probably treated in one of our next articles. The list of applications is so long: face recognition, object detection, Generative Adversial Networks (GAN), orchestration, anomaly detection, audio translation, automation, ...

Streaming technology is evolving, just as broadcast technology is transitioning to streaming. New potential for expanded functionality within the streaming workflow are emerging as software development shifts from siloed, monolithic architectures to scalable, containerized microservices. Whether it is using AI and ML at the edge or automating processes like ticket production based on surpassing KPI criteria, the streaming technology stack's future appears to have no limits.?

What is the future of AI hiding for us?

Here is one example of what we could be able to see in the upcoming years due to the exponential increase in AI applications throughout the media and entertainment industry! Today, AR and VR have evolved into unique elements for several industry leaders. VR is expected to develop at the quickest rate in the entertainment industry, according to a Global Entertainment and Media Outlook study. This study shows that the US market will use roughly 68 million VR headsets over the next three years, propelling the market's expansion to almost 5 billion US$.

This tech sector will continue to settle the groundwork and progress toward a metaverse future. We will definitely tackle this 'newborn' phenomenon in one of our upcoming posts.

Another important event is when video ad tech will be more seriously considered. The importance of AI and ML in the media and entertainment business is expected to grow significantly by the end of this year, with revenue exceeding 9.5 billion US$. Due to competition challenges from direct-to-consumer services, incumbents (such as pay TV operators and broadcasters) have had to reduce expenses, limit churn, and extract as much value as possible from their existing consumers. Here, AI/ML will play a bigger role in helping operators better target households with promotions, automate more workflows, and protect their content and services. Just wait and see what will happen with the emergence of Free Ad Supported Streaming TV (FAST)! Is there any need to mention again that this concept also interests C2M and will be one of our future hot topics? So stay tuned!

Conclusion

To sum up, we tried to cover in this article some of the benefits of AI in the media and entertainment industry. AI is rapidly evolving and sweeping the media and entertainment industry, working behind the scenes to boost efficiency and personalization to new heights. Clearly,?AI is revolutionizing this sector. In today's competitive environment, media organisations need to accept and deploy AI tools.

Do you have any questions for us? Please feel free to share them in the comments section of this article. We would be thrilled to hear from you!?Also, if you are looking to advance your knowledge in this exciting field and learn more details about something mentioned, do not hesitate to contact us via [email protected] so we can come back to you directly or maybe in a new article ;)