Content Moderation in the Realm of Machine Learning

Originally posted on?Dataloop AI Blog ?by?Jodi Lifschitz

• Dataloop AI Blog

Online content generation is now synonymous with social media and this presents a challenge: how do you moderate the content curation and ensure it stays safe??In 2020, over 3.6 billion people ?were using social media worldwide, a number that is projected to increase to almost 4.41 billion by 2025. Within the social media platforms worldwide, Facebook was the first social network to surpass one billion registered accounts and currently boasts more than 2.89 billion monthly active users. What about video popularity online? As of February 2020, more than 500 hours of video were uploaded to YouTube every minute. This correlates to approximately 30,000 hours of newly uploaded content per hour. The?amount of video content ?hours on YouTube has drastically grown 40% between 2014 and 2020.??

?

Source

In fact, online video is one of the most popular digital activities worldwide, with?27 % of internet users watching more than 10 hours ?of online videos on a weekly basis in 2020. In 2021, YouTube was one of the leading media and entertainment brands, with a brand value of more than?$47 billion U.S. dollars . From the number of people to the amount of content online, we can clearly see that the above data shows an increase in social media usage and users. With this content boom, it is essential for brands and businesses to ensure that they can create a safe space online for their readers. This is where content moderation steps in to ensure that users remain protected from harmful content such as hate speech, violence, abuse, and nudity.

What Is Content Moderation?

Content moderation is a means of screening as well as monitoring user-generated content online. The purpose is to ensure a safe environment not only for the brand but their users as well. Platforms have a responsibility to monitor this content to ensure that it is appropriate and that pre-determined guidelines are adhered to. It is also important to ensure that the online behavior is appropriate in relation to the platform and the audience at hand. The responsibility of weeding out harmful content (such as nudity and profanities) is huge. This is where machine learning becomes a necessity to keep sites clean. Machine learning can categorize the content and pre-process images as well as screening out any inappropriate content.

Defining Content Moderation’s Key Challenges?

Brands, lawmakers, and social media execs realize that moderating content is a huge undertaking due to the sheer volume and reach of distributed data and cannot be accomplished easily and solely by the human eye. And the problem is that these social platforms continue to grow to billions of users producing constant content. The question becomes- how do moderators keep up with this growing scale??

Due to this vast spread of content, most platform moderators have to resort to reactive content moderation, thereby moderating content that has already been posted. In addition, monitoring this vast content requires a huge human workforce. In 2019,?Facebook had 15,000 full-time worldwide ?content moderators in order to flag malicious content. The problem is that even if the content is posted on the platforms for a short amount of time, the malicious content still impacts the users on the platform. Another consideration is the toll it takes on these moderators to continuously be filtering through constant disturbing and sometimes traumatizing content, which commonly leads to moderator burnout.?

Another challenge surrounding content moderation is defining appropriate policies. Defining when and how to delete or label objectionable content without crossing the line of stifling users’ free speech to engage in the dialogue of their choice is tricky. The pressure is definitely high for social platforms to deal with offensive content swiftly, but finding the correct balance has been proven to be quite tricky.?

The Significant Role AI Plays in Content Moderation

How do you monitor vast amounts of content? Well, artificial intelligence can play a crucial role in accurately moderating user-generated content. This is accomplished with the help of machine learning algorithms that learn from the existing data, which in turn allows content moderation teams to review and make further decisions in this process. AI infuses automation into the moderation process resulting in faster delivery and error-free results. This process is generally handled in two phases: pre-moderation and post-moderation.?

In the pre-moderation phases, AI can help flag content that needs attention from human moderators. This can be done by keywords, image detection, or object detection methods.?In the post-moderation phase, automation can help flag inappropriate content according to a set of guidelines allowing moderators to check the flagged content and make decisions accordingly.?

Overall,?AI assists human annotators in being more productive?and can help define which content needs to be re-assessed by human moderators, saving them loads of time and frustration.?

Classifying Content Moderation Types?

Image and Video

Object detection is a visual analysis that can identify images. This is essential in order to be able to identify targeted objects that may be inappropriate in images or in videos that are not up to your platform’s standards. Techniques involve the use of different algorithms that are able to detect harmful image content and pinpoint it on the image or video. When it comes to video moderation, it is required that the entirety of the video is analyzed from start to finish or scene-by-scene. Computer vision techniques review each and every shot to ensure the content is appropriate.?

When it comes to images and videos a major challenge is the large ontology sizes. Defining the guidelines to detect inappropriate content is not only time-consuming, but it is also not feasible to search among all the labels of all the images on hand.

Text?

Understanding text is not just about understanding the written word, it also involves understanding the intent of those words. Text classification assigns categories to analyze the context or the sentiment of the text according to labels such as positive, negative, or neutral, usually referring to the tone of the text; this is referred to as sentiment analysis.?

How are AI companies assisting in content moderation?

Scene Text Recognition (OCR)?is a task that entails locating and recognizing the textual content, embedded in the images and the frames of the videos. This can surely be a cumbersome task, especially when you have professional images that contain a lot of text such as conference, lecture slides, quotes, etc. OCR allows you to identify offensive text, objects, and body parts within all types of unstructured data and moderate it accordingly.

Natural language processing (NLP)?is used in order to give a synopsis of text as well as extract the emotion connected to that specific text. This task is accomplished by locating and recognizing the textual content embedded in the images (this is also relevant to videos). NLP is used in order for computers to get a grasp of the human language. This is accomplished with techniques such as keyword filtering in order to flag and remove offensive language.

Visual/Video Moderation: Bounding Boxes/Polygons/Polyline/Ellipses/3D Cuboids – having these types of tools gives you flexibility in order to detect, locate, and define inappropriate objects in images and videos. Once detected, you can easily track an object across multiple frames and image sequences using unique identifiers.

Automation?can provide you with a variety of AI-Assisted tools and automation in order to speed up detection processes. You can integrate your own ML models to trigger auto-annotation of data. This allows a platform such as?Dataloop ?to utilize active learning to progressively increase AI accuracy, which will eventually lead to a point where human intervention is required to handle use cases. Another automation capability is automating functions like cutting video files into individual frames, selecting only high variance items for manual annotations, enhancing the image and video quality, uploading sampled data to train/test the set.?Overall, AI automation helps support human moderators by speeding up the review process.

Effective Content Moderation with Dataloop

When it comes to content moderation tools, how does Dataloop outshine the competition?

Media and content applications require high volumes of targeted data in near real-time, meaning models must perform well at scale and in diverse environments. At Dataloop, we try to focus on incorporating human knowledge and machine learning. We want real-time humans to validate content to ensure that no harmful content goes out. Dataloop accelerates machine learning projects, by adding human validation in a continuous loop, improving the likelihood of success when shifting the model out of the lab and seamlessly transferring it to the real world.

How does Dataloop reduce the costs and improve the process?

• Dataloop provides a semi-automated model assessment.?
• We can help you create your first model (even pre-trained data). This will reduce the amount of data that needs to be reviewed by a human being.?
• Built-in automation tools to ensure quick work which in turn saves you money?

Key Points to Consider When Investing In?a Platform

1. Better instructions/guidelines:?to detect malicious content. This helps to define the issues and enable them to communicate what needs to be flagged and annotated.
2. More automation:?AI-assisted tools and automation speed up the detection process and improve your model-generated content categorizations. Pre-annotating data prior to human labeling turns the manual annotation process into a simple auditing task. Labeling teams can save around 60%-90% of time spent on every batch.
3. Robust search tool:?To help search among large topic tags at different levels which makes the process more efficient and effective. This enables you to find what your data doesn’t have or what edge cases you may have missed. When building an AI model, you’re building a communication system between human knowledge and a machine. This machine essentially knows nothing and the only way to teach it is by feeding it with examples. You’ll also need to figure out if you’ve fed the model all examples and if possible, you have some cases that are under-represented in your data.?
4. ML and human-in-the-loop:?reduce the moderator’s exhaustion and help in detecting more volumes of “bad” content. Human-in-the-loop accelerates machine learning projects, by adding human validation in a continuous loop, improving the likelihood of success when shifting the model out of the lab and seamlessly transferring it to the real world.

Summed Up

Keeping up with user-generated content is going to continue to be a struggle. The advantage of moderating your content in near real-time allows users to keep up with the expectation of having content available upon posting. At the same time, viewers aren’t going to be exposed to as much harmful or inappropriate content.?The task becomes even harder when you’re trying to scale your content and trying to ensure your moderators don’t burn out. But you can easily manage it with Dataloop’s intuitive platform that offers a wide variety of AI-assisted tools and automation that makes the detection processes faster and improves your model-generated content categorizations.?

Social media platforms will continue to grow and naturally so will the content generated by their users. It is imperative that these platforms successfully manage and moderate the content created on them. Find out how Dataloop assisted LinkedIn, a network with over 740 million users across 200 countries in dozens of languages moderate their content.

Read their story.