Build a Generative AI-Powered Content Moderation Solution on E2E Cloud

Introduction

Content moderation is vital for companies to maintain safe, constructive online networks. Without proper moderation, harmful content like hate speech, bullying, misinformation, and illegal material can spread unchecked on social media, user forums, comment sections, and other user-generated spaces.?

The use of AI in content moderation is increasing due to the changing nature of online content and the growth of digital platforms. Overall, content moderation is enabling companies to mitigate risks and cultivate the kind of positive online environments that are essential for their long-term success.

Here, we will design an AI-powered content moderation system using the model Mixtral 8x7B, wrapped in a Gradio interface, and deployed on E2E’s cloud GPU server.

E2E’s Cloud GPU Solutions

E2E Networks provides advanced cloud GPU solutions, catering to various industry verticals with high-performance computing needs. Their GPU cloud offerings feature NVIDIA Tesla V100 and T4 GPUs, designed for tasks like deep machine learning, architectural visualization, video processing, and scientific computing.

Customers can choose from various OS flavors like Ubuntu 16, Ubuntu 18.04, CentOS 7, or Windows 2016 when utilizing E2E's Tesla V100 and T4 GPU servers. E2E Networks' cloud GPUs are known for their cost-effectiveness, high performance, and reliability, making them a popular choice for machine learning, deep learning, and image processing workloads.

Code - Content Moderation System

First, install the following dependencies.

pip install transformers bitsandbytes torch gradio accelerate
        

Importing the required libraries.

from transformers import (
AutoTokenizer,
AutoModelForCausalLM,
BitsAndBytesConfig,
pipeline
)


import transformers
import torch
        

We’ll be using the Mixtral-8x7B-Instruct model from Hugging Face.

# Mixtral Model
model_name='mistralai/Mixtral-8x7B-Instruct-v0.1'
        

Set up the bits and bytes config needed to quantize the model.

# Activate 4-bit precision base model loading
use_4bit = True
# Compute dtype for 4-bit base models
bnb_4bit_compute_dtype = "float16"
# Quantization type (fp4 or nf4)
bnb_4bit_quant_type = "nf4"
# Activate nested quantization for 4-bit base models (double quantization)
use_nested_quant = False


#################################################################
# Set up quantization config
#################################################################
compute_dtype = getattr(torch, bnb_4bit_compute_dtype)


bnb_config = BitsAndBytesConfig(
load_in_4bit=use_4bit,
bnb_4bit_quant_type=bnb_4bit_quant_type,
bnb_4bit_compute_dtype=compute_dtype,
bnb_4bit_use_double_quant=use_nested_quant,
)
        

Load the tokenizer and the model on to the GPU.

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"


model = AutoModelForCausalLM.from_pretrained(
model_name,
quantization_config=bnb_config,
)
        

Create the function for our Gradio interface. This function takes in the content as input and then places it within a prompt. The prompt is designed to instruct the LLM to raise a flag if the content in any way does not meet the moderation policy.

Then this prompt is sent to the LLM and the output is generated using the model.generate function.?

def predict(message, history):
   
    review_template = f"""
        Please review the following content for any violations of our community guidelines. Specifically, only from the content below, please identify any instances of hate speech, graphic violence, nudity or sexual content, harassment or bullying, spam, intellectual property violations, misinformation or fake news, illegal content, self-harm or suicide, or offensive language. If any lines violate our guidelines, please highlight them and provide a brief explanation of the violation. If there are no violations, please indicate that the content is compliant with our guidelines.


        Content:
        {message}


       
        """
   
    input_ids = tokenizer(review_template, return_tensors="pt").input_ids


    # sample up to 30 tokens
    torch.manual_seed(0)
    outputs = model.generate(input_ids, do_sample=True, max_length=1024)
    return tokenizer.batch_decode(outputs[:, input_ids.shape[1]:], skip_special_tokens=True)[0]
        

Then launch the Gradio interface.

import gradio as gr
gr.Interface(fn= predict, inputs = gr.Textbox(placeholder="Paste your content here"), outputs=gr.TextArea()).launch(server_name='0.0.0.0')
        

We set the server_name to 0.0.0.0 so that it is accessible by external IP addresses.

Results

Below are screenshots of an instance where I put in content that was violating the content moderation policy.?

Here’s the output I received:

Now let’s put in something that does not violate content moderation policies in any way.

Output:

Conclusion

In conclusion, content moderation is crucial for companies to maintain constructive online communities free of harmful content. Investing in a mix of human reviewers and AI tools to flag policy-violating posts allows companies to moderate content at scale while building user trust.?

As seen in this blog post, large language models like Mixtral-8x7B can be deployed with interfaces like Gradio to create customizable AI systems for content moderation. While not perfect, such AI systems can augment human moderators and provide an efficient first line of defense.?

Overall, content moderation enables brands to mitigate risks and nurture the positive environments that drive business success.

Code

The code for this blog can be found here: https://github.com/vardhanam/Content_Moderation_System