Generative AI Part 9 Text summarization using Salesforce xgen-7b-8k model and Gradio

Text summarization isnbsp;an NLP task that creates a concise and informative summary of a longer text. In the previous blog, we demonstrated how to do Text summarization using Open-source tools, we can use those for use cases where the data summarized is Open sourced, but in other cases we need to build a private bot to achieve the same. In this blog we will use the Salesforce Xgen-7b-8K model and the Gradio app to create a UI interface to summarize the text.

We will be covering the following topics

• Creating a Text Summarization bot based on Salesforce xgen-7B opensource model.
• Create a UI Application using Gradio.
• Prerequisites

The EC2 instance in which we will run the python code.

• AWS Management Console
• EC2
• Security Groups
• Walkthrough

Let us create the necessary security groups required.

EC2 security group inbound rules.

Next let us create the ec2 instance and install the necessary packages.

Press “Launch instance”.

The instance is up and running. We will “SSH” into the instance and show the setup required for running this demo.

These are the packages that are required.

Steps to setup a virtual environment. Let us first ssh to the EC2 instance.

Run the command as shown below to set up virtual environment

Run the below commands to activate the virtual environment

Next, we will install the required packages

This package is installed. Let us install all the other packages.

Next, we will create the app.py file and run it to test the model by providing a content to summarize.

The code in app.py file is shown below.

Let us run the program.

Cleaning Up

Shutdown and terminate the EC2 instance in which we have deployed the bot. All these activities are done using AWS Console.

Conclusion

In this Blog, we learned how to create a Text summarization bot using Sales force XGEN-7b model. In the next series we will deploy the same using GGML model and extend with doing the same using sage maker endpoint for text summarization.