Setting up Pega Platform on Amazon EC2 with Multi-Container Docker Deployment

Pre-requisites:

1.??????? Basic linux & docker commands, docker networking, docker volumes.

2.??????? Basic knowledge of Amazon EC2, S3 & IAM roles.

Before I begin, I would like to let you know that I'm not a seasonal technical blog writer, so I have used very simple words to put things into perspective. Because the Kevin from "The Office" show says so..??

At first, I would like to cover about Docker terminologies in very layman terms.

• Dockerfile – Consider it as a list of instructions that one should execute step by step to set up a software or an application.
• DockerImage – Think of it as a pre-made package of that software by executing that Dockerfile instructions step by step.
• Container: This is the run time box for that Docker image. Software comes to live, and server is up and serving its purpose.

Docker Commands:

• build – If we apply build command to a Docker file with a name, it generates a Docker Image for it.
• run – This command runs the docker image. Creates a container out of it. Meaning which, the software comes to live and up.
• ps -a : Lists all the containers in the system.
• ps : Lists down the running containers.
• network: Creates a network for the containers to communicate. If you don’t create one, it will run by default network. We will discuss about this in next article.
• volume: A way to persist the data generated by the container. Will discuss in next article.
• start: starts the container
• stop: stops the container
• kill: forcefully kill the container.
• rm: removes the container
• rmi – removes the docker image

You can find more about these commands online.

https://www.geeksforgeeks.org/docker-tutorial/

Amazon EC2 instance:

It is more of an virtual machine on aws cloud. You specify whatever OS, storage, CPU ram you want, whatever the initial scripts you want to run and create the machine.

You can refer this short video: https://www.youtube.com/watch?v=MmHWh4p2Sqs

Now, lets talk about our Pega Application setup:

In a typical web application scenario, we need to have the below things to make a web app work:

1.??????? Web application with a Web Server running.

2.??????? A data base server that the web application can talk to.

In a traditional scenario, the above two things would be setup by the infrastructure team in their own servers on premises. But, with the ever-growing landscape of cloud and containerization platforms, every company is moving from on premises to Cloud platforms to make their applications much more robust and scalable with relatively faster pace and lesser maintenance cost.

And with the growing popularity of containerization techniques, every company in the market is coming up with their own Docker images to support the cloud-based deployments for their clients. Similarly, Pegasystems, also has come up with their own Docker images to support the clients who wants their Pega cluster to be setup in their own cloud environments.

In this article, I will try to utilize the Pega provided Docker images to setup a Pega platform on Amazon based Linux image.

High Level Architecture:

This is pretty much everything what we're gonna setup here today.

I will split this article into five parts.

Part 1: Setting up an Amazon EC2 instance. Install docker on it.

Part 2: Set up a Postgresql database container.

Part 3: Create a Docker network and connect the DB container.

Part 4: Set up the Tomcat-web server with Pega Platform.

Part 5: Running the web server on same network as our DB server container to start our Pega Server.

NOTE: As far as this article is concerned, I’m not going to customize any of the Docker Images or Dockerfiles that Pega provided in their documentation. I will try to re-use them mostly to avoid any confusion for the Docker beginners.

Part 1: Setting up an Amazon EC2 instance. Install docker on it.

“Make sure you select the x86 architecture machines and not ARM machines.”

Just in case if you want to setup these Pega provided docker images on your local machine, it only supports x86 architecture machines and not fully compliant with Arm machines yet. Refer to the below article as to why CPU architectures matter for Docker images: https://www.knowbe4.com/careers/blogs/engineering/multiple-architectur-builds-in-docker-and-why-they-matter#:~:text=Docker%20supports%20multiple%20platforms%20(%20architectures,it%20in%20the%20host's%20architecture.

So, if you plan to use these Pega provided docker images on Macbook m1 and above, it is not compatible. You rather have to clone the code and build a docker image again on your laptop.

To set up an EC2 instance, utilize an Amazon Linux image with at least a t3.large instance type, ensuring visibility of the running Pega server. Initially, I attempted to work within the constraints of the AWS free tier limits but couldn't even see the server running. The performance of the machine was that slow. Therefore, I opted for the t3.large instance type, which incurs hourly charges. Be careful of the AWS pricing structure before you attempt to create one for POC purposes.

Below is the instance that I have setup and started the instance.

Login using instant connect. I didn’t use any key-pair to login.

After logging in run, the below commands to install docker.

sudo yum update -y

sudo yum install docker -y

sudo systemctl start docker

sudo docker run hello-world

sudo systemctl enable docker

You should see docker installed.

Now, we are good to start our work.

Part 2: Set up a Postgresql database container.

This is the most time taking part.

-????We will use the base image provided by Pegasystems.

• ? pegasystems/postgres-pljava-openjdk

-?????And then create a docker volume.

-?????Run the container and attach the volume to it.

-?????And then finally, we will copy the “pega.dump” folder into the db container.

o?? This dump must be executed to get the basic database setup which includes the Data & Rules schema for our Pega platform.

I will straightaway pull the Pegasystems postgres docker image into my ec2 from docker hub.

Image is installed as shown below:

Create a Docker volume as shown below by running the command

sudo docker volume create pega-data

It will create a docker volume. You can check as shown below.

Now, run the docker image to create a container using below command.

sudo docker run --name pega-pg -p 5432:5432 -d -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=12345 -v pega-data:/var/lib/postgresql/data 6b27a29a7139

?I’m setting environment variables using -e

--name = name I want to give to my DB container i.e pega-pg

And docker volume using -v

You can see the “pega-data” docker volume from ec2 instance is mounted to “/var/lib/postgresql/data” – This is the default mount point for a postgresql docker container. You can find more details on postgresql website.

Now, you will see the docker container is up & running as shown below.

Once the postgresql database is up and running, you can access it from a “pgadmin” on your local laptop by connecting it to the remote server.

In our case, remote server is an “ec2-instance” and its public ipv4 is our server hostname.

Configure the Security Group of your Ec2 instance to access it from your local laptop.

Refer this video for Security Group setup: https://www.youtube.com/watch?v=nA3yN76cNxo

We have finished setting up our Pega Postgresql database container.

Is our job done? Not yet!!

We need to run few more scripts on Postgresql db in order to setup our PegaRULES and PegaDATA schema on our data base. Only then we will be able to see all those pr-tables pc tables etc.

Now, I borrowed this “pega.dump” file from my friend’s personal edition. You can find it in the main unzipped folder of your pega personal edition.

Now, what is this “pega.dump” ? I took the help of chatGPT ??

So, basically “Pegasystems” only provides this pega.dump file in order to set up the basic schema that requires for our Pega engine.

Now, finally – I uploaded this “pega.dump” file into a S3 bucket and then copied into my ec2 instance. The file is around 3.5GB.

You can refer to my last article where I created a S3 bucket:

https://www.dhirubhai.net/posts/kvpvishnu11_pega-amazons3-pega-activity-7178067413450530816-lG75?utm_source=share&utm_medium=member_desktop

Before executing below steps, you need to create a "IAM Role"which contains permissions to full S3 access as shown below. This is to allow S3 objects to be copied from S3 to EC2.

And add the IAM role to EC2 instance. Now execute below commands.

aws s3 cp s3://pega-related-files/pega-dump /home/ec2-user/pega-stuff        

aws s3 cp s3://path-of-source? /destination-path        

"You need not use S3 bucket. You can copy from your local laptop to cloud Ec2 but you need to have setup your ec2 with a key-pair for that"

Follow below article: https://dearsikandarkhan.medium.com/files-copying-between-aws-ec2-and-local-d07ed205eefa

Now, from ec2 instance local storage, we need to copy this pega.dump file into the running container. Because, that’s where our Postgresql pega database is running.

Use the below command:

Syntax:

sudo docker cp <pega.dump> <DB-container-name>:/<mount-path-in-psql-container>

Execution:

sudo docker cp pega.dump pega-pg:/var/lib/postgresql/data

I have copied my pega.dump into the “postgresql” volume mount folder.

You can check the pega.dump file in the below path by entering the terminal of your running container as shown below.

Use "sudo docker exec -it <container-id> /bin/bash" to get linux like terminal.

I have deleted that pega.dump file when I took this screenshot – But you should be able to find it here along with those list of items.

Now, that we have copied the dump file into container, are we done ? Not yet!!

Feeling stuck at the same place ?

Don't worry, we are almost there..

Now, inside the same container terminal, execute the below command

pg_restore -U postgres -d postgres pega.dump?

-U = username

-d = database where you want to dump that schema

I have selected the default postgres schema.

Now, this process will take some time and you wont see anything on screen. If you see any logs on screen, you would have done some mistake before arriving at this point. So, cross check all the steps again.

Once the process is done, control returns to command line, meaning which, you have successfully retrieved the dump. You can now connect to the Postgresql server remotely on your laptop and can see the “rules” & “data” schema as shown below.

You can expand the schema and see the tables.

WE HAVE SUCCESSFULLY SETUP THE POSTGRESQL DB SERVER ON AMAZON EC2 instance.

Part 3: Create a Docker network and connect the DB container.

-??????We need to create a docker network and then link our containers to it.

-?????Docker network helps to establish communication between multi containers.

-?????We are going to set up a Docker image for “Pega web application with the web server” in next step. In order to establish communication between “Db server container” and “Web server container”, we have to setup a Docker network.

-??? ?Use below commands to create docker network.

sudo docker network create pg-net

-????????????? This will create a docker network named “pg-net”

-????????????? You can find it as shown below

Now, link the running postgresql container to the network.

sudo docker network connect pg-net pega-pg

This will link the “pega-pg” - which is our postgresql container to the network.

Part 4: Set up the Tomcat-web server with Pega Platform.

-????????We will use the base image provided by Pegasystems.

• pegasystems/pega-ready

-????????In Personal edition, you can find a “prweb.war” file in tomcat/bin folder.

• Copy it into the Ec2 instance just like how you copied the pega.dump file into ec2 by uploading it to the amazon s3.
• Now, we cannot directly use this docker image like how we have used the above Database container image but using the Dockerfile sample provided by Pegasystems. Use the Pega documentation from the github - README file to create a Custom docker file on top of “pegasystems/pega-ready”.
• ? Build the docker image from it.
• Run it on a docker network to communicate with the database.

Lets go back to Ec2 instance:

I created a Dockerfile.

The contents of the docker file should be as below.

I have obtained this code from the below github link.

https://github.com/pegasystems/docker-pega-web-ready

You can literally use the same code. Just replace the “prweb.war” and “postgresql” driver path in the above Dockerfile content.

Learn more about prweb.war: https://docs-previous.pega.com/sites/default/files/help_v718/definitions/w/warfile.htm

You can download “postgresql” driver from internet. Just search for Postgresql driver. Put that downloaded jar file in the same location as your prweb.war file.

https://jdbc.postgresql.org/download/

Now, what is there in that Dockerfile ???

It is extracting the prweb.war and setting some file permissions. You can see it is using “Pegasystems” provided docker image ?and postgresl driver is being made available for the tomcat application. Go through the above git hub documentation to know about how to control various environment variables for this Dockerfile.

Finally, we are good to build the Docker image for this customized dockerfile.

sudo docker build -t pega-web:1 .

Basically, the syntax is: sudo docker build -t <your-container-name>:<some-tag> /path-to-dockerfile

I ran it in home directory where I wrote my Dockerfile and hence the dot symbol after pega-web:1

If everything is successful, you can see a docker image built as shown below.

pega-web:1

Part 5: Running the “WEB SERVER ON SAME NETWORK AS OUR DB SERVER CONTAINER” to start our Pega Server.

You can use the below command to start the Docker container of Pega web server.

sudo docker run -p 8080:8080 --net=pg-net -e JDBC_URL=jdbc:postgresql://pega-pg:5432/postgres -e JDBC_CLASS=org.postgresql.Driver -e DB_USERNAME=postgres -e DB_PASSWORD=12345 pega-web:1

--net = mention your network name

-e = mention your env variables

Now you can also run this in detach mode, if by adding “-d” in between somewhere of the above command.

But I want to see what’s happening in my server logs, and hence I ran it on foreground.

Since my ec2 instance is of type t3.large – it took me nearly 10 minutes for the server to finally be up and ready to serve.

But 2nd time, it took me only 7 minutes to be up and running. It is very slow.

AT LAST THE EUREKAA MOMENT IS HERE..????????????

Hit the url : https://<ec2-public-ipv4>:8080/prweb

This url is a publicly exposed ipv4 on internet. You can access this link from anywhere. (Don't worry about my account security, I have deleted the resources anyway??)

You can use [email protected] and “install” as password.

If you made it till here, congratulations on setting up your personal Pega cloud instance that can be accessed from anywhere.

You can also see the node joining the cluster. Refer to the below table in DB and its “running”.

That’s how you setup a single node Pega platform on your own cloud leveraging the Pega provided docker images.

Since this is a very long article, I would like to stop here for now and I will be talking about few more interesting things in my future articles. I will talk about the below items in the “Pega” containers context( in no particular order)

1.??????? Why you need docker volumes in the above containers?

2.??????? Why you need docker network for above containers?

3.??????? Bottles necks I faced during the setup on cloud.

4.??????? How to classify nodes on server for Pega multi node system?

5.??????? How to use “docker-compose” to automate these docker commands?

6.??????? How to use AWS Cloud Formation to automate this stack?

7.??????? What is the scope of Kubernetes here?

8.??????? Why is K8s preferred more in real time environments?

9.??????? How can we scale this setup without Kubernetes?

10.? What are the options that we have in AWS Cloud to setup one such environment for Pega platform ?

11.? My analysis on how the Pegasystems could have setup the “Pega cloud mission’s challenges related instances that we use on academy.pega.com”

12.? My understanding of a production environment with a multi node cluster setup.

13. How do we use AWS application load balancers for pega web user nodes ?

14. How to set up Amazon Route 53 for the load balancer for public facing web user nodes?

And many more.

Final Words:

As I was trying to cut down the cost and using the most of the AWS-Free tier resources, I couldn’t show more. Even after pushing my limits to t3.large and a above 30GB ebs volume, it was still slow for me. Since it is the official Pega tomcat docker image, it has so much of java code to run which exhausts my free tier resources. But, if you want to spend few bucks to see all it come live and play around with the application, you can opt for memory optimized instances. But do keep in my mind that AWS charges hourly based and the costs can go higher if you are negligent.

And...

I would like to end this article by quoting one of my favourite Phoebe Buffay's iconic lines from F.R.I.E.N.D.S.

If you want me to do more content using AWS standard rates, pay-as-you-go model to use high performance instances & services, donate me money, so that I can do more content for Pega using AWS services ????--pun intended.

That's all guys!!

If you find my article useful, please share it among your Pega network and community to make it available for the wider audience.

Thanks for your support and keep watching out for more articles.

#PegaDeveloper #PegaLearning #PegaxAWS #PegaAWS #AWSLearning