Private Repositories to store the artifacts (Nexus)

What is an Artifacts

• Artifacts are the output of any software build process, for example .exe file is an artifacts
• When we create a docker image using docker build command, it creates a docker image as an output and this is an artifacts
• Basically an Artifacts can be anything, including Docker images, text files, or Helm charts.
• There are other examples as well, for JAVA based software the artifacts are WAR files, reports, log files, zip, tar etc
• So, to store all these artifacts we need a repository and one of those repositories is Nexus
• Nexus is one of the products of a company called sonatype

• Here is a list of programing language and its tool needed to build

Why do we need a repository, we can directly push the artifacts to our deployment server?

• Ok, so, we need to understand, deployment is a different process and that comes under CD
• & Creating the artifacts is coming under CI
• During CI, the pipeline creates the artifacts and stores in some remote repo like Nexus
• & during the CD process, the artifacts gets pulled from repo and get applied into the Deployment
• Everytime we push the artifacts into Nexus, it gets versioned and its easier to rollback to older version, rather rebuilding a new artifacts using old source commitments.
• If we use any public repositories like Dockerhub, npm, apt-get, yum we dont have any control on the access, but the Private repo like Nexus has granular control on user level and on policy level as well
• So, we have to configure the Nexus to be a single point of truth for all the repositories an organisation needs to protect their IP.
• To have controll on the Private repository like Nexus, we need something called Repository Manager which ensures

- Configure users, provide RBAC to controll the access control

- It helps to store or retrieve any build artifacts

- It helps to create different types of Repositories (`3 types` - Proxy Repo, Hosted Repi, Group Repo )

- Repository Manager helps proxy any remote repositories & caches its content locally for further use by the internal orgs

- Maintains a repo for its own internal artifacts

What are the different types of Repositories Nexus has?

• Nexus Repository Manager has three types of repositories:

- Proxy repositories: Acts as a cache for public repositories like Dockerhub for Docker images.

- The public repo url https://hub.docker.com/ should be whitelisted in the proxy server

- Hosted repositories: Store build artifacts on Nexus, accessible only to the user with SSO, or Ldap or users from orgs. Not from outside world.

- Group repositories: Combines multiple hosted and proxy repositories into a single logical group,providing a unified view for clients.

Install Nexus on Ubuntu server

• Install Java first (Ensure the right jdk version)

sudo apt install openjdk-17-jre-headless        

• create a nexus user

sudo adduser --system --no-create-home --disabled-login --gecos "" nexus        

• Download the installation files

cd /opt

sudo wget https://download.sonatype.com/nexus/3/latest-unix.tar.gz

sudo tar -zxvf latest-unix.tar.gz        

• Change the user ownership of nexus

cd /opt

sudo mv nexus-3.* nexus

sudo mkdir -p /opt/sonatype-work

sudo chown -R nexus:nexus /opt/sonatype-work

sudo chown -R nexus:nexus /opt/nexus /opt/sonatype-work        

• Add the nexus user to run the nexus

sudo vim  /opt/nexus/bin/nexus.rc

Add the following line:

run_as_user="nexus"         

• Start the Nexus

sudo -u nexus /opt/nexus/bin/nexus start        

• Verify if the nexus is running

ps aux | grep nexus        

Configuration of the Nexus Server

From the browser type this

Nexus_ip_address:8081 (Or DNS name, if the DNS entries are done with the org DNS)        

User name : admin | Password - Need to ssh to the nexus server and find the password based on the location that the UI instructs.

• setup a new password

What we can do with Nexus Repository and its facts?

• Creation: Create hosted, proxy, or group repositories.
• Configuration: Set policies for retention, cleanup, and indexing.
• Artifact Formats:

- Support: Nexus supports various artifact formats, including Maven, npm, Docker, and more.

- Configuration: Configure specific settings for each format.

• Security:

- Authentication: Configure authentication mechanisms like LDAP, Active Directory, or internal users.

- Authorization: Define roles and permissions for different users.

- Encryption: Protect sensitive data like passwords and credentials.

- There are two types of user we can create on Nexus

- 1. Anonymous user - They get only access to the repo contents (GET - Read only ), but they are restricted on other actions like create, delete, upload etc

- 2. Regular user - Each user can have different Roles with different privileges

- There are some default privilege available for Nexus see the lists here https://help.sonatype.com/en/privileges.html

- generally Jenkins integrations are pretty common, where we want the Jenkins to upload the artifacts into Nexus, so we have to create a user Jenkins and assign a role with privilege as create & read - but not delete

Is there any best practices to manage and maintain a private repository?

- Regular Cleanup: Regularly clean up old or unused artifacts to optimize storage.

- Security Audits: Conduct regular security audits to identify and address vulnerabilities.

- Performance Optimization: Monitor performance and optimize settings as needed.

- Version Control: Use version control for Nexus configuration to track changes and facilitate rollback.

- Backup and Recovery: Implement a robust backup and recovery plan to protect your repository data.

- Promotion of repository

- When we build an artifacts, that build does not get used in the production deployment

- First it stores itself into staging repository

- The tests are performed on the staging repo and if it passes all the tests, vulnerability tests,

- The artifacts gets promoted into production repository to be used by prod deployment

Let`s see an example of creating a private Repo for Docker images.

Create a Private Nexus Repo for Docker images

- Go to Nexus, click Create Repo

- Select Docker Hosted

- Update the Realm as active Docker Bearer Token Realm

- This is necessary for authentication when using Docker with Nexus.

To test the repository, we will mimic the stages of Jenkins

• 1. Install Docker to a Ubuntu server
• 2. Install git on the server
• 3. Update the Docker Deamon to connect with the Nexus through insecure path

- This is often necessary when using HTTP instead of HTTPS.

- For production environments, it's recommended to secure the Nexus instance with HTTPS to avoid using insecure registries.

sudo vi /etc/docker/daemon.json

    {

    "insecure-registries" : ["https://3.16.26.143:8091"] 

    // "insecure-registries" : ["Nexus_URL:Port_mentioned_for_Docker_Repo"] 

    }        

- Restart the Docker Daemon

systemctl restart docker        

- verify the connectivity of client(Jenkins/Docker) & the Nexus

sudo docker login -u admin 3.16.26.143:8091         

- Enter the password set for the Nexus

Note : Make sure to log out of Docker if the login test with the credentials are done to avoid potential security issues. sudo docker logout 3.16.26.143:8091

• 4. Clone a github repo (similar to Checkout stage for Jenkins during automation)

    sudo git clone https://github.com/partho-dev/sample-code.git        

• 5. Build the image (build stage as in Jenkins)

    sudo docker build -t 3.16.26.143:8091/express .

    sudo docker build -t 3.16.26.143:8091/express:1.0 . 

# For Prod, its good to use version in auto increment        

- Verify if the image creation successfull

sudo docker images         

• 6. Push the image to Nexus Repo (Push the image to Repo)

    sudo docker push 3.16.26.143:8091/express        

• 7. Verify if the Image gets uploaded into the Nexus Docker Repo

The same above manual process can be automated by integrating Jenkins, and the declaritive pipeline can be somewhat like the below. I have added two more stages in that.

pipeline {
    
    agent any #Good to add some agent to execute the jobs
    
    environment {
        imageName = "Give_Image_Name"
        registryCredentials = "admin"
        registry = "3.16.26.143:8091"
        dockerImage = ''
    }
    
    stages {
        stage('checkout') {
            steps {
                git url: 'https://github.com/partho-dev/sample-code.git', branch: 'main'                  }
        }
    
    // Docker images
    stage('Docker image') {
      steps{
        script {
          dockerImage = docker.build(imageName)
        }
      }
    }

    // Push the image from Jenkins to Nexus
    stage('Push to Nexus') {
     steps{  
         script {
            docker.withRegistry('https://' + registry, registryCredentials) {
            dockerImage.push('latest')
            //or 
            //dockerImage.push("${env.BUILD_NUMBER}")
          }
        }
      }
    }
    
    // Run the container - Stop the container if its running
    stage('stop previous containers') {
         steps {
            sh 'docker ps -f name=container_name -q | xargs --no-run-if-empty docker container stop'
            sh 'docker container ls -a -f name=container_name -q | xargs -r docker container rm'
         }
       }
      
    stage('Docker Run') {
       steps{
         script {
                sh 'docker run -d -p 3000:3000 --rm --name container_name ' + registry + '/' + imageName + ':latest'
            }
         }
      }    
    }
}