CI-CD Pipeline with Jenkins

Continuous Integration and Continuous Deployment (CI/CD) pipelines have become an essential part of modern software development. They help ensure that your code is stable, reliable, and deployable to production with minimal effort.

In our previous article, we went through the steps to make an application docker ready. At the end of the article, we were able to build a docker image, push it to repository and deploy it on server.

In this guide, I'll walk you through creating a CI/CD pipeline in Jenkins, so the whole process of build to deployment can be automated.

If you need help with making you application docker ready, you can take reference from the article below

Dockerize Your Angular App with a Java Backend
In today’s fast-paced digital landscape, developers face numerous challenges when building and deploying web applications. With the rise of microservices architecture and distributed systems, it’s becoming increasingly important to ensure that your application is scalable, secure, and efficient. One effective way to achieve these goals is by containerizing your application

Prerequisites

  • List the required tools and software:
    • Jenkins Server
    • Docker ready application

Creating a Jenkins Build file

We will start with creating file named jenkins-build.groovy in the deployment directory. Then add the following content in it

pipeline {
agent any

environment {
  GIT_CREDENTIALS_ID = 'Github-Jenkins-Service-Account'
  BUILD_TAG = 'latest'
}

parameters {
  string(name: 'BRANCH_NAME', defaultValue: 'main', description: 'Branch to build')
}

stages {

stage('Build Name') {
  steps {
    script {
        // Generate a timestamp for the tag
        BUILD_TAG = new Date().format("yyMMdd-HHm")
        currentBuild.displayName = BUILD_TAG
    }
  }
}

stage('Checkout') {
  steps {
    script {
        echo "Checking out branch: ${params.BRANCH_NAME}"
        git branch: "${params.BRANCH_NAME}",
        url: 'https://github.com/navalgandhi1989/angular-java-docker-jenkins.git'
    }
  }
}

stage('Build Docker Images') {
  steps {
    script {
        // Build Docker images
        dir("${WORKSPACE}") {
            echo "----------------- Comencing build process -----------------"
            sh 'docker compose -f deployment/docker-compose.build.yml build'
        }
    }
  }
}

stage('Tag Docker Images') {
  steps {
    script {
        // Tag Docker images
        dir("${WORKSPACE}") {
  
            sh "docker tag sample-app-frontend sample-app-frontend:${BUILD_TAG}"
            sh "docker tag sample-app-backend sample-app-backend:${BUILD_TAG}"
  
            sh "docker image tag sample-app-frontend:${BUILD_TAG} ghcr.io/navalgandhi1989/angular-java-docker-jenkins/sample-app-frontend:${BUILD_TAG}"
            sh "docker image tag sample-app-backend:${BUILD_TAG} ghcr.io/navalgandhi1989/angular-java-docker-jenkins/sample-app-backend:${BUILD_TAG}"
            
            echo "----------------- Tagged Docker images Build:${BUILD_TAG}  -----------------"
        }
  
    }
  }
}

stage('Push Docker Images to registry') {
  steps {
    script {
        // Push Docker images 
        dir("${WORKSPACE}") {
            echo "----------------- Login to registry -----------------"
            docker.withRegistry('https://ghcr.io', GIT_CREDENTIALS_ID) {
                
                def uiImage = docker.image("ghcr.io/navalgandhi1989/angular-java-docker-jenkins/sample-app-frontend:${BUILD_TAG}")
                /* Push the container to the Registry */
                echo "----------------- Pushing UI image to registry -----------------"
                uiImage.push()
            
            }
  
            docker.withRegistry('https://ghcr.io', GIT_CREDENTIALS_ID) {
  
                def backendImage = docker.image("ghcr.io/navalgandhi1989/angular-java-docker-jenkins/sample-app-backend:${BUILD_TAG}")
                /* Push the container to the Registry */
                echo "----------------- Pushing Backend image to registry -----------------"
                backendImage.push()
            
            }
        }
  
        echo "----------------- Finished -----------------"
        echo "----------------- Build Number : ${BUILD_TAG} -----------------"
    }
  }
}
}

post {
	always {
		echo "Pipeline completed"
	}
}
}

jenkins-build.groovy

Here's an explanation of each section in jenkins-build.groovy:

pipeline { ... }

  • Introduction to Pipelines: This is the top-level section that defines the pipeline in a Jenkinsfile. It contains multiple stages, parameters, and environment variables.

agent any

  • Agent Selection: This line specifies that the pipeline can run on any available agent (i.e., a node or executor) in the Jenkins instance. This allows the pipeline to scale horizontally across multiple agents if needed.

environment { ... }

  • Environment Variables: This section defines environment variables that are available throughout the pipeline execution. In this case, we have two variables:
    • GIT_CREDENTIALS_ID: The credentials ID for accessing the Git repository.
    • BUILD_TAG: The generated timestamp for the build (initially set to "latest").

parameters { ... }

  • Pipeline Parameters: This section defines parameters that can be passed to the pipeline when it is executed. In this case, we have one parameter:
    • BRANCH_NAME: The branch name to build (defaults to "main"). This allows users to specify a specific branch for the build.

stages { ... }

  • Pipeline Stages: This section defines multiple stages that are executed sequentially within the pipeline. Each stage represents a distinct step in the build process.
    • Build Name: Generates a unique timestamp for the build.
    • Checkout: Checks out the specified branch from the Git repository.
    • Build Docker Images: Builds the Docker images using docker compose.
    • Tag Docker Images: Tags the built Docker images with the generated timestamp.
    • Push Docker Images to registry: Pushes the tagged Docker images to the container registry.

stage('...') { ... }

  • Individual Stages: Each stage is defined within a separate section, using the stage() function. This allows for more fine-grained control over the build process and makes it easier to debug specific stages.
    • Within each stage, we have multiple steps that are executed sequentially.

steps { ... }

  • Stage Steps: This section defines the individual steps within a stage. Each step is typically executed using a Jenkins plugin (e.g., sh, git, etc.) or a custom script.
    • In this pipeline, we use various plugins and scripts to check out the branch, build Docker images, tag them, and push them to the registry.

dir(...) { ... }

  • Working Directory: This section specifies the working directory for the current stage. In this case, we use dir("${WORKSPACE}") to execute commands within the workspace directory.
    • Within each stage, we execute various commands using plugins or scripts (e.g., sh, git, etc.).

post { ... }

  • Post-Build Actions: This section defines actions that are executed after the final stage of the pipeline. In this case, we have an always block that logs a message indicating pipeline completion.

Creating a Jenkins Deployment file

Create a file named jenkins-deployment.groovy in the deployment directory. Then add the following content in it

pipeline {
	agent any

parameters {
	string(name: 'BACKEND_BUILD_NUMBER', description: 'Build number for Backend')
	string(name: 'UI_BUILD_NUMBER', description: 'Build number for UI')
	choice(name: 'TARGET_ENVIRONMENT', choices: ["UAT", "PRODUCTION"], description: 'Target Environment')
}

environment {
	SSH_USER = ''
	SSH_HOST = ''
	SSH_KEY = ''
	TARGET_DIR = ''
}

stages {
	stage('Print Build Numbers and set Environment') {
		steps {
			script {
				if(params.TARGET_ENVIRONMENT == "UAT") {
					SSH_USER = "root"
					SSH_HOST = "10.1.21.51"
					SSH_KEY = "SSH_KEY_CT-12251"
					TARGET_DIR = "/root/sample-app"
				} else if (params.TARGET_ENVIRONMENT == "PRODUCTION") {
					SSH_USER = "root"
					SSH_HOST = "10.1.22.51"
					SSH_KEY = "SSH_KEY_CT-13251"
					TARGET_DIR = "/root/sample-app"
				}
			}
		echo "Backend Build Number: ${params.BACKEND_BUILD_NUMBER}"
		echo "UI Build Number: ${params.UI_BUILD_NUMBER}"
		echo "Target Directory: ${TARGET_DIR}"
		}
	}

	stage('SSH to Server and Run Commands') {
		steps {
			sshagent(credentials: [SSH_KEY]) {
				sh """
				  [ -d ~/.ssh ] || mkdir ~/.ssh && chmod 0700 ~/.ssh
				  ssh-keyscan -t rsa,dsa ${SSH_HOST} >> ~/.ssh/known_hosts
				  
				  ssh -t -t  ${SSH_USER}@${SSH_HOST} 'bash -s << 'ENDSSH'
						  cd ${TARGET_DIR}
						  UI_BUILD_NUMBER=${params.UI_BUILD_NUMBER} BACKEND_BUILD_NUMBER=${params.BACKEND_BUILD_NUMBER} docker compose up -d
						  docker compose logs
						  exit
					ENDSSH'
				  """
			}
		}
	}
}
}

jenkins-deployment.groovy

This script is similar to the jenkins-build.groovy file, but it has some additional sections and modifications specific to deployment. Here is the explanation of those sections:

parameters { ... }

  • Pipeline Parameters: This section defines parameters that can be passed to the pipeline when it is executed. In this case, we have three parameters:
    • BACKEND_BUILD_NUMBER: The build number for Backend.
    • UI_BUILD_NUMBER: The build number for UI.
    • TARGET_ENVIRONMENT: The target environment (choices: UAT or PRODUCTION).

environment { ... }

  • Environment Variables: This section defines environment variables that are available throughout the pipeline execution. In this case, we have four variables:
    • SSH_USER: The SSH user name.
    • SSH_HOST: The SSH host IP address.
    • SSH_KEY: The SSH key credentials.
    • TARGET_DIR: The target directory on the remote server.

stages { ... }

  • Pipeline Stages: This section defines multiple stages that are executed sequentially within the pipeline. Each stage represents a distinct step in the build process.
    • Print Build Numbers and set Environment: Prints the build numbers and sets the environment variables based on the target environment.
    • SSH to Server and Run Commands: SSHs to the remote server, runs commands, and executes Docker Compose.

Creating a Jenkins Build + Deployment file

With two separate pipelines, we will need to manually trigger each one separately whenever we wanted to build and deploy the sample app. This may be required some time, but most of the time this may lead to unnecessary manual effort, potential errors. So we will create one more pipeline to call the above pipelines internally.

We will create a file named jenkins-build-deployment.groovy in the deployment directory. Then add the following content in it

pipeline {
    agent any

    parameters {
        string(name: 'BRANCH_NAME', defaultValue: 'main', description: 'Branch to build and deploy')
        choice(name: 'TARGET_ENVIRONMENT', choices: ["UAT", "PRODUCTION"], description: 'Target Environment')
    }

    environment {
        BUILD_TAG = 'latest'
        UI_BUILD_NUMBER = 'latest'
        BACKEND_BUILD_NUMBER = 'latest'
    }

    stages {
        stage('Build Docker Images') {
            steps {
                script {
                    def sampleAppBuild = build job: 'sample-app-build',
                          propagate: false,
                          wait: true, 
                          parameters: [
                              string(name: 'BRANCH_NAME', value: params.BRANCH_NAME)
                          ]
                    
                    BUILD_TAG = sampleAppBuild.displayName
                    echo "BUILD_TAG is: ${BUILD_TAG}"

                    if (BUILD_TAG == 'latest') {
                        error "Failed to capture BUILD_TAG from the sample-app-build."
                    }
                    
                    if (sampleAppBuild.result != 'SUCCESS') {
                        error "sample-app-build failed with status: ${sampleAppBuild.result}"
                    }
                }
            }
        }

        stage('Deploy Docker Images') {
            steps {
                script {
                    // Use the captured BUILD_TAG for deployment
                    def UI_BUILD_NUMBER = BUILD_TAG
                    def BACKEND_BUILD_NUMBER = BUILD_TAG
                    def TARGET_ENVIRONMENT = params.TARGET_ENVIRONMENT

                    def sampleAppDeployment = build job: 'sample-app-deployment',
                          propagate: false,
                          wait: true, 
                          parameters: [
                              string(name: 'UI_BUILD_NUMBER', value: UI_BUILD_NUMBER),
                              string(name: 'BACKEND_BUILD_NUMBER', value: BACKEND_BUILD_NUMBER),
                              string(name: 'TARGET_ENVIRONMENT', value: TARGET_ENVIRONMENT)
                          ]
                                        
                    if (sampleAppDeployment.result != 'SUCCESS') {
                        error "sample-app-deployment failed with status: ${sampleAppDeployment.result}"
                    }
                }
            }
        }
    }

    post {
        always {
            echo 'Pipeline execution completed.'
        }
        success {
            echo 'Pipelines executed successfully.'
        }
        failure {
            echo 'Pipeline failed.'
        }
    }
}

jenkins-build-deployment.groovy

Here's an explanation of each section in jenkins-build-deployment.groovy:

parameters { ... }

  • Pipeline Parameters: This section defines parameters that can be passed to the pipeline when it is executed. In this case, we have one parameter:
    • BRANCH_NAME: The branch name to build (defaults to "main"). This allows users to specify a specific branch for the build.
    • TARGET_ENVIRONMENT: The target environment (choices: UAT or PRODUCTION).

environment { ... }

  • Environment Variables: This section defines environment variables that are available throughout the pipeline execution. In this case, we have two variables:
    • BUILD_TAG: The generated timestamp for the build (initially set to "latest").
    • UI_BUILD_NUMBER: The build number for UI (initially set to "latest").
    • BACKEND_BUILD_NUMBER: The build number for Backend (initially set to "latest").

stages { ... }

  • Pipeline Stages: This section defines multiple stages that are executed sequentially within the pipeline. Each stage represents a distinct step in the build process.
    • Build Docker Images: Call Jenkins job named sample-app-build by passing BRANCH_NAME to it. This job will return BUILD_TAG.
    • Deploy Docker Images: Call Jenkins job named sample-app-deployment by passing UI_BUILD_NUMBER (BUILD_TAG) , BACKEND_BUILD_NUMBER (BUILD_TAG) and TARGET_ENVIRONMENT to it.

Configuring Jenkins

To set up the pipeline, we need to configure Jenkins with the necessary credentials and repositories. First, we add a new credential for our Git repository using the Github-Jenkins-Service-Account as the ID.

    • Go to Jenkins > Manage Jenkins > Credentials and click on Add Credentials
    • Select Username with password as the credential type
    • Enter the ID as Github-Jenkins-Service-Account
    • Enter Username as your Git repository username
    • I am using GitHub. So I will enter enter the Personal access tokens (classic) with write:packages permission in Secret. But it can be your password or token from your private repository

Creating Build Pipeline in Jenkins

Click on '+ New Item' in jenkins and

  • Set Name of job, I am using sample-app-build
  • Set item Type and Pipeline

Go to to the pipeline section of job and

  • Set Definition as Pipeline script from SCM
  • Set SCM as Git
  • Set Repository URL as url of the repo where jenkins pipeline files are present. In my case it is the angular and java project repo itself.
  • Set Credentials as your previously created git credentials
  • Set Branch Specifier (blank for 'any') as */main
  • Set Script Path as ./deployment/jenkins-build.groovy

Creating Deployment Pipeline in Jenkins

We can follow the same steps above but change only two thing

  • Set Name of job assample-app-deployment
  • Set Script Path as ./deployment/jenkins-deployment.groovy

Creating Build + Deployment Pipeline in Jenkins

We can follow the same steps above but change only two thing

  • Set Name of job assample-app
  • Set Script Path as ./deployment/jenkins-build-deployment.groovy

With this we are finished with all the configurations on jenkins. Now we can move to utilize these newly created Jobs to build and deploy the apps

Triggering the Job

When you will run the jobs first time, it will not present any Parameters as Jenkins itself need to pull the pipeline file from git. But any build after this will show parameters to choose from.

Result

Conclusion

That's it! We've walked through the process of setting up a Jenkins pipeline from scratch, covering everything from creating a new job to writing a pipeline script.

Remember to keep your pipeline simple, use version control, test thoroughly, and monitor performance. By following these best practices, you can ensure that your pipelines are efficient, reliable, and easy to maintain.

Happy pipeline-building!


GitHub Repository:

You can find all the code from this blog on GitHub:

GitHub - navalgandhi1989/angular-java-docker-jenkins
Contribute to navalgandhi1989/angular-java-docker-jenkins development by creating an account on GitHub.