Working with Microservices-2: Installing and Preparing the Jenkins server for the microservice application’s CI/CD pipeline.

16 min readJul 23, 2023

We will install the Jenkins server with Terraform and manual installation. Then, we will install almost all the tools that we will need during our CI/CD pipelines in the Jenkins server;

Jenkins,
Jenkins plugins,
Helm,
Docker,
Docker Compose,
Ansible,
Terraform,
Git,
kubectl,
eksctl,
Rancher CLI,
Maven,
Java-11,
AWS CLI Version2,
Python 3,
Boto3,
SecurityGroup-80,8080,22,
Necessary Role and Policies

We will use Jenkins Server in Development, Testing, Staging, and Production environments by using different DevOps tools (Kubernetes and Helm, Docker, Docker Compose, Terraform, Rancher, Nexus Repository, Maven, Ansible, Prometheus and Grafana, GitHub, GitLab, Amazon Route 53, AWS Certificate Manager, AWS EKS, AWS RDS MySql Database, AWS S3 bucket, Selenium Jobs and Jacoco.) hands-on in this article’s series.

We will do them all step by step.

Topics we will cover:

1. What is Jenkins?

2. Installing the Jenkins server from the terraform file.

3. Installing the Jenkins server manually

3. a. Creating a custom Security Group

3. b. Launching Instance and Adding The Scrip To User Data

3. c. Creating a role and assigning it to the Jenkins Server

4. Running Jenkins for the first time

5. Configuring Jenkins

5. a. Changing Jenkins’ Language from a Different Language to English

5. b. Installing the necessary plugin on Jenkins

5. c. Downloading the project (Spring Petclinic Microservices App) into the Jenkins Server

5. d. Installing Helm

5. e. Installing kubectl

5. f. Installing eksctl

5. g. Installing Rancher CLI

6. As a result

7. Next post

8. References

1. What is Jenkins?

Jenkins is one of the most popular tools used worldwide for continuous integration and continuous delivery (CI/CD). Jenkins is a free and open-source automation server. It enables developers to build, integrate, test, and deploy code automatically as soon as it is committed to the source repository (in Figure-1). This enables developers to detect errors and bugs at an early stage and deploy more swiftly. (2)

When we need CI/CD concepts in our projects for example; if we want to update our server after each change in our GitHub repository. Jenkins can handle this process and pull the source code from GitHub to run our project after each change.

2. Installing the Jenkins server from the terraform file.

2. a. Prerequisites

Amazon AWS Account: An Amazon AWS Account is required to create resources for the Jenkins server in the AWS cloud. If you don’t have one, you can register here.
Amazon AWS Access Key: Use this link to follow a tutorial to create an Amazon AWS Access Key if you don’t have one yet.
An Amazon EC2 key pair. If you don’t have one, refer to creating a key pair.
Install Terraform: We will use it as an “Infrastructure as code” to provision the server and cluster in Amazon AWS.

2. b. Downloading the Jenkins Terraform file

Clone the Jenkins Terraform file to your folder in local compute from my GitHub repo, using the command below;

git clone https://github.com/cmakkaya/microservices-msp-jenkins-server-latest.git

This Terraform Template (check Figure-2) creates a Jenkins server on EC2 Instances.

2. c. JenkinsTerraform file’s explanation

Our Terraform Template will create the following;

I. Jenkins Server will run on Amazon Linux 2 EC2 Instance with a custom security group allowing HTTP(80), Custom TCP(8080), and SSH (22) connections from anywhere.

II. With this terraform file; “Git, Docker and Docker Compose, AWS CLI Version 2, Terraform, Java 11, Python, Ansible, and Boto3” were installed on the Jenkins Server by using the “jenkinsdata.sh” script.

III. Terraform sets up these by using “jenkins-server.tf” and “jenkins.auto.tfvars” files below:

It creates AWS IAM policies and assigns them to the AWS IAM role. So, Jenkins Server can use Amazon ECR, Amazon ECS, and Amazon EC2.

It uses ami= “ami-04823729c75214919”, instance_type=“t3.medium” and volume size “16 gb”

provider "aws" {
  region = var.region
  //  access_key = ""
  //  secret_key = ""
  //  If you have entered your credentials in AWS CLI before, you do not need to use these arguments.
}

resource "aws_instance" "tf-jenkins-server" {
  ami           = var.ami
  instance_type = var.instance_type
  key_name      = var.mykey
  vpc_security_group_ids = [aws_security_group.tf-jenkins-sec-gr.id]
  iam_instance_profile = aws_iam_instance_profile.tf-jenkins-server-profile.name
  ebs_block_device {
    device_name = "/dev/xvda"
    volume_type = "gp2"
    volume_size = 16
  }
  tags = {
    Name = var.jenkins-server-tag
    server = "Jenkins"
  }
  user_data = file("jenkinsdata.sh")
}

resource "aws_security_group" "tf-jenkins-sec-gr" {
  name = var.jenkins_server_secgr
  tags = {
    Name = var.jenkins_server_secgr
  }
  ingress {
    from_port   = 80
    protocol    = "tcp"
    to_port     = 80
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 22
    protocol    = "tcp"
    to_port     = 22
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 8080
    protocol    = "tcp"
    to_port     = 8080
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    protocol    = -1
    to_port     = 0
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_iam_role" "tf-jenkins-server-role" {
  name               = var.jenkins-role
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
EOF

  managed_policy_arns = ["arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess", "arn:aws:iam::aws:policy/AWSCloudFormationFullAccess", "arn:aws:iam::aws:policy/AdministratorAccess"]
}

resource "aws_iam_instance_profile" "tf-jenkins-server-profile" {
  name = var.jenkins-profile
  role = aws_iam_role.tf-jenkins-server-role.name
}

output "JenkinsDNS" {
  value = aws_instance.tf-jenkins-server.public_dns
}

output "JenkinsURL" {
  value = "http://${aws_instance.tf-jenkins-server.public_dns}:8080"
}

The jenkins-server.tf file will pull the necessary values from jenkins.auto.tfvars file below.

mykey = "cumhurkey"
ami = "ami-04823729c75214919"
region = "us-east-1"
instance_type = "t3a.medium"     #  if it doesn't work, do it t3a.large.                         
jenkins_server_secgr = "petclinic-jenkins-server-secgr"
jenkins-server-tag = "Jenkins Server of Petclinic"
jenkins-profile = "petclinic-jenkins-server-profile"
jenkins-role = "petclinic-jenkins-server-role"

VI. In output, it will give “JenkinsDNS” and “JenkinsURL” to us when the terraform installation is complete, as shown in Figure 3.

2. d. Modifying and applying “Jenkins Terraform files”

Go into the folder containing the Terraform files.

We must modify appropriate variables in “jenkins.auto.tfvars” file before launching the instance;

mykey, You can use your existing key.pem in AWS, as shown in Figure-4. If you want you can create a new key.pem with the following command for Jenkins Server too.

Log into the AWS console and create “jenkins.pem” key-pair for Jenkins Server on the EC2 menu or you can use the AWS CLI command below to create “jenkins.pem”;

aws ec2 create-key-pair --region us-east-1 --key-name jenkins --query KeyMaterial --output text > ~/.ssh/petclinic-jenkins.pem
chmod 400 ~/.ssh/jenkins.pem

The system will work even if other values are not changed but If you are working in a different region, you can change its values and so on.

ami, “ami-04823729c75214919” check it on AWS EC2 menu, it should be Amazon Linux 2 AMI’s value, as shown in Figure 5.

region , Amazon AWS region, choose the closest instead of the default (us-east-1)

instance_type - EC2 instance size used, minimum is t3a.medium but t3a.large or t3a.xlarge could be used if within budget.

If you haven’t run the “aws configure” command on your computer before(which means there is no “aws configuration” on the local computer), you must enter the values “secret_key” and “access_key”.

aws_access_key, Amazon AWS Access Key

aws_secret_key, Amazon AWS Secret Key

jenkins_server_secgr and others , name of Secruity group of Jenkins server. You can give any name it and others.

After the settings are finished, run the terraform initcommand in the folder where the terraform files are. To initiate the creation of the environment, run terraform apply --auto-approve, as shown in Figure 6.

Then wait for output similar to the following, as shown in Figure 7, and go into your AWS account EC2>Instance, you must see “Jenkins Server of Petclinic”, as shown in Figure 8.

3. Installing the Jenkins server manually

If we want, we can also perform the manual installation by creating them ourselves in the respective AWS menus. (9–10)

3. a. Creating a custom Security Group

Create a custom Security Group in AWS Console with the link. In the left-hand navigation bar, select Security Groups, and then select Create Security Group. Name it as “petclinic-jenkins-server-secgr”. Set up a custom security group allowing HTTP(80), Custom TCP(8080), and SSH (22) connections from anywhere, as shown in Figure-2. Click the link for its detailed explanation.

3. b. Launching Instance and Adding The Scrip To User Data

Open the Amazon EC2 console by selecting EC2 under Compute. From the Amazon EC2 dashboard, select Launch Instance.

In order to create manually a Jenkins server instance, set up the settings below, as shown in Figure-11. Set up;

ami : “ami-04823729c75214919”,

instance type : “t3.medium”

volume size : “16 gb”

key pair : You can use your existing key.pem in AWS, If you want you can create a new key.pem by clicking the “Create a new key.pem” button.

Security group : Choose “petclinic-jenkins-server-secgr” that we created above.

“Git, Docker and Docker Compose, AWS CLI Version 2, Terraform, Java 11, Python, Ansible, and Boto3” are installed on the Jenkins Server by using the “jenkinsdata.sh” script below in user data. (3)

For this, copy the script and paste “user data” on the “Advanced details” pane, as shown in Figure-12. Then, click on the “launch instance” button.

#! /bin/bash
# update os
yum update -y
# set server hostname as jenkins-server
hostnamectl set-hostname jenkins-server
# install git
yum install git -y
# install java 11
dnf install java-11-amazon-corretto -y
# install jenkins
wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key
yum upgrade
amazon-linux-extras install java-openjdk11 -y
dnf install java-11-amazon-corretto -y
yum install jenkins -y
systemctl daemon-reload
systemctl start jenkins
systemctl enable jenkins
# install docker
amazon-linux-extras install docker -y
systemctl start docker
systemctl enable docker
usermod -a -G docker ec2-user
usermod -a -G docker jenkins
# configure docker as cloud agent for jenkins
cp /lib/systemd/system/docker.service /lib/systemd/system/docker.service.bak
sed -i 's/^ExecStart=.*/ExecStart=\/usr\/bin\/dockerd -H tcp:\/\/127.0.0.1:2375 -H unix:\/\/\/var\/run\/docker.sock/g' /lib/systemd/system/docker.service
systemctl daemon-reload
systemctl restart docker
systemctl restart jenkins
# install docker compose
curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" \
-o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
# uninstall aws cli version 1
rm -rf /bin/aws
# install aws cli version 2
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install
# install python 3
yum install python3 -y
# install ansible
pip3 install ansible
# install boto3
pip3 install boto3
# install terraform
yum install -y yum-utils
yum-config-manager --add-repo https://rpm.releases.hashicorp.com/AmazonLinux/hashicorp.repo
yum -y install terraform

Jenkins server instance will be created and we check it EC2 > Instance menu, as shown in Figure 13.

3. c. Creating a role and assigning it to the Jenkins Server

Create a role named “tf-jenkins-server-role” in AWS Console the link, as shown in Figure 14.

Choose these policies in the next step, as shown in Figure 15.

“arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess”, “arn:aws:iam::aws:policy/AWSCloudFormationFullAccess”, “arn:aws:iam::aws:policy/AdministratorAccess”

Assign policies to the “tf-jenkins-server-role” role, as shown in Figure 16.

Assign the role to Jenkins Server in the EC2 Actions menu. So, Jenkins Server uses; Amazon ECR, Amazon ECS, and Amazon EC2, as shown in Figures 17–18.

4. Running Jenkins for the first time

Jenkins is now installed and running on your EC2 instance. In order to configure Jenkins, connect to http://<your_server_public_ip>:8080 from your browser. Copy your_server_public IP, as shown in Figures 19–20.

Important Note: After the Jenkins server is installed, wait for about 5 minutes to connect to it. Otherwise, Jenkins may give an error.

You will be able to access Jenkins through its management interface, as shown in Figure 21.

Use the “ssh” command below to connect to the Jenkins server instance, as shown in Figure 22.

$ ssh -i /path/my-key-pair.pem ec2-user@ec2-198-51-
100-1.compute-1.amazonaws.com

Or connect to the Jenkins server instance by using Vscode to work more easily, as shown in Figure 23. For more information on how it’s done (step-by-step explanation) check my article. If you haven’t used it before, you will love it.

You should definitely try: Great convenience auto-connect to our EC2 instance using VScode

In this article, I will talk about a feature of VScode (Visual Studio Code) that I am sure you will like very much. We…

cmakkaya.medium.com

Use the following command to display this password, as shown in Figure 24.

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Enter the password in the box. Then, click on the “launch instance” button. as shown in Figure 25.

The Jenkins installation script directs you to the Customize Jenkins page. Click Install suggested plugins, as shown in Figure 26.

It installs suggested plugins, as shown in Figure 27.

Once the installation is complete, the Create First Admin User will open. Enter your information, and then select Save and Continue, as shown in Figure 28.

Jenkins connection URL will appear, click on the “Save and Finish” button, as shown in Figure 29.

“Jenkins is ready” window will appear, click on the “Start using Jenkins” button, as shown in Figure 30.

Jenkins setup is complete, and the following window will appear, as shown in Figure 31.

Note: Jenkins’ language may differ depending on your location.

5. Configuring Jenkins

5. a. Changing Jenkins’ Language from Different Language to English

You can change Jenkins’ Language from your Language (mine is Turkish) to English. I recommend it.

For this, We will install the Locale plugin and convert the language to English with it, as shown in Figure 32.

For more information on how it’s done (step-by-step explanation) check my article.

Changing Jenkins’ Language from Different Language to English

We will change Jenkins’ Language from Different Language (Turkish) to English. We will do it all step by step.

cmakkaya.medium.com

5. b. Installing the necessary plugin on Jenkins.

Set up these plugins for CI/CD pipeline that we use on our hands-on in these articles series:

“GitHub Integration, Docker, Docker Pipeline, Jacoco, AnsiColor, Copy Artifact, Deploy to container, Job DSL, SonarQube Scanner, and Amazon EC2" plugins. “Locale” is installed before.

In order to install suggested plugins; open your Jenkins dashboard and navigate to “Manage Jenkins” >> “Manage Plugins” >> “Available” tab on the left-hand side.

Enter the plugins’ name in the Search box and select the plugins (select the checkbox next to plugins) then click on the “Install without restart” button, as shown in Figure 33.

Note: No need to install the other “Git plugin” which is already installed and can be seen under the “Installed” tab.

Figure 33- Installing the plugin on Jenkins.

All plugins that will be installed, as shown in Figure 34.

You can see the uploaded plugins in the “Installed plugins” section, as shown in Figure 35.

Figure 35- Viewing the uploaded plugins on Jenkins.

5. c. Downloading the project (Spring Petclinic Microservices App) into the Jenkins Server

Connect to your Jenkins Server via “ssh” and clone the Spring Petclinic Microservices App from my GitHub repository, using the command below, as shown in Figure 36.

https://github.com/cmakkaya/microservices-with-MySql-db.git

5. d. Installing Helm

Install Helm [version 3+] on Jenkins Server, using the command below, as shown in Figure 37. (5–6)

curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash
helm version

Create a helm chart named “petclinic_chart” under the “microservices-application-with-db/k8s” folder, as shown in Figure 38.

cd k8s
helm create petclinic_chart

Remove all files under the petclinic_chart/templates folder.

rm -r petclinic_chart/templates/*

Add the “k8s/petclinic_chart/values-template.yaml” file as below. Change the DNS name by looking from AWS Route53, as shown in Figures 39–40.

Note: This adjustment will be required to connect to our microservice application as DNS and HTTPS in later parts of the article series. It will automatically get the DNS address from Helm’s values.yaml file, for this reason, we replace the value here with the DNS address of our microservice application. (in Figure 26 in the link below)

https://cmakkaya.medium.com/working-with-microservices-16-preparing-and-running-the-production-jenkins-file-and-pipeline-a545ea7590ef

IMAGE_TAG_CONFIG_SERVER: "${IMAGE_TAG_CONFIG_SERVER}"
IMAGE_TAG_DISCOVERY_SERVER: "${IMAGE_TAG_DISCOVERY_SERVER}"
IMAGE_TAG_CUSTOMERS_SERVICE: "${IMAGE_TAG_CUSTOMERS_SERVICE}"
IMAGE_TAG_VISITS_SERVICE: "${IMAGE_TAG_VISITS_SERVICE}"
IMAGE_TAG_VETS_SERVICE: "${IMAGE_TAG_VETS_SERVICE}"
IMAGE_TAG_API_GATEWAY: "${IMAGE_TAG_API_GATEWAY}"
IMAGE_TAG_ADMIN_SERVER: "${IMAGE_TAG_ADMIN_SERVER}"
IMAGE_TAG_HYSTRIX_DASHBOARD: "${IMAGE_TAG_HYSTRIX_DASHBOARD}"
IMAGE_TAG_GRAFANA_SERVICE: "${IMAGE_TAG_GRAFANA_SERVICE}"
IMAGE_TAG_PROMETHEUS_SERVICE: "${IMAGE_TAG_PROMETHEUS_SERVICE}"
DNS_NAME: "DNS Name of your application"

Note: You must create this “A record”for the DNS name of the microservice app manually yourself in Route53. You can check out my article on this in the link. https://cmakkaya.medium.com/working-with-microservices-12-setting-domain-name-and-tls-certificate-for-production-pipeline-649aef11924d#bae6

5. e. Installing kubectl

The Kubernetes command-line tool,kubectlallows you to run commands against Kubernetes clusters. You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs. (7)

Install the kubectl, using the following command, as shown in Figure 9.

# Download the Amazon EKS vended kubectl binary
curl -o kubectl https://s3.us-west-2.amazonaws.com/amazon-eks/1.23.7/2022-06-29/bin/linux/amd64/kubectl
# Apply execute permissions to the binary.
chmod +x ./kubectl
# Move the kubectl binary to /usr/local/bin.
sudo mv kubectl /usr/local/bin

After you install kubectl, you can verify its version with the following command, as shown in Figure 41.

kubectl version --short --client

5. f. Installing eksctl

eksctl is a simple command line tool for creating and managing Kubernetes clusters on Amazon EKS. eksctl provides the fastest and easiest way to create a new cluster with nodes for Amazon EKS. (8)

eksctlcommands similar to kubectlcommands. If you have installed kubectlwill be enough for you. You can not install eksctl.

Download and extract the latest release of eksctl with the following command, as shown in Figure 42.

curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp

Move the extracted binary to /usr/local/bin.

sudo mv /tmp/eksctl /usr/local/bin

Test that your installation was successful with the following command.

eksctl version

Attention: When we create clusters in the Jenkins pipeline by using kubectl or eksctl commands, check whether they are working properly in “Jenkins user” using the following commands, as shown in Figure 43.

sudo usermod -s /bin/bash jenkins
sudo su - jenkins

5. g. Installing Rancher CLI on Jenkins Server

Rancher is a software product to manage Kubernetes clusters. This includes not only managing existing clusters, but building new clusters as well. The Rancher Command Line Interface (CLI) is a unified tool to manage your Rancher server. With this tool, you can control your environments, hosts, stacks, services, and containers.

Download and extract the latest release of Rancher CLI with the following commands, as shown in Figure 43.

curl -SsL "https://github.com/rancher/cli/releases/download/v2.6.9/rancher-linux-amd64-v2.6.9.tar.gz" -o "rancher-cli.tar.gz"
tar -zxvf rancher-cli.tar.gz
sudo mv ./rancher-v2.6.9/rancher /usr/local/bin/rancher
chmod +x /usr/local/bin/rancher
rancher --version

6. As a result

We installed and prepared the Jenkins server for the microservice application’s CI/CD pipelines. We installed it with Terraform and manual installation. We have installed almost all the tools that we will need during our CI/CD work in the Jenkins server.

We will use Jenkins in Development, Testing, Staging, and Production environments by using different DevOps tools (Kubernetes and Helm, Docker, Docker Compose, Terraform, Rancher, Nexus Repository, Maven, Ansible, Prometheus and Grafana, GitHub, GitLab, Amazon Route 53, AWS Certificate Manager, AWS EKS, AWS RDS MySql Database, AWS S3 bucket, Selenium Jobs and Jacoco.) hands-on in this article’s series.

Share this article with friends in your network and help others to upskill.
I frequently share articles about Cloud and DevOps tools and resources. Follow me on Medium so you don’t miss future articles.

For more info and questions, please contact me on Linkedin.

7. Next post

In the next article, “Working with Microservice-3: Installation and use of the Kompose conversion tool”, as shown in Figure 43.

We will convert our docker-compose.yaml to Kubernetes manifest yaml files using the Kompose tool. So, we easily deploy our microservices app into the Kubernetes cluster. You should definitely try it, it’s great convenience to deploy the app into the Kubernetes cluster. This simple program will save us from writing the 25 Kubernetes files of our Springboot microservice app. Kompose will save us a lot of time. We will do them all step by step.