Using a private GitHub repository with a “GitHub Personal Access Token” in the Jenkins pipeline that runs automatically the development stage via Amazon ECR and Docker container.
In this article, we will see how to use the private GitHub repository with a “GitHub Personal Access Token” in the Jenkins pipeline that runs the development stage via Amazon ECR(Elastic Container Registry) and Docker container automatically.
For this, we will create a private GitHub repository, then we’ll add the JavaScript application to it for testing, then we will create a Dockerfile that will build and run the code (using “yarn” and “node” in the Node JS runtime environment that is used to run JavaScript codes), later we will create a “GitHub Personal Access Token”, and finally we will integrate this token into the Jenkins pipeline and use it in Jenkins pipeline that it will run automatically the development stage using Amazon ECR and Docker container. We will do it all step by step.
Topics we will cover:
1. Creating a private GitHub repository.
2. Adding a Nodejs application to the private GitHub repository for testing.
3. Creating a “GitHub Personal Access Token” on GitHub.
4. Writing a Dockerfile that will build and run the code and description of the Dockerfile.
5. Preparing a Jenkins pipeline and integrating this token into the Jenkins pipeline.
5. a. As a prerequisite, Installing the Jenkins server from the Terraform file.
5. b. Installing necessary plugins in Jenkins.
5. c. Preparing a Jenkinsfile.
5. d. Preparing a Jenkins pipeline.
6. Running Jenkins pipeline and monitoring the output.
7. As a result.
8. Next post.
9. References.
If you like this article, I will be happy if you click on the Medium Following button to encourage me to write more, and not miss future articles.
Your clap, follow, or subscribe, they help my articles to reach the broader audience. Thank you in advance for them.
1. Creating a private GitHub repository
Go to your GitHub Account and click on the “+” icon on the right of the top then New repository (1), as shown in Figure 1.
You should see the following page. Type a name for your repository, and an optional description. Select “Private” and “Add a README file” boxes. Click on “Create repository”, as shown in Figure 2.
2. Adding a Nodejs application to the private GitHub repository for testing
Copy your HTTPS address, as shown in Figure 3.
Run Gitbash in your local host, and copy the repository to your local host via the “git clone” command, as shown in Figure 4.
You should see your GitHub project folder in your local host, as shown in Figure 5.
Download and copy the Nodejs project files from this address into this folder. You should see your GitHub project folder, as shown in Figure 6.
Extract the “to-do-app-nodejs.tar” file into the project directory with the “tar” command below.
tar -xvf to-do-app-nodejs.tarThen, right-click inside the folder and run the GitBash program, as shown in Figure 7.
To send files from local to the GitHub project repository, run the following commands in order in the GitBash window, as shown in Figures 8,9,10.
You should see your GitHub repository project folder, as shown in Figure 11.
The GitHub repository is ready now.
3. Creating a “GitHub Personal Access Token” on GitHub
If you haven’t a GitHub token, You must add the token to the GitHub. For this, go to your GitHub Account Profile, on the right of the top > Settings > Developer Settings > Personal access tokens > Generate new token, as shown in Figure 12,13.
Write a name to “Note” for this token that will create, set an expiration date, and select the scope that you need or all. Finally, click on “Generate token”, as shown in Figure 14.
Copy the token and save it in a safe place for later use, as shown in Figure 15.
4. Writing a Dockerfile that will build and run the code.
We must write a Dockerfile that will build and run the code. For this, go under your GitHub project folder in your local host, for example, mine “cmakkaya-javascript-deploy/” folder, and then create a Dockerfile in the folder. The file must is named “Dockerfile”. (You can use Vim editor, for more information about step-by-step using the Vim editor, you can read my article or this link as well).
cd cmakkaya-javascript-deploy/
vi DockerfilePress “i” to edit, and write the below code into Dockerfile. Then to save and quit, press “:wq”.
FROM node:12-alpine
WORKDIR /app
COPY . .
RUN yarn install --production
EXPOSE 3000
CMD ["node", "/app/src/index.js"]Description of the Dockerfile (2):
FROM: It creates a container by using node:12-alpine image (Node JS is a runtime environment and is used to run JavaScript codes),
WORKDIR and COPY: It copies all files in our working directory (in the Jenkins server) to “/app” folder in the container that we will create.
RUN: It builds the code via “yarn”.
EXPOSE: It publishes on port 3000.
CMD: It runs the artifact (indexs.js) via “node”
Push the Dockerfile you prepared to your GitHub project repository that we created in item 1, as shown in Figure 16.
5. Preparing a Jenkins pipeline and integrating this token into the Jenkins pipeline
5. a. Installing the Jenkins server from the terraform file.
As a prerequisite, the Jenkins server must be installed. If you do not have a ready Jenkins server, you can install it from the terraform file in this link.
This terraform file creates a Jenkins Server using JDK 11 on EC2 Instance. Jenkins Server is enabled with Git, Docker and Docker Compose, AWS CLI Version 2. Jenkins Server will run on Amazon Linux 2 EC2 Instance with a custom security group allowing HTTP(80), Custom TCP(8080, 3000), and SSH (22) connections from anywhere. Also, it sets the AWS IAM role to Jenkins Server for using Amazon ECR and Amazon ECS. It uses ami= “ami-087c17d1fe0178315” and instance_type=“t3.micro”. When the terraform installation is complete you should see the following output, as shown in Figure 16.
5. b. Installing necessary plugins in Jenkins.
Open your Jenkins dashboard and navigate to Manage Jenkins > Manage Plugins > Available tab.
Search and select “GitHub Integration, Pipeline: GitHub” plugins, then click to “Install without restart”. You can see the uploaded plugins in the “Installed plugins” section, as shown in Figure 17.
Note: For more information about step-by-step installing the Jenkins plugin, you can read my article as well.
5. c. Preparing a Jenkinsfile
We need to create a Jenkinsfile for the Jenkins pipeline to work. The following Jenkinsfile consists of 4 stages (3).
Stage 1: Provides the necessary AWS credential, and connects to Amazon ECR then creates a private repository for the image. Note: You can review the article below for a detailed explanation of the aws ecr commands in this stage.
Stage 2: Builds Docker Image.
Stage 3: Sends the created images to the Amazon ECR repository.
Stage 4: Pull images from the Amazon ECR repository and this image creates a container on which our application runs.
Finally, It deletes all local images so that the server does not run out of capacity in a short time.
pipeline {
agent any
environment {
ECR_REGISTRY="592681249063.dkr.ecr.us-east-1.amazonaws.com"
APP_REPO_NAME="cmakkaya/cmakkaya-javascript-deploy"
AWS_REGION="us-east-1"
PATH="/usr/local/bin/:${env.PATH}"
}
stages {
stage('Create ECR Repo') {
steps {
echo "Creating ECR Repo for nodejs app"
sh 'aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin "$ECR_REGISTRY"'
sh '''
aws ecr describe-repositories --region ${AWS_REGION} --repository-name ${APP_REPO_NAME} || \
aws ecr create-repository \
--repository-name ${APP_REPO_NAME} \
--image-scanning-configuration scanOnPush=true \
--image-tag-mutability IMMUTABLE \
--region ${AWS_REGION}
'''
}
}
stage('Build Docker Image') {
steps {
sh 'docker build --force-rm -t "$ECR_REGISTRY/$APP_REPO_NAME:latest" .'
sh 'docker image ls'
}
}
stage('Push Image to ECR Repo') {
steps {
sh 'aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin "$ECR_REGISTRY"'
sh 'docker push "$ECR_REGISTRY/$APP_REPO_NAME:latest"'
}
}
stage('Deploy') {
steps {
sh 'aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin "$ECR_REGISTRY"'
sh 'docker pull "$ECR_REGISTRY/$APP_REPO_NAME:latest"'
sh 'docker rm -f todo | echo "there is no docker container named todo"'
sh 'docker run --name todo -dp 80:3000 "$ECR_REGISTRY/$APP_REPO_NAME:latest"'
}
}
}
post {
always {
echo 'Deleting all local images'
sh 'docker image prune -af'
}
}
}5. d. Preparing a Jenkins pipeline
For preparing a Jenkins pipeline, go to the Jenkins dashboard and click on “New Item” to create a pipeline.
Enter an item name, for example, mine “cmakkaya-javascript-deploy” then select “Pipeline” and click “OK”, as shown in Figure 18,19 below.
Write a description of the pipeline, as shown in Figure 20.
Make the following setting to delete automatically the builds of the pipeline so that the Jenkins server does not reach its capacity in a short time, as shown in Figure 21.
Configure the Jenkins pipeline for integrating the “GitHub Personal Access Token” as described below;
Select “Pipeline script from SCM” in the definition, Select “Git” in the SCM, Paste your GitHub repository address into “Repository URL”, and Correct the branch as “*/main”, as shown in Figure 22.
You will see a failed message because Jenkins can’t connect to your GitHub repository since they don’t have credentials. To fix it, click on “Add” and then “Jenkins” (4).
In the opened “Jenkins Credential Provider: Jenkins” window; Choose exactly what is shown below and paste your “GitHub Personal Access Token” in the “Password”, as shown in Figure 23. Click on the “Add” button.
Choose “GitHub-token/******(GitHub-token for nodejs project) in the Credentials section, as shown in Figure 24.
The red failed warning will disappear, as shown in Figure 25.
Write “Jenkinsfile” to the “Script Path”, as shown in Figure 26.
Note: We didn’t specify any path because Jenkinsfile is in the root directory.
The Jenkins pipeline is ready now.
6. Running Jenkins pipeline and monitoring the output
To launch the Jenkins pipeline click on “Builld now”, as shown in Figure 26. Note: We could create a webhook here so that the pipelines start automatically. In the next article, we will do this with a webhook while running the production pipeline.
Monitor to stage View, as shown in Figure 28. The pipeline successfully completed.
By entering the pipeline and clicking on the console output, we can see from the output that the GitHub Personal Access Token is used, and the image is built and run, as shown in Figure 29.
Also, We can see the same output by clicking the logs in the stage view section, as shown in Figures 30, 31, and 32.
Finally, we can see the running application on the docker container at the Jenkins server public IP address, as shown in Figure 33.
7. As a result
Before the production stage, we ran our code on a docker container. We built our code, then uploaded its image to Amazon ECR. Later, when we were going to run the Jenkins pipeline, we pulled the image from the Amazon ECR and ran it on a docker container.
So, we did our final tests before running the developer’s code on the cluster (production stage). We have created an environment and Jenkins pipeline that can be useful for Development and Staging phase.
Note: You can access the necessary files from the GitHub account in the link.
If you liked the article, I would be happy if you click on the Medium Following button to encourage me to write and not miss future articles.
Your clap, follow or subscribe, they help my articles to reach the broader audience. Thank you in advance for them.
For more info and questions, please contact me on Linkedin or Medium.
8. Next post
In the next post, “Creating a CI/CD Jenkins Pipeline with GitHub Webhook that runs automatically the Production Stage via Amazon ECS-Fargate, and Amazon ECR.”
We talk about CI/CD (Continuous Integration/Continuous Delivery) software development process. Using Amazon ECR(Elastic Container Registry), Amazon ECS(Elastic Container Service), Jenkins, and GitHub to deploy Developer’s code to the production environment, step by step. Also, this time we will create a webhook so that the pipelines start automatically.
Happy Clouding…
Don’t forget to follow my LinkedIn or Medium account to be informed about new articles.
