Here is what you are going to learn in this lab,
- You would begin by defining branch protection rules on GitHub
- Then you would learn about enforcing code reviews via raising pull requests
- You would then start learning how to write pipeline as a code
- Learn the syntax to write declarative Jenkins pipeline
- Create Jenkinsfiles for a Java and NodeJS applications
- And finally launch it with multi branch pipelines on Jenkins
Reading List:
Lets create branch policies based on trunk based development workflow. With trunk based development,
- Master branch is the trunk/main line of code, which is locked. No direct checkins to master allowed.
- Every bugfix gets its own branch
- Every feature gets its own branch too
- Features branches are typically short lived
- Merges to master require pull request
- Further policies could be added to enforce the code review as well as integrate with jenkins ci to run per branch pipeline
This is how it should look like,
To define the branch policies,
-
Goto settings for your github repository and select branches options
-
From Branch Protection Rules section, click on Add rule button.
-
Add the following set of rules
- Require Pull request reviews before merging
- Dismiss stale pull request approvals when new commits are pushed
- Require status checks to pass before merging
- Include Administrators
- Require Pull request reviews before merging
The following image depict the branch protection rules 
You need to clone the repository to your machine if you are not cloned before. Once you have the cloned repository in your local, made some changes in your README.md file and check and commit those changes once it done by using following commands.
git diff
git commit -am "test"
git push origin master
Whenever you are trying to push some changes, it throws an error master protected branch hook declined, because you can't directly push to master and atleast it should be approved by at least one reviewers.
Now you need to reset your commit by using previous commit id.
replace YOURLASTCOMMITID with the actual commit id following
git log
git reset --hard YOURLASTCOMMITID
Once you done hard reset, it will back to your previous commit.
With this sub section, you will learn how to make changes in master branch by using code reviews and pull requests. You'r task this time is just to update the existing README.md file to the project and incorporate those changes to the trunk i.e. master branch.
- Create a branch
readmeusing command following,
git checkout -b readme
Running this command, git will create and switch to the readme branch.
- Update the
README.mdfile by adding a description of the app.
e.g.
filename: README.md
Example Voting App
=========
This is a sample voting app.
- Save and commit the changes you have made.
git status
git commit -am "added application info"
git push origin readme
This pushes the changes to readme branch of the repo on GitHub.
-
Now go to the github repository and choose branch
readme, there you should seenew pull request. The pull request is to help you to merge those changes from the feature branch to master, to the upstream repo you forked, or even to a completely differnt fork. -
Select
new pull requestand choose your base as master branch, add the description and create pull request. -
Add a reviewer: Creating a pull request does not allow you to merge immediately. It would demand a code review which is been manadated as part of the branch protection rules you have set. Add at least one of your collaborators as a reviewer. If you do not have a collaborator, create another account on GitHub, configure it as a collaborator, and add it as a reviewer.
-
Request a review. If you are the reviewer, go to pull requests, review the changes and approve.
After approval you could merge the pull request to master. You could delete the branch after merge.
Reading List:
Following is an example pipeline code. Examine it to learn how to create a Jenkinsfile and define the jobs as a code.
Some of the important directves are,
- Pipeline
- Agent
- Tools
- Stages | Stage | Steps
- Post
In this sub section you will learn how to create and execute a declarative pipeline.
Begin by creating a sample pipeline job. To create it, go to jenkins page and select new item to create pipeline-01 job, while creating pipeline job use project as pipeline
goto pipeline-01 configuration page, in pipeline step choose hello-world script and save the configuration to run the build. Once you run the build it will successfully build helloworld job.
Now you are going to write declarative pipeline, visit declarative pipeline for your reference.
Use the following code to configure your pipeline job.
pipeline {
agent any
stages{
stage(one){
steps{
echo 'step 1'
}
}
stage(two){
steps{
echo 'step 2'
}
}
stage(three){
steps{
echo 'step 3'
}
}
}
post{
always{
echo 'This pipeline is completed..'
}
}
}
After configure the job, build it view the stage view to know how the pipeline works and how much time it will take to complete the job.
Now add some additional step like sleep time with the same job, refer following code for the configuration. Save the configuration and build the job for the result.
pipeline {
agent any
stages{
stage(one){
steps{
echo 'step 1'
sleep 3
}
}
stage(two){
steps{
echo 'step 2'
sleep 9
}
}
stage(three){
steps{
echo 'step 3'
sleep 5
}
}
}
post{
always{
echo 'This pipeline is completed..'
}
}
}
you could see the time interval between each steps in stage view and select to see a specific stage logs.
With this section, your will define pipeline as a code for a java worker app which uses maven as a build tool.
Create new feature/workerpipe branch, by using following command,
git checkout -b feature/workerpipe
git branch
Now you are going to write Jenkinsfile for worker application using previous sample code the following steps to write Jenkinsfile.
Steps:
Create a Jenkisfile inside worker dir of your code. Before you start writing the code, check your maven version in your jenkins Manage Jenkins --> Global Tools Configurations -> Maven and note down the exact name you are referring to the configuration (e.g. Maven 3.6.1).
file: worker/Jenkinsfile
pipeline {
agent any
tools{
maven 'maven 3.6.1'
}
stages{
stage("build){
steps{
echo 'Compiling worker app'
dir('worker'){
sh 'mvn compile'
}
}
}
stage(two){
steps{
echo 'Running Unit Tets on worker app'
}
}
stage(three){
steps{
echo 'Packaging worker app'
}
}
}
post{
always{
echo 'Building multibranch pipeline for worker is completed..'
}
}
}
Once you created Jenkinsfile, commit the file and push the changes to feature/workerpipe
git status
git add worker/Jenkisfile
git commit -am "added Jenkinsfile for worker with build job"
git push origin feature/workerpipe
With this section, you are going to create and execute a multi branch pipeline which
- scans your repository automatically for Jenkinsfile
- setup the pipeline jobs for the branches which contain the above pipeline script
- start executing the CI jobs
Refer to the following steps to create a multi branch pipeline.
** Steps:**
- Create a new job
worker-pipein your instavote directory, choose project type as multibranch pipeline. - In the job configuration page, under the branch sources choose
GitHub, add your account credentials, and choose the project repository. example image is given following,
- Under build configuration, mention your Jenkinsfile path as
worker/Jenkisfile. - Add periodical checks interval as
5 minutesunder Scan multibranch pipeline triggers and save the configuration,
it will automatically scan your repository. Whenever it detects new branch under repository, it scans and runs the pipeline build automatically.
Once the pipeline run is succesful, add two more jobs in the same Jenkinsfile viz. mvn clean test and mvn package
pipeline {
agent any
tools{
maven 'maven 3.6.1'
}
stages{
stage("build"){
steps{
echo 'Compiling worker app..'
dir('worker'){
sh 'mvn compile'
}
}
}
stage("test"){
steps{
echo 'Running Unit Tets on worker app..'
dir('worker'){
sh 'mvn clean test'
}
}
}
stage("package"){
steps{
echo 'Packaging worker app'
dir('worker'){
sh 'mvn package'
}
}
}
}
post{
always{
echo 'Building multibranch pipeline for worker is completed..'
}
}
}
After making changes in Jenkisfile, commit the file from your feature/workerpipe and push the changes into the same branch.
git status
git add worker/Jenkisfile
git commit -am "added Test and package job for worker pipeline"
git push origin feature/workerpipe
Once you push the changes to feature/workerpipe branch, webhook/scan will automatically trigger the feature/workerpipe branch build in instavote multibranch pipeline.
Now after running package, it will create jar/war file. You need to archive that artifact and for your feature use, you could download that file from jenkins.
pipeline {
agent any
tools{
maven 'maven 3.6.1'
}
stages{
stage(one){
steps{
echo 'Compiling worker app..'
dir('worker'){
sh 'mvn compile'
}
}
}
stage(two){
steps{
echo 'Running Unit Tets on worker app..'
dir('worker'){
sh 'mvn clean test'
}
}
}
stage(three){
steps{
echo 'Packaging worker app'
dir('worker'){
sh 'mvn package -DskipTests'
}
}
}
}
post{
always{
archiveArtifacts artifacts: '**/target/*.jar', fingerprint: true
echo 'Building multibranch pipeline for worker is completed..'
}
}
}
Commit the chanegs to feature/workerpipe and push to the branch.
git status
git add worker/Jenkisfile
git commit -am "archive artifacts, skiptest and package"
git push origin feature/workerpipe
You have setup a multibranch pipeline so that the CI pipelines are run automatically for every branch that you create in future. This is extremely useful to get the feedback from CI system even before you merge the code into the trunk. This should get rid of master branch with a broken build completely.
As with the current configuration, the pipeline you have created will launch for any and every change committed to the repository. However, it makes sense to restrict that only to the subset of changes made for the worker application. Specially a desirable feature with mono repositories which contain multiple sub projects.
You may also want to restrict certain jobs to run only on specific branches e.g. integration and acceptance tests should run only for the master branch, packaging applications to create and distribute artifacts also is relevant only for the master branch.
How to achieve that is by setting up conditional execution. You are going to define following conditionbs for the worker pipline,
- run packaging job only on master
- run the jobs in this pipeline only if code in worker subdir is updated
Read the description of when directive in the Jenkinsfile documentation for reference before you start refactoring the code as follows,
file: worker/Jenkinsfile
pipeline {
agent any
tools{
maven 'maven 3.6.1'
}
stages{
stage(one){
when{
changeset "**/worker/**"
}
steps{
echo 'Compiling worker app..'
dir('worker'){
sh 'mvn compile'
}
}
}
stage(two){
when{
changeset "**/worker/**"
}
steps{
echo 'Running Unit Tets on worker app..'
dir('worker'){
sh 'mvn clean test'
}
}
}
stage(three){
when{
branch 'master'
changeset "**/worker/**"
}
steps{
echo 'Packaging worker app'
dir('worker'){
sh 'mvn package -DskipTests'
archiveArtifacts artifacts: '**/target/*.jar', fingerprint: true
}
}
}
}
post{
always{
echo 'Building multibranch pipeline for worker is completed..'
}
}
}
Once you make changes, commit the changes to feature/workerpipe and push to GitHub.
git status
git add worker/Jenkisfile
git commit -am "run package step only on master, run stages only worker changes "
git push origin feature/workerpipe
After making this changes, you would notice that the build it will run only for first two stages and skip package stage with artifacts.
Warning: Even though Jenkinsfile is part of the worker subpath, its not considered as part of the changeset to trigger the builds. You would need to make changes to other files in order to see the jobs being triggered.
You could make some changes in README.md file in root directory and commit those chanegs, push into feature/workerpipe. Build will be trigger but it will skip all three stages.
You don't need this conditional build always, use git log and reset to go back your previous commit.
git log
git reset --hard yourlastcommitid
git push origin feature/workerpipe -f
Now you have learned how to use conditional pipeline stages.
this is an optional section. Refer to it only if you wish to integrate with slack for notifications
Here you will learn, how to integrate slack with jenkins and this will helpful to send notifications related to your build from jenkins to slack channel.
Prerequsite for this, you need a slack account and you should be the slack channel administrator.
Follow the following steps to complete this setup.
Steps:
-
create slack account and create a channel with the name of
instavote-cd.
-
After creating the channel, go to
administrator -> manage apps. It will open a browser window. searchJenkins CIand select configure. From the configuration pagechooseinstavote-cdand add jenkins integration, it will show step by step process to setup jenkins configuration. -
Go to your jenkins,
manage jenkins -> manage plugins -> available pluginssearchslack notificationplugin and install. -
Go to
manage jenkins -> configure system, at the bottom you could find Global slack notifier settings. copy yourteam sub domainfrom slack configure page which you got in previous step and add in jenkins page. -
Add integration credential id, that is secret text, add a secret text using integration token credential id given in slack jenkins configuration page.
-
Add your channel id as
instavote-cdand save the configuration.
-
From the browser's slack configuration page, click on save the configuration atleast onec, to enable the notification.
-
Now from your Jenkins, choose any job and go to the configuration page. Add
slack notificationas the post build action. Save and build the job. Voila ! Now you could see the notifications right into your slack channel.
You could add slack notifications to any pipeline job with slackSend directives. You would have to set up the intgration as desribed above before you attempt this.
Following is a code snipped, which demonstrates how to achieve this.
file: worker/Jenkinsfile
pipeline {
agent any
tools{
maven 'maven 3.6.1'
}
stages{
stage(build){
when{
changeset "**/worker/**"
}
steps{
echo 'Compiling worker app..'
dir('worker'){
sh 'mvn compile'
}
}
}
stage(test){
when{
changeset "**/worker/**"
}
steps{
echo 'Running Unit Tets on worker app..'
dir('worker'){
sh 'mvn clean test'
}
}
}
stage(package){
when{
branch 'master'
changeset "**/worker/**"
}
steps{
echo 'Packaging worker app'
dir('worker'){
sh 'mvn package -DskipTests'
archiveArtifacts artifacts: '**/target/*.jar', fingerprint: true
}
}
}
}
post{
always{
echo 'Building multibranch pipeline for worker is completed..'
}
failiure{
slackSend (channel: "instavote-cd", message: "Build Failed - ${env.JOB_NAME} ${env.BUILD_NUMBER} (<${env.BUILD_URL}|Open>)")
}
success{
slackSend (channel: "instavote-cd", message: "Build Succeeded - ${env.JOB_NAME} ${env.BUILD_NUMBER} (<${env.BUILD_URL}|Open>)")
}
}
}
Once you make changes in Jenkinsfile, commit and push to feature/workerpipe branch.
git status
git add worker/Jenkisfile
git commit -am "add slack notifications"
git push origin feature/workerpipe
After push the changes to branch, the pipeline will build automatically and send notification to slack. From slack you will get link to visit the build status of your job.
Since this feature is complete, it is time to merge it all into the master and delete this feature branch. You could raise a pull request, get it reviewed, observe the CI feedback right onto the pull request page, before you merge it to the master. Once merged, dont forget to delete the branch from remote repo as well as from the local workspace.
You have been tasked to write declarative pipeline for result application.
- Two stages, build and test
- Stages should run conditinoally only when there is a change to result subdir
- You could refer the workflow following
