=========================== What is Build & Deployment =========================== => Take latest code from Git Hub Repo => Build Source code using Maven => Perform Code Review Using Sonar => Upload Project Artifact into Nexus => Deploy code into server. => In single day multipe times code will be committed to git hub repository from Development team so multiple times we have to perform build and deployment process. Note: If we do build and deployment process manually then it is time taking process and error prone. => To overcome above problems, we need to automate Project Build and Deployment process. => To automate project build and deployment process we will use "JENKINS". ======== JENKINS ======== => Open source Software & free of cost => Developed by using Java Language => It is called as CI CD Server CI : Continous Integration CD : Continous Deployment => CI CD is one appraoch to automate project Build & Deployment process. => Using Jenkins we can deploy any type of project (ex: java, python, dot net, react, angular). =============== Jenkins Setup =============== Git Repo : https://github.com/ashokitschool/DevOps-Documents/blob/main/01-Jenkins-Server-Setup.md =========================== what is job in jenkins ? =========================== => JOB means set of steps that we are giving to jenkins to perform the task Step-1 : Take code from git repo Step-2 : Perform maven build Step-3 : Perform code review using sonar Step-4 : Upload artifact into nexus Step-5 : Deploy war file into tomcat server ========================================================= Jenkins Job with with GIT Hub Repo + Maven - Integeration ========================================================= ## Step-1 : Configure Maven as gloabl tool in Jenkins (Jenkins Dashboard -> Manage Jenkins --> Global Tools Configuration -> Add maven) name : Maven-3.9.9 ## Step-2 : Create Jenkins job with "free style project" -> Select New item & Enter Job Name -> Select Free Style project -> Goto "source code mgmt" tab and select git -> Configure project git repo url and branch name (https://github.com/ashokitschool/maven-web-app.git) -> Goto "build step" and select "Invoke Top level maven targets" -> Select Maven version we configured as global tool -> Enter maven goals => clean pacakge -> Click on 'Apply & Save' Note: With above steps we have created JENKINS Job ## Step-3 : Run Jenkins Job with "Build Now" option ## Step-4 : Click on 'Build Number' and then click on 'Console Ouput' to see job execution details. => Jenkins Home Directory in EC2 : /var/lib/jenkins/workspace/ => Go to jenkins workspace and then go to job folder then go to target folder there we see jar/war file created. ============ Assignment ============ => Create Jenkins Job to perform below operation 1) Take source code from git repo Git Repo : https://github.com/ashokitschool/maven-web-app.git 2) Build that code using maven 3) Deploy war file into tomcat server (diff linux vm) Note: We need to install "deploy to container plugin" to deploy war file into tomcat server using jenkins. Go to Jenkins Dashboard -> Manage Jenkins --> Manage Plugins -> Goto Available Tab -> Search For "Deploy To Container" Plugin -> Install without restart. Note: We need to configure "tomcat container" as "post build action" in jenkins job. ================= Jenkins Pipeline ================= => Jenkins pipeline is a way to define CI CD process as a code. => The whole CI CD workflow we will define as a code in pipeline. => when we are dealing with complex CI CD process then pipelines are highly recommended. => Pipeline contains set of stages to perform CI CD Stage-1: Clone Git Repo Stage-2: Maven Build State-3: Code Review Stage-4 : Artifact Upload Stage-5: Build Docker Image Stage-6: Push Image to Registry Stage-7: Deploy App in K8S Stage-8: Send Email Notification => We can create jenkins pipelines in 2 ways 1) Declarative Pipeline 2) Scripted (groovy) Pipeline ============================= Jenkins Declarative Pipeline ============================= pipeline { agent any tools{ maven "maven-3.9" } stages { stage('Git Clone'){ steps{ echo 'cloning git repo' } } stage('Maven Build'){ steps{ echo 'Maven Build' } } stage('Deploy'){ steps{ echo 'Tomcat Deployment' } } } } ======================================== Jenkins - Master & Slave Architecture ======================================== => If we use single machine jenkins, then burden will be increased if we run multiple jobs at a time. => If burden increased then system can crash. => To reduce burden on jenkins server we will use Master & Slave Configuration. => Master & Slave configuration is used to reduce burden on Jenkins Server by distributing tasks/load. ================ Jenkins Master ================ => The machine which contains Jenkins s/w is called as Jenkins Master machine. => It is used to create the jobs => It is used to schedule the jobs => It is responsible to distribute Jobs execution to slave machines. Note: We can run jobs on Jenkins Master machine directley but not recommended. ============== Jenkins Slave ============== => The machine which is connected with 'Jenkins Master' machine is called as 'Jenkins-Slave' machine. => Slave Machine will recieve task from 'Master Machine' for job execution. =================================== Step-1 : Create Jenkins Master vm =================================== 1) Launch Linux VM (t2.medium) 2) Install Java s/w 3) Install Jenkins s/w ================================== Step-2 : Create Jenkins Slave vm ================================== 1) Create EC2 instance (Ubuntu with t2.micro) 2) Connect to EC2 using ssh client 3) Change hostname for readability $ sudo hostname jenkins-slave $ exit 3) Install Java Software $ sudo apt install default-jre 4) Create one directory in /home/ubuntu $ mkdir slavenode ===================================================== Step-3: Configure Slave Node in Jenkins Master Node ===================================================== 1) Go to Jenkins Dashboard 2) Go to Manage Jenkins 3) Select Nodes option 4) Click on 'New Node' -> Enter Node Name -> Select Permanent Agent 5) Enter Remote Root Directory ( /home/ubuntu/slavenode ) 6) Enter Label name as "Slave-1" 7) Select Launch Method as 'Launch Agent Via SSH' 8) Give Host as 'Slave VM DNS URL' 9) Add Credentials ( Select Kind as : SSH Username with private key ) 10) Enter Username as : ubuntu 11) Select Private Key as Enter Directley and add private key Note: Open gitbash and read pem file content and copy content add add it. 12) Select Host Key Strategy as 'Manually Trusted Key Verification Strategy' 13) Click on Apply and Save (We can see configured slave) *********** With above steps Master and Slave Configuration Completed **************** =============== Assignments =============== 1) Jenkins CI CD pipeline with below 3 stages Stage-1 : Git Clone Stage-2 : Maven Build Stage-3 : Tomcat Deployment using SSH Agent 2) How to take jenkins backup @@ Reference Video : https://www.youtube.com/watch?v=5Tb-AOUFuKQ ========================= SSH Agent Configuration ======================== => SSH Agent is used to establish remote ssh connection from one linux vm to another linux vm Ex: jenkins server should connect with tomcat server to copy war file => Install SSH Agent plugin (Manage Jenkins -> Plugins -> Available -> Search for SSH Agent -> Install) => Use pipeline syntax and create ssh-agent for tomcat server vm. -> Snippet Generator -> Sample Step -> Select SSh Agent -> Add -> SSH Username with private key -> Configure Tomcat server username and pem file content -> Click on generate pipeline script to get ssh-agent => Configure SSH Agent details in Deployment stage steps like below sshagent(['tomcat-server-credentials']) { // some block } => With the help of ssh-agent we will copy war file to tomcat server using scp command sh 'scp -o StrictHostKeyChecking=no target/01-maven-web-app.war ec2-user@public-ip:/home/ec2-user/apache-tomcat-9.0.91/webapps' ======================================== Git + Maven + Tomcat + Jenkins Pipeline ======================================== pipeline { agent any tools{ maven "maven-3.9.8" } stages { stage('Git Clone') { steps { git branch: 'develop', url: 'https://github.com/ashokitschool/maven-web-app.git' } } stage('Maven Build'){ steps{ sh 'mvn clean package' } } stage('Deployment'){ steps{ sshagent(['tomcat-server-credentials']) { sh 'scp -o StrictHostKeyChecking=no target/01-maven-web-app.war ec2-user@public-ip:/home/ec2-user/apache-tomcat-9.0.91/webapps' } } } } } =========================================== Integrate Email Notifications in Jenkins =========================================== -> We can configure Email notifications in Jenkins -> With this option we can send email notification to team members after jenkins job execution completed. -> We need to configure SMTP properties to send emails -> Go To Manage Jenkins -> Go To System -> Go to "Extended E-mail Notification" -> We will add company provided SMTP server details to send emails. Note: For practise we can use GMAIL SMTP Properties SMTP Server : smtp.gmail.com SMTP Port : 587 Note: Under Advanced section add your gmail account credential for authentication purpose. ##### Note: Instead of gmail password we need to add gmail app password ###### ##### URL To generate gmail app pwd : https://myaccount.google.com/apppasswords => Select use TLS checkbox => For testing purpose we can use "Email Notification option which is available at the bottom of the page" ============================================== Declarative Pipeline with Email Notification ============================================== pipeline { agent any tools{ maven "Maven-3.9.9" } stages { stage('Clone') { steps { git 'https://github.com/ashokitschool/maven-web-app.git' } } stage('Build') { steps { sh 'mvn clean package' } } } post { failure { emailext( subject: "Build Failed: ${currentBuild.fullDisplayName}", body: "The build ${currentBuild.fullDisplayName} failed. Please check the console output for more details.", to: 'ashokitschool@gmail.com', from: 'ashokit.classes@gmail.com', attachLog: true ) } success { emailext( subject: "Build Successful: ${currentBuild.fullDisplayName}", body: "The build ${currentBuild.fullDisplayName} was successful.", to: 'ashokitschool@gmail.com', from: 'ashokit.classes@gmail.com', attachLog: true ) } } } ====================================== Jenkins Pipeline with Parallel Stages ====================================== => In general jenkins job stages will execute in sequential manner (one after other) => Jenkins support parallel stages execution also. pipeline { agent any stages{ stage('git clone'){ steps{ echo 'git clone....' } } stage('maven build'){ steps{ echo 'maven build...' } } stage('parallel stage'){ parallel{ stage('code-review'){ steps{ echo 'code review....' } } stage('nexus-upload'){ steps{ echo 'nexus upload...' } } } } stage('deploy'){ steps{ echo 'deployment...' } } } } ========================================= Working with Shared Libraries in Jenkins ========================================= => Lets understand "ECommerce Application" usecase in real-time => In E-Commerce application we will have multiple microservices like below.. Note: Every microservice will have its own Jenkins CI CD pipeline. products-listing-service ==> jenkins pipeline cart-service ==> jenkins pipeline checkout-service ==> jenkins pipeline tracking-service ==> jenkins pipeline cancel-service ==> jenkins pipeline admin-service ==> jenkins pipeline reports-service ==> jenkins pipeline => When we are writing multiple pipelines, we can see some common logics in all pipelines. Ex: Maven build + code review + nexus upload... => Instead of writing common logics in all pipelines we can write the logic at one place and we can re-use it at multiple places. => To achieve pipeline logic re-usability we can use 'shared libraries' concept in jenkins. => To create shared libraries we will use 'groovy scripting'. Note: Shared Libraries we can store in our git repo. ====================================== Jenkins Pipeline with shared library ====================================== ## Step-1 : Create shared library and store in git repo @@ Git Repo : https://github.com/ashokitschool/my_shared_libraries.git ## Step-2 : Configure shared library in jenkins => Go to Jenkins Dashboard > Manage Jenkins > Configure System. => Scroll to the "Global Pipeline Libraries" section. => Click Add to create a new library. => Enter the following details: => Name: A unique identifier for the library (e.g., ashokit_lib). Default Version: The branch or tag you want to use by default, such as main or master. => Retrieval Method: Choose Modern SCM. => Source Code Management: Select Git (or your preferred SCM). => Project Repository: Enter the Git repository URL of your shared library. => Click Save to apply the changes ## Step-3 : Create Pipeline and use shared library in pipeline like below ------------------------------------------------- @Library('ashokit_lib') _ pipeline { agent any tools{ maven "maven-3.9.9" } stages { stage('Hello') { steps { welcome() } } stage('git clone'){ steps{ gitClone(); } } stage('maven build'){ steps{ mavenBuild() } } } } ------------------------------------------------- ====================== What is Jenkinsfile ? ====================== => It is used to store pipeline code. => We will maintain Jenkinsfile in project git repo only. ====================== Multi Branch Pipeline ====================== => In one git repo we can have multiple branches a) main b) develop c) feature d) release => Creating seperate jenkins pipeline for every branch is difficult. => We can use "Multi Branch Pipeline" to build the code available in multiple branches at a time using single pipeline. => When we create "Multi Branch Pipeline" it will scan all the branches in given git repo and it will execute pipeline for all branches. Note: When we run multi branch pipeline for second time it will verify code changes happend in which branch and it will execute pipeline for only those branches. ================== User Management ================== => In our project multiple team members will be available a) developers b) testers c) devops engineers => For every team member we need to provide jenkins login access. => Ops team is responsible to create user accounts in jenkins for team members. Ops Team : Should have all priviliges in Jenkins Ex : Create / edit / update / delete / run Dev Team & Testing Team : Only Job Execution Priviliges Ex : Run ================================================ How to create users and manage user permissions ================================================ -> Go to Jenkins Dashboard -> Manage Jenkins -> Manage Users -> Create Users -> Go to Configure Global Security -> Manage Roles & Assign Roles Note: By default admin role will be available and we can create custom role based on requirement -> In Role we can configure what that Role assigned user can do in jenkins -> In Assign Roles we can add users to particular role ===================================== Working with User Roles in Jenkins ===================================== ## Step-1 : Install Required Plugins => Install "Role-based Authorization Strategy" Plugin => This plugin allows you to define roles and assign them to users or groups. ## Step-2 : Configure Security => Go to "Manage Jenkins" > "Configure Security." => Select Authorization as "Role-Based Strategy" => Click "Save" to apply the changes ## Step-3 : Create User Roles => Go to "Manage Jenkins" > "Manage and Assign Roles." => Click "Manage Roles" and define new roles based on your requirements (e.g., admin, developer, tester). => Click "Add" to create a new role, and specify the permissions for that role. ## Step-4 : Assign Users to Roles => After creating roles, go to "Manage Jenkins" > "Manage Users & Roles." => Select a user and click "Assign Roles" to add them to one or more roles. ## Step-5 : Test the user login functionality