Ci/cd End To End Flow

Understanding Jenkins Pipelines: A Comprehensive Guide

Key Concepts of Jenkins file

Pipeline

A pipeline is a sequence of steps and stages used to automate the continuous delivery process. This involves creating, testing, and deploying an application. In declarative pipeline syntax, a pipeline is a collection of all stages within a Jenkinsfile, where each stage represents a distinct part of the CI/CD process.

pipeline { } Node

In scripted pipeline syntax, a node represents a machine in Jenkins that executes the pipeline. It defines where the pipeline or stage should run.

node { } Stage

A stage in Jenkins is a block that contains a set of processes or steps. Each stage in the pipeline is a specific phase in the CI/CD workflow, such as building, testing, or deploying the application.

Steps

Steps are the individual tasks that are performed within a stage. Multiple steps can exist within a stage, and they are executed sequentially. Steps define the specific actions to be performed at a particular point in the pipeline.

Agent

An agent is a directive in Jenkins that specifies where the entire pipeline or a specific stage will execute. It helps distribute the load by running projects on different machines. The agent can be defined globally for the entire pipeline or separately for each stage. Some common agent parameters include:

• Any: Executes the pipeline on any available agent.

• None: No global agent is defined; each stage must specify its own agent.

• Label: Executes the pipeline or stage on a labeled agent.

• Docker: Uses Docker images as the execution environment.

Jenkins Master-Slave Setup

Benefits

• Reduce System Load: Distributes the workload across multiple machines.

• Separate Pipelines: Different systems can be tied to different code pipelines.

• Failover Mechanism: Provides redundancy and ensures high availability.

Setup Steps

1. Configure the master node.

2. Set up slave nodes.

3. Connect slave nodes to the master.

4. Distribute the load across nodes.

5. Implement failover mechanisms.

Types of Jenkins Pipelines

Declarative Pipeline Syntax

The declarative syntax is a newer feature that uses a predefined structure for defining pipelines. It simplifies the creation of continuous delivery pipelines by providing a straightforward syntax and encapsulating the stages within a pipeline block.

Example: Declarative Pipeline

groovyCopy codepipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
        stage('Test') {
            steps {
                echo 'Testing...'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying...'
            }
        }
    }
}

Scripted Pipeline Syntax

The scripted pipeline syntax is the traditional method of writing Jenkinsfiles using Groovy syntax. It offers greater flexibility and allows for the development of more complex pipelines as code.

Example: Scripted Pipeline

groovyCopy codenode {
    stage('Build') {
        echo 'Building...'
    }
    stage('Test') {
        echo 'Testing...'
    }
    stage('Deploy') {
        echo 'Deploying...'
    }
}

Example Pipelines

Child Pipeline (Dev Branch)

1. Code Commit: Developers commit code to GitHub.

2. Trigger Pipeline: A webhook triggers the child pipeline.

3. Stage 1 - Sonar/Blackduck/Fortify: Execute security and quality scans in parallel.

4. Stage 2 - Merge Request: Create a merge request and send an email to the approver.

Master Pipeline (Post Approval)

1. Trigger Pipeline: A webhook triggers the master pipeline after the merge request is approved.

2. Stage 1 - Sonar/Blackduck/Fortify: Execute security and quality scans in parallel.

3. Stage 2 - Build Code: Build the code using Maven.

4. Stage 3 - Artifact Push: Push the artifact to JFrog.

5. Stage 4 - Pull Artifact: Use Ansible to pull the artifact.

6. Stage 5 - Deployment: Deploy to the QA environment (on-prem/cloud).

7. Stage 6 - Testing: Perform performance and integration testing.

8. Stage 7 - Deployment to Stage: Deploy to the staging environment.

Conclusion

Understanding Jenkins pipelines, both declarative and scripted, is crucial for automating and streamlining the CI/CD process. By leveraging the different components such as stages, steps, and agents, along with a well-structured master-slave setup, teams can effectively manage and distribute their build and deployment processes.