CI/CD Pipeline

The acronym is Continuous Integration/Continuous Deployment(Delivery). Let's consider CD as Continuous Deployment. 

It is a Site Reliability Engineering approach. 

It enables the addition of new code without a human effort. This code comes from hundreds of contributors from all over the world or in other words, hundreds of repositories into a single project. It would be impossible to do any feature without CI/CD system. 

Somebody in the ecosystem sees that something is not working, they submit a Pull Requests proposing the change.

Some terms to understand the Git/SVN (Subversion) before we begin.

Working Copy 

Working Copy is the individual copy of the working repository branch. 

Checkout (Repository to Individual Machine)

The process of extracting a new working copy from the repository is called 'Checkout'. If anything goes wrong horribly, we can have always a fallback plan this does mean you can checkout a new working copy as a fresh.

Update  (Repository to Individual Machine)

The process of re-synchronising the working copy with the new changes from the repository. 

Commit (Individual Machine to Repository)

Nice and safely committing the changes you've done on the working copy is called Commit. Until the commit happens, the changes will not be reflected in the repository.

Revision

Each time we commit, the commit is given with unique revision number.  This is hexadecimal code.

Commit Message

We have the opportunity to add some message when we commit every time. 

Fork the upstream project (Checkout)

Fork/Clone the upstream project that creates a local repository in your local GitHub. Then, separate your tasks by creating more branches, otherwise it will be more complex. 

Push into Local GitHub Repository

I have everything that upstream has, now I can update it and fix bugs whatever I want to do. Once the changes are done, I can push that into my Local GitHub Repository. 

Submit a Pull Request (PR)

Document the PR pretty well for the reviewers, so that can understand it easily. If the PR is not well documented, it will not be reviewed. The following things needs to be covered which will make the community grow and learn. 

• Why you made this request? 
• What was broken? 
• How you fixed it?

There is a button that says 'Open a Pull Request'. When I submit a PR, It is my branch against an upstream branch. 

There will be mostly two reviewers at least. Not everyone is going to be happy with code because people will do different things. Sometimes the reviewers might propose to change the code in other way, but it is up to you if you want to incorporate or stand in your ground. Now, this will be bounced back to me, and again I will be fixing it. If everything is okay for reviewers, they might say 'The PR looks good to me'. It goes for the automation testing. 

Reviewers review the PR submitted

After this reviewers will look into the code, and if something fails in the automation, there is something wrong in the submitted PR code.  But, if everything is green, they might look for code owners to approve their code. Though the code is written well, they also might look if this is leading the project in the right direction as the owner needs it. 

Automation Testing after PR

Developer, who pushed the code to repository, now needs to ensure that all the tests are running.

There are 15 automation testcases which ensure that the PR doesn't break any existing feature. All these testcases have to be Green before merging the PR. 

Automation Testing Failure

This failure leads me into logs and digging out what is happening with my code and figuring out how my change broke the cluster. 

Flaky Issues

The tests can be flaky. The flake test does not do the same thing when every time we run it, and there is also no guarantee that tests will do always the same thing. We might see a test that fails once in a week. This is called flaky. 

Often times, flake is a real issue it can break anyone's environment when the changes are merged. 

Test Environment

The pull requests are merged finally after getting the green signal from the automation testing as well as from the reviewers. This merging is an automation process like I tell the bot that I'm happy, let's merge this. 

Here, we don't want a human to push the merge button. It does all automatic process. 

Merge Conflicts (Rewind)

If the PR is not merged quickly after the approvals, you could run into merge conflicts. 

Git Hubs nicely tells that these files are conflicting. 

For example, This guy and I (Local Repo) both forked the same repository version.

But, this guy was fast enough and have finished the changes and submitted pull requests. 

This went for the automation testing when I was still doing the coding. 

Once the test pass, it merged into the central repository.

This means that I have to go to the local environment and pull in the changes from the upstream project. 

Later, fix all the conflicts which happen all the time. 

If I have pushed pull request while this guy was doing automation testing,  I have to re-push my pull requests and all the tests need to start over.  

It is like a rewind, and even if you have started working on the code changes earlier, it does matter which pushed first. 

It is essential to have CI system managing that, but it is a huge complex infrastructure because a lot goes into maintaining this. 

Integration Hell 

Once the PR is accepted, it goes to the integration hell.  

Integration hell happens when CI/CD goes down, but its far most worst GitHub goes down than CI/CD

Once the pull requests are merged, the environment becomes the test environment for everyone. 

After the PR had been through rough waters, it finally arrived the tranquil tide pool. If the changes are in tide pool, it meant it is merged. Tide is the ocean of the PR requests. 

Tide is able to merge the PR requests after the final green signal to the main repository. 

Advantages 

• Versioning control system always used for the backup purpose. 
• It facilitates when hundreds of people working together to create a single project. 
• It reduces the Man power or in other words human effort. 
• Job is easy for every one in the eco-system as it is automated. 
• Every project with often changes and releases should use CI/CD 
• Every project, which has different repositories to make the whole application work, should use this. 
• Developers receive immediate notification of the broken code they wrote. 
• Not having CICD is intimidating in a large project. 

References:

Redhat CICD pipeline