CI/CD Basic Example:
When we are talking about CI/CD (Continuous Integration/Continuous Delivery) many times the concepts are considered more ‘philosophical’, forgetting about how to put them into practice. In this blog we exactly want to present you an approach of how to put into practice a basic CI/CD flow.
It is very important to understand the CI/CD basic flow for a Software Development project, because at the beginning it may look confusing to know what step leads to the next one, what should be done in every stage, or what element is responsible for what task.
Once this flow has been understood, the second more complex point to understand could be the configuration of the Continuous Integration Systems itself (what tools should be used, what plugins should be installed, how to set up the permissions, how to generate the containers, etc)
Although there are many ways to address a CI/CD implementation, we are going to take a basic example represented as follows:
(Basic Flow for a Back-End CI/CD Project)
This flow starts when a Developer sends source code (‘push’) to a repository, which can be GitHub. When GitHub detects a push, it sends a notification to a Continuous Integration System (for example, Jenkins). Once Jenkins receives the information from GitHub, it builds (‘build’) it through a previously established configuration (Jenkins jobs):
- Unit tests
- Integration tests
Among others, on the project. This build depends on the type of project: we can have Front-End or Back-End projects. For this example we are going to work with Java.
If the build was successful without any error, Jenkins generates a container and this container can be uploaded to Amazon ECR. On the contrary, if there’s any error, the process won’t make it until ECR.
It’s up to the Developer how often to push to the repository (from his/her IDE).
For GitHub, it is necessary to setup ‘webhooks’, which represent the ‘information point’ to notify an external service everytime there is a push:
In the previous image, the URL corresponds to a webhook associated with Jenkins. Jenkins, at the same time, allows to set up jobs associated with this webhook:
A job (in Jenkins) is a project that can be configured to specify how to compile, build and distribute a system:
The ‘trigger’ associated to this Jexz
It allows to specify the repository and the ‘branch’ associated to the project:
In this case, the build passes through a file (Jenkinsfile):
In the file Jenkinsfile we could configure the different stages an application may pass through:
In the previous image the file Jenkinsfile specifies different build stages. Every stage must be successfully executed before the next one starts.
The first stage is the ‘checkout’, in this one Jenkins collects all the sources modified by the repository once GitHub informs a new ‘Push’.
In the second stage (‘compile’) the project is built. For this example we are building a Java project through Maven, by executing this command:
mvn -U clean compile
With this command, from the source code, we generate the .class Java files (which will be later interpreted by the Java Virtual Machine).
For the third stage, the ‘Unit Tests’, the command to execute is the following one:
Once all tests have completed, the project is ‘packaged’ with the command:
Creating a .jar, which could be considered as the final ‘delivery’ that contains all the required .class for the Java project:
Finally we perform integration tests:
An example of an integration test could be to do a request to a REST API and verify that the response is always the expected one (call the API providing an argument and compare the response). For this point a temporary Docker container is used, which lasts only during the execution of the integration tests.
When the integration tests complete successfully, the final deployment is performed. The ‘deploy’ consists of taking the image recently generated and take it to the images repository (in this case Amazon ECR). In the previous example, this deployment is being done in an UAT environment, the last step before going into production.
Each build is tracked in Jenkins:
The ‘route’ (pipeline) that the project follows in Jenkins can also be graphically displayed in the next screen:
In the previous image we can observe how a flow is (in Jenkins):
In the event of an error, Jenkins shows the exact stage in which it occurred. For example in the build #39 the error happened in the Integration Tests. In the other builds there were no errors:
It is possible to obtain more information about the error:
When a stage fails the next ones (if available) won’t execute:
In each stage of the build we can see its duration (in milliseconds or seconds). Jenkins also shows the average duration for each stage in the whole project.
Once Jenkins sends the new image to Amazon ECR, Jenkins restarts the service so ECS can detect the new image (the currently available containers instances start to finish, and new instances begin to launch).
The images that are being sent to Amazon ECR are stored in repositories:
Each repository can contain one or more images:
Everytime a restart is requested from Jenkins, what is actually restarted is a service associated to an AWS ECS cluster:
In the previous image there are several services inside the cluster. Each one of those services (for this example) has an instance in execution because they have been configured for that number of instances as desired.
Once the service has been restarted, you can access the differents options the developed software may have. For example, if the project is a REST API you could start to make requests to this API.
It is important to highlight that the only ‘human’ step was done at the beginning, when the Developer sent his/her code (‘push’) to the files repository, and this single event triggered the rest of the automated process. The only concern was to do his/her code good, execute the unit tests (make sure they didn’t fail, these tests are important to avoid unnecessary execution of the automated process) and that his/her code compiled correctly. From there on the Developer can ‘forget about’ what’s happening.
Jenkins allows to configure email notification for some events. For example if the build failed, the Developers Team can be informed, to someone responsible for this stage. This allows a constant feedback to the Team so they can make the proper checks.
Once all stages are executed, the same Developer could check the last version of his/her current development. For example, if it’s about a new icon in the website, this one could be checked at the end of the whole process (there are automated tools to check ‘Front-End’ developments).
In case of a ‘Front-End’ development, Jenkins could send (when all stages are successfully completed) the updated file(s) to a Files Repository (such as Amazon S3):
(Basic CI/CD Flow for a ‘Front-End’ Project)