The development of SDN (Software-Defined Networking) projects is a logistical challenge. How to develop software for the data plane and the control plane at the same time? How to ensure compatibility between different layer applications? How do you quickly find bugs in these applications? Jenkins, the Swiss Army Knife for special software development tasks, is here to help you! It can be used to perform two roles: a Guardian and a Conductor. The Guardian oversees the quality of the software, while the Conductor ensures that all tests run in the right order and at the right time.
Such collaboration between the Guardian and the Conductor in the case of software development is called a process of continuous integration and continuous deployment. These are the processes by which we are able to deliver high-quality code faster using special tools.
Continuous Integration
Let’s start with explaining what the “integration” is. Continuous Integration is often abbreviated to CI. When many developers are working on an application they often encounter a problem synchronizing their work. Continuous Integration involves frequent committing code snippets to the repository. Then such a code is built and further tests are run. In case of a bug, the developer quickly gets feedback. This process is usually controlled by the Continuous Integration server. If integration is performed several times a day, it reduces the number of bugs and the cost associated with finding the same.
Continuous Delivery
Continuous Delivery – CD. After the integration phase and testing of the program, the automatic delivery phase can follow. It involves preparing the application for installation, packaging it and placing it in a central repository. With this approach, the development team can regularly send the latest version of the application to other teams in the company. For example, a team developing SDN controller software can use a prepared network device application. With its use, they can perform interface or data flow conformance tests. This way, each delivered application can be used as part of a larger system.
Continuous Deployment in SDN
Continuous Deployment is also abbreviated as CD. Thanks to the use of Continuous Deployment, after the phase of automatic testing and preparation for installation the application can be automatically installed on a production server/device. CD implementation is difficult for network devices at the data layer. Their interface often forces physical access to perform updates, and when devices are spread across the country, this can pose a logistical challenge. On the other hand, it is necessary to switch the device to the maintenance mode. It cannot then provide services and the network traffic must be switched to another path.
It is much easier to implement in the context of an SDN controller, especially when its functionality is provided through an application cluster. In this case, it is possible to update each node in the cluster in turn until they are all in their latest version. Such a solution does not disturb the continuity of the controller’s work.
Now let’s have a look at what Jenkins is and how we can use it for Continuous Integration.
What is Jenkins?
Jenkins is an application for automating software development. As the Conductor: it detects changes to the repository, launches application building and triggers test execution. As the quality Guardian: it checks code coverage with tests, performs analyses with regard to build errors and generates test reports.
To smoothly manage all these functionalities, we can create jobs in Jenkins. A job implements a continuous integration process for a single application. The tasks usually take the form of Pipelines, i.e. consecutive stages/events. An example of such a Pipeline can be seen in the figure below:
How is Jenkins supposed to know how to build and test our application? It uses Jenkinsfile – a file written in Groovy with a fairly limited syntax. To build the above Pipeline, we can use the following example code:
As you can easily guess, each of the green dots in the Pipeline reflects a stage section. Within each stage, there are steps i.e. commands to follow to complete the stage. For a complete overview of the Jenkins Pipeline syntax, please refer to the documentation page.
Continuous Integration Pipeline for SDN projects
This begs the question: why is Continuous Integration in SDN projects a challenge? Here we are dealing with two influencing layers: control and data one. Devices operating in individual layers are often manufactured using completely different technologies. Their operating characteristics, as well as performance requirements, are also different. Configuration of network devices with a controller is performed “from time to time”, whereas efficient data flow is a constant requirement.
The Continuous Integration Pipeline for the SDN project can take many forms. The approach presented here is one of many ideas, rather than the only valid approach. I encourage you to experiment and tailor the Pipeline to the requirements in your particular project.
One way to develop software for network devices is to use containers. Thanks to them we can use a common interface that is independent from the technology used to write the application. Our Jenkins Conductor watches the code repository to find changes (new commits). When it detects them, it starts the build process. The image of the built container after basic testing (Continuous Integration) can be uploaded to the central binary repository (Continuous Delivery).
It is worthwhile to note that manual testing is out of the question in case of the SDN software. Tests must be performed using specialized software like network traffic generators or simulation environments. Running the network device integration tests (see figure below) will therefore involve preparing the test environment, including downloading software containers . Next, the actual test scripts are run. Finally, a report is generated and the test environment is cleaned up. This cleanup ensures that a given test run will not affect other test runs on the same machine.
Once the network device application has been properly tested, it is time to test it with the controller. In this case, it is necessary to reproduce the real environment as closely as possible. Therefore, complex network device topologies are applied. These topologies can be created using virtual machines. To automatically create and configure multiple units, it is recommended to use Terraform or similar software with which we can describe all topology settings as a code.
Unfortunately, running such a simulation environment is time-consuming. How can this problem be solved? You only need to run the environment once and only restore the desired configuration for each subsequent test. This allows developers and testers to get feedback from tests faster if a bug is found. An example of such a Pipeline is presented in the graphic below:
Quality management
Quality management is a job for the Guardian Jenkins. At any point during the project, it is good to have an overview of the status of the entire system or application. What is the quality? “Quality” means that the software has all metrics that determine whether the application code meets coding standards. What is important is not only the current state, but also the trend, that is, whether the quality improves as the project advances. Examples of metrics that can be tracked using Jenkins:
- code coverage with tests
- warnings caught by the compiler
- results of static code analysis
- summary test results
- compliance with the accepted code format
- and many, many more
The Jenkins graphical interface makes it possible for us to display these metrics via graphs as in the figure below.
Summary
In this article, I described the process of Continuous Integration and Continuous Delivery of software. I described the challenges of implementing such a process in SDN projects. I also provided example Pipelines for implementing CI/CD in the context of developing and testing network device and controller applications. Why is this approach worth pursuing? There are at least a few reasons. Thanks to continuous integration, any software bugs are much easier to detect – developers receive the feedback much faster. Another issue is a regular delivery of applications in containers. This allows them to be tested frequently in an entire system environment. This allows for faster “delivery” of full functionality to the end-user. With Jenkins, we can also constantly check the software quality of the entire project. It is not only a tool for development teams, but also stakeholders. This tool is worth testing to automate further areas of the project step by step.
About the author
Darek has been programming for several years now. He is interested in many forms of automation and a true Jenkins enthusiast. He runs the blog called Szkoła Jenkinsa (the Jenkins School). He is also the author of the e-book titled “Top 10 wtyczek do Jenkinsa, które ułatwią Ci pracę” (Top 10 Jenkins plugins that will make your job easier) available for free.
