Is your organization taking advantage of cloud-native computing? Modern cloud computing is built on a diverse ecosystem of open-source projects and infrastructure.
Small startups and large enterprises alike depend on open-source projects to build critical container orchestration, CI/CD, and monitoring infrastructure. But how can an open-source project thrive and adapt to be so powerful across a variety of use cases and platforms?
The Cloud Native Computing Foundation (CNCF), an alliance of users, vendors, and developers, helps to expand the cloud-native community and ecosystem of projects. As stated in their charter statement:
The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and sustaining an ecosystem of open source, vendor-neutral projects. We democratize state-of-the-art patterns to make these innovations accessible for everyone.
The CNCF stewards a wide array of cloud-native tools and software. Utilizing these tools, engineering organizations can turbocharge their existing infrastructure and workflows, extending and adding powerful capabilities.
Cloud-Native CI/CD
Anyone that has worked with continuous integration or continuous delivery/deployment in the last several years is likely familiar with Jenkins, an open-source server software aimed at providing end-to-end CI/CD capabilities. Engineering teams have a variety of options when it comes to deploying Jenkins, including internal infrastructure, cloud, and managed service platforms. If you take a look inside the deployment infrastructure at a majority of companies today with significant software assets, you will most likely find a Jenkins install.
In recent years, the CNCF has stewarded several CI/CD projects with a cloud-native focus. One of those projects is Spinnaker, a multi-cloud continuous delivery tool that initially came from the Netflix engineering team. Spinnaker provides application management and deployment, with the added bonus of native integration with Jenkins, enabling teams to extend their existing capabilities with CNCF tooling.
This article will examine the three primary ways that teams can integrate both Jenkins and Spinnaker, utilizing the flexibility of Spinnaker to add multi-cloud delivery capabilities to existing CI platforms.
Jenkins as a Pipeline Trigger
Using Jenkins as a continuous integration system, with Spinnaker acting as the continuous delivery side, is probably the most familiar and commonly used implementation pattern. Jenkins is a powerful tool for CI, but Version 1 was designed and released before the ubiquitous need for cloud-first deployment scenarios. The cloud-native focus of Spinnaker means that cloud deployments are first-class concerns in the tool, providing a batteries-included implementation pattern for software delivery across a variety of platforms.
The first step to integrate Jenkins and Spinnaker is to connect them. This assumes you have a Jenkins master of Version 1.x – 2.x installed, as it is required to implement any of the scenarios presented in this article. Once that’s complete, you only have to add a Jenkins trigger to a Spinnaker pipeline.
Use Case
A great example use case for this type of implementation is a hypothetical engineering organization with an existing, on-premises Jenkins deployment. As they make plans to migrate some of their workload to the cloud, there are various options to consider. They can utilize Jenkins to handle delivery and deployment, requiring additional development cycles to configure and integrate Jenkins with a cloud provider. Conversely, they can continue to have Jenkins handle CI and utilize one of the managed services, like AWS CodeDeploy.
The issue with these two options is that both of them will leave the platform tightly coupled with a single vendor platform, potentially causing “lock-in.” What happens if, in the future, the team needs to expand their service to Google Cloud as well? By going with Spinnaker as their CD platform instead, they’re empowered to scale out to multiple cloud platforms as future needs arise.
Are you a tech blogger?
Jenkins as a Spinnaker Pipeline Stage
What about engineering teams that are further along in their journey to being cloud native? They may already be employing a hybrid or 100% cloud production system. They may have a cloud-first deployment system already in place, such as Spinnaker, but may still need to rely on special integration testing or automation that remains in their legacy Jenkins deployment.
Fortunately, Spinnaker provides this exact functionality, allowing Jenkins to be defined as a specific pipeline stage. Like the previous integration, the first step is to connect Jenkins and Spinnaker.
For teams that have an extensive collection of tests and post-build automation, this can be a great way to bridge Jenkins and Spinnaker functionalities during a migration, without consuming precious engineering resources to port and refactor automated testing.
Jenkins as a Script Stage
In some cases, deployments require more flexibility in automation and scripting. Scripting languages like Bash and Python are often employed to provide additional capabilities in DevOps workflows, and some CI/CD platforms are fairly limited in what types of custom automation can be defined.
In the case of Spinnaker, it utilizes Jenkins as a sandbox environment, allowing the execution of any arbitrary Python, Bash, or Groovy script that might be needed. As before, Jenkins needs to be connected as a CI provider inside Spinnaker. There are some additional steps required to configure Jenkins as a script provider for a pipeline stage, detailed here.
Consider the deployment workflow for an app with a UI component. Testing software with UI features has consistently been a thorn in the side of software engineers, who often have to depend on manual, interactive testing to validate that the software functions correctly. In a CI/CD workflow where many deploys might happen per day, that simply isn’t scalable. However, utilizing a Jenkins script stage, engineering teams can create automated UI testing functionality. Plus, a Jenkins script stage with shell scripting allows you to pull a Selenium Docker container into the pipeline environment, providing self-contained, automated UI testing.
Take Advantage of the Rich CNCF Ecosystem
Beyond just continuous integration and deployment, a variety of other cloud-native tools call the CNCF community home. By employing these tools, engineering teams can provide their businesses with an end-to-end, cloud-native infrastructure.
For monitoring and observability, Prometheus has quickly grown to become one of the best choices for modern cloud environments. With its powerful data querying and visualization capabilities, easy integration, and broad language support, it’s easy to see why. In the context of Jenkins and Spinnaker, Prometheus is a perfect fit to monitor both the infrastructure the application lives on, as well as the infrastructure that Spinnaker itself occupies.
A production-level deployment infrastructure will be generating a lot of event-based data as well. Unfortunately, event producers and consumers don’t always provide any consistent specification when it comes to the format of the event data itself. CNCF has the solution: The CloudEvents specification aims to define a common, easy-to-understand specification for all major event formats.
Deploying container-based workloads to multiple cloud platforms additionally brings unique security challenges. Teams that make multiple deployments per day need to be able to integrate as much security automation as they can into their deployment pipelines, catching and preventing issues before they make it into production.
Open Policy Agent provides a “unified toolset and framework for policy across the cloud native stack.” With OPA deployed, an engineering team can configure a specific policy against, say, Docker files. Developers that check in new commits to container-based applications will have their builds validated by the OPA API. Any build or configuration that fails will stop the CI workflow, alerting relevant engineers to a potential issue, and avoiding a possible deployment rollback.
If there’s any downside to the CNCF ecosystem, it’s that there isn’t nearly enough space in a blog post to cover all the projects and tools that exist across the cloud-native landscape. To see all the projects in one place, visit the CNCF landscape page. As of this writing, there are 1,477 projects represented!
Cloud-Native Adds New Capabilities
The strong ecosystem of cloud-native tools can enable organizations to extend their existing infrastructure, adding new, cloud-focused capabilities, such as multi-cloud deployments.
One caveat: Teams should be empowered to suggest and ultimately engage in a ground-up rebuild if warranted. Not all existing infrastructure makes sense for the cloud, and sometimes it’s more cost-effective and will result in better performance to implement modern design patterns versus trying to graft a modern band-aid onto a legacy platform. Fortunately, the cloud-native ecosystem has a full spectrum of tools to enable this.
By utilizing solutions such as Spinnaker, an organization gets a cloud-first deployment tool backed by a strong open-source community for support, along with broad compatibility and integration capabilities with a variety of platforms and vendors; plus, it’s platform-agnostic. Using cloud-native tools, teams can extend and improve their existing architecture while, at the same time, laying the foundation for their eventual path into the modern cloud.
