In today’s modern software development, the answer to this provocative question should be obvious to everyone—yes! Yet, the thought of giving developers complete access to production systems still gives nightmares to several seasoned IT leaders.
In this article, I’m going to explore the challenges this question poses for different roles across an organization and why it’s so important to build modern digital solutions. I’ll also cover how IT governance groups can establish proper guardrails that minimize risks without hindering the speed of your product/service development and operations.
Risk or Opportunity?
Giving software developers full access to production environments was, up until a few years ago, unthinkable in many organizations, especially large enterprises.
The risks that such a bold policy carried were truly frightening. Widening the group of employees with privileged access increases the potential liability concerns and privacy implications in case of incidents.
Traditionally, the access to the production system was shielded and restricted to a separate operations group, which received the application code or packages from the developers and was tasked to ensure deployment and smooth continuous execution.
Increasing the number of people who have production access increases the likelihood of the risks typically associated with higher-privileged access. The most common risks are operational impairments due to misconfiguration (e.g., a malformed production change causes the system to become unavailable to its customers), security breaches due to negligent actions, or confidential information leaks due to mishandling datasets.
The business impact of these risks is as bad as it gets for any organization. Typically, it can involve lost revenue due to system unavailability, reputational damage, corporate liability, or overall loss of customer trust. For many years, these potential consequences kept organizational policies increasingly strict in terms of keeping developers away from production environments.
Today, this mindset has fundamentally shifted with the DevOps movement and the emergence of cloud technologies and services.
For all the potential risks allowing developers full access to production carries, software engineers saw that an even larger opportunity awaited. By assembling multidisciplinary teams—developers, operations, quality assurance, design—to build and deliver a new product or service, software engineering teams were able to achieve an unmatched sense of empowerment and true end-to-end ownership.
The Modern Software Development Paradigm Shift
Nowadays, operations are no longer separate from development, with the two activities deeply intertwined. Long gone are the days of cumbersome handovers and processes across separate groups whenever you wanted to release new functionalities into production. In fact, a single software developer is empowered to code, test, and release new features multiple times per day. The entire role of operations and quality engineers within a team has also evolved.
Today, DevOps and quality assurance engineers on product or service development teams are, first and foremost, expert coaches that bring structure and cohesion to the team’s ways of working and tooling. Rather than being gatekeepers, they ensure the whole team is empowered and capable of operating at full speed.
Allowing the software engineer team full access to production is all about trust. You need to assume the engineering team has the best of intentions; they are, after all, work colleagues that want the best for your organization. One fact we often don’t consider is that by setting organizational policies that shield production environments from the applications developers, governance teams are only creating an incentive for developers to find workarounds in their day-to-day that are often suboptimal and potentially lower the security of the system.
Developers are the ones creating the code that goes into the system. So you should ask yourself, why restrict access to production if you’re not reviewing all the code changes that developers create?
Are you a tech expert, blogger, influencer, writer, editor, or marketer?
How to Establish Good Guardrails?
When properly considered, the question of empowering development teams with full production access ends up being about trust and establishing good guardrails. This is a classic example of where governance groups should relinquish strict control policies to ensure the organization actually has control and a solid foundation for future success.
In essence, the challenge that IT leaders in strategy and governance roles need to tackle is to balance speed, agility, and empowerment with methods that can eliminate or minimize the inherent risks. Luckily, there are several building blocks you can easily leverage in cloud-native software development that make this quite easy to achieve. Below, we cover three different methods you can use to establish good guardrails and enable this mindset.
Cloud Security Control Policies
These consist of a feature accessible to cloud administrators and can make a huge impact in empowering teams while retaining control over what really matters. This built-in capability is available in every public cloud provider and allows an organization to combine and manage all their cloud accounts and subscriptions from a single point. Moreover, it can ensure that each environment is aligned and compliant with defined organizational policies. These policies are handcrafted and customized by the organization’s administrators and act as guardrails that make it impossible for software engineering teams to override them from inside each cloud environment.
In practical terms, this enables your organization to define a security policy that ensures a certain cloud region or service can’t be used or that has some conditions. Applied to the organization’s cloud accounts, this policy would block any action in violation of it, even if the software engineering team has full privileges on a specific cloud environment.
Identity and Access Management (IAM)
This is not a new concept, but it’s definitely something worth exploring in the context of public cloud capabilities. Every public cloud provider has made their IAM service a key piece of their customer operating model and security posture. With a tight-knit and built-in integration across the entire provider ecosystem, it is virtually impossible for a developer to take an action that is not authorized by an IAM role or policy in their cloud environment.
This makes it very easy to define the proper access and also create audit trails to see who did what and when. Moreover, it can be integrated with your employee directory and system architecture, making it easy to both enable single sign-on and establish security constraints and dependencies between cloud resources within your system.
Infrastructure as Code (IaC) and CI/CD
These are common practices and processes a software engineering team will establish to build, deploy, and operate their workload. It might not be immediately obvious, but IaC and CI/CD play a key role in how a team can define and establish their own guardrails.
By making infrastructure as code a must-have practice, engineers are decreasing the risk of human error in their deployments and also enabling a clear auditing and peer review process in their day-to-day operations. When combined with CI/CD, the changes can be reviewed, tested, and automatically deployed from a single service account, rather than using the developer’s local machine with their own personal privileged access.
Conclusion
The three guardrail concepts illustrated in this article are even better when you apply them simultaneously across different levels of your organization. They give the necessary flexibility for different groups in the organization to define different privileged access levels while significantly lowering operational risks.
Empowering software engineering teams to have end-to-end responsibility and ownership of their system is at the very core of the modern software development mindset. The opposite—strict control policies and blind mechanisms—is just an open invite for engineers to come up with their own workloads; plus, it doesn’t build a culture of trust and empowerment.
I urge any IT leader today to explore how to make the most out of modern practices and reap the benefits that this modern and value-driven organizational approach brings.
