With hybrid and multi-cloud deployments becoming the norm, secrets management in the cloud is one of the biggest challenges currently faced by cloud administrators.
As organizations lean toward serverless and container-based architectures, where the application components are largely decoupled, secrets management becomes its own workflow that needs special attention.
Hard coding of keys and secrets is a strict no-no, as the cloud should be approached with a “zero trust” model for security when compared to an on-premises environment. Modern cloud-based applications need agile secrets management solutions that can be integrated with popular PaaS services, the evolving cloud operating model, and DevOps practices.
Cloud service providers offer several innovative services for security management, for example, serverless authentication using AWS Amplify. Similarly, there are several such native solutions available for managing secrets and application keys in the cloud.
In this post, I am going to explore the native solutions offered by the leading players. But, are these solutions enough? Should you explore an independent multi-cloud secrets management solution? Let’s get into it.
Why Secrets Management Is Relevant in the Cloud
Traditional on-premises deployments assume a certain trust in internal networks. The environments were planned with heightened perimeter security and a more lenient approach toward security for internal application communications.
But with the cloud, the complexity of applications has increased with multiple VM-to-VM or container-to-container communications happening for authentication, authorization, and other sensitive operations. Hence, the secure storage and transfer of keys, passwords, certificates, etc. becomes a key consideration when designing the architecture.
The good news is there’s never a lack of choices in the cloud, with multiple out-of-the-box solutions for everything, be it identity, security, or even secrets management.
A Few Considerations
Centralized storage, distribution, and access control of dynamic secrets are the basic functionality expected from all secrets management solutions. But there are a few other considerations that you should take into account:
- The process of retrieving secrets should not curb cloud flexibility.
- End-to-end access audits should be supported due to the sensitivity of information.
- A solution should offer easy integration with your application ecosystem; for instance, the ability to call secrets through CI/CD pipelines from existing DevOps tools.
Although there are cloud-native solutions, independent third-party solutions that work with different providers can be considered in multi-cloud scenarios. However, in such scenarios, the deployment location of the solution, as well as security, availability, and control-plane access should all be taken into account.
Secrets Management Options
Let’s explore some of the prominent secrets management solutions available for cloud applications.
AWS Secrets Manager
This service offered by AWS can be used to store and retrieve secrets, API Keys, OAuth tokens, and DB credentials. It can also be used to automatically rotate secrets used by several native AWS services like Amazon RDS, RedShift, and DocumentDB, plus it can be extended to other types of secrets, including OAuth tokens and API Keys, through simple Lambda functions. IAM policies can also be used to control access to credentials stored in Secrets Manager.
This tool can be integrated with AWS Key Management Service (KMS) as well to encrypt secrets for additional security, and you can leverage AWS CloudTrail logs and CloudWatch to audit and monitor key management-related activities.
As with any cloud-first applications, integration with CI/CD pipelines is a must-have for any secrets management solution, as leaving passwords exposed in CI variables and logs or, even worse, hard-coding them, would make your application vulnerable. AWS secrets manager can be used in your CI/CD pipelines to avoid these situations; a good example of such an implementation is covered here.
We know tech blogging.
AWS Systems Manager Parameter Store
AWS Systems Manager helps to logically group resources from an operations point of view; it also enables centralized configuration management and the automation of tasks. Parameter Store is a feature of Systems Manager that helps you in this process by centrally storing configuration data such as DB connection strings and secrets, thus separating them from the code used in operations.
Secrets are stored as secure strings by encrypting them via customer-specified keys stored in AWS KMS, while access to secrets in the Parameter Store is controlled through AWS IAM.
Azure Key Vault
Azure Key Vault is the native secrets management service offered by Azure to centrally manage secrets, keys, and certificates used by cloud applications. The standard tier of the service offers encryption via a software key, while the premium tier supports keys protected by hardware security modules (HSM).
Access to secrets is controlled using Azure Role-Based Access Control (RBAC) for management-plane access; Key Vault access policies are used for accessing secrets in the data layer. Recently, a Key Vault-managed HSM service was released in public preview; it delivers single-tenant, highly available FIPS 140-2 Level 3 validated HSMs as a managed service for key management.
Key Vault’s integration with native Azure services is seamless, for example, with Azure VM disk encryption or transparent data encryption for the Azure SQL service. You can also audit the usage of secrets through Key Vault logs, which can be sent to Azure storage, event hubs, or Azure Monitor for centralized visibility. Key Vault can integrate with your DevOps processes with minimal overhead as well, be it a simple Infrastructure as Code (IaC) deployment or complex application CI/CD pipelines.
To simplify the access process, you can use Azure managed identities in your code to authenticate to Key Vault and retrieve secrets. Integrating Key Vault with your ARM templates is explained in this simple example; the same process can be used for IAC deployments as well.
GCP Secret Manager
This service is the counterpart of AWS Secrets Manager and Azure Key Vault in GCP.
GCP Secret Manager is used for the centralized storage and retrieval of passwords, API Keys, certificates, and other sensitive information. It offers advanced features like key rotation and versioning of keys, along with built-in monitoring and auditing capabilities through Cloud Audit Logs. Authentication and authorization are managed by GCP Cloud IAM, with explicit permissions restricted to GCP project owners only.
In hybrid environments, context-aware access to Secret Manager can be configured through VPC access controls. It can also be used with providers like Terraform or GitHub Actions in DevOps deployments.
GCP Key Management
Cloud Key Management service from GCP focuses on cryptographic key management, with an option to choose between software- and hardware-protection for keys. It offers a one-stop solution to create, import, and manage keys for your GCP application from a centralized location. Keys stored in Cloud KMS can be used to encrypt sensitive information like secrets before transmitting them. However, GCP Secret Manager could be a more sensible choice here, as the process of encryption and decryption of secrets through KMS can be a complex process.
Vault by HashiCorp
When organizations have applications deployed across multiple clouds with secrets, passwords, keys, and certificates scattered between them, there will inevitably be a “secrets sprawl.” HashiCorp Vault is an open-source key management solution that can be used to manage secrets, certificates, and encryption keys of applications deployed across multiple cloud environments as well as on-premises deployments.
For cloud-native services that are dependent on a respective managed identity service, Vault has an advantage, as it integrates with multiple identity services like Active Directory, Okta, and SAML-based authentication, as well as managed identity services from AWS, GCP, Azure, and Alibaba. Vault also supports dynamic secrets that are generated on demand and expire within a limited lease period. The service even integrates with your favorite DevOps tools, making it easier to call those secrets from a CI/CD pipeline.
Although multi-cloud integration and unified secrets management is an advantage, Vault needs customers to deploy and manage their vault cluster, meaning aspects such as high availability, storage backend management, and infrastructure security need to be taken into account. On the other hand, native secrets management services are managed services, meaning all such aspects are taken care of by the cloud service provider. You would need to carefully weigh having to manage such elements when choosing between a native secrets management solution or an independent solution like Vault by Hashicorp
Conclusion
When it comes to secrets management in the cloud, one size does not fit all. You need to carefully analyze your application requirements and ecosystem to find the best solution for you. The path that entails both a minimal learning curve and minimal overhead would be to use the native solution offered by your cloud service provider.
However, if multi-cloud is your big picture, you might want to explore the independent players as well. Hopefully, the information provided in this blog will help you in the process!
