September 19–20, 2016: Training
September 20–22, 2016: Tutorials & Conference
New York, NY

Securing application deployments in CI/CD environments

2:25pm–3:05pm Wednesday, 09/21/2016
Financial systems Continuous delivery, Security Gramercy Audience level: Intermediate
Average rating: **...
(2.50, 4 ratings)

Prerequisite knowledge

  • An understanding of CI/CD and DevOps
  • What you'll learn

  • Understand the security risk of running ND using a multitenant CI/CD platform
  • Explore Yahoo's experience securing a CI/CD platform
  • Description

    Continuous integration and continuous deployment (CI/CD) is an orchestration of independent systems put together to form a workflow engine that automates the way we build and deploy applications to production environments. For instance, GitHub, Jenkins, Docker, Mesos-Marathon, Kubernetes, and Chef are all integrated in a certain way to achieve continuous delivery of applications to production. The workflow jobs running in CI/CD often require access to systems outside of CI/CD as part of their execution. For example, your build job running on a Jenkins slave needs permission to push newly built packages to package repositories, or you may need to use credentials to access a protected web service or a database server as part of functional or integration test phase. There are many more such cases where workflow jobs require app or user specific credentials, such as write access to (OAuth) protected source repos like GitHub, deploying an application using Mesos-Marathon, Chef, or SSH, and provisioning TLS private keys and application secrets after deploying your application.

    These requirements are often addressed by granting coarse permissions to CI/CD systems to gain access to those services. One common practice is to preconfigure a CI/CD shared headless account and store the nonephemeral credentials locally in CI/CD systems. Similarly, independent hosted systems (e.g., TravisCI and Heroku) collect and store user OAuth tokens to access source and package repositories on behalf of the user to build, distribute, and deploy applications. The current model treats the CI/CD system components as trust anchors, but this approach present problems. If a CI/CD system component gets compromised, the attacker may easily gain the ability to touch and reach any other system within the company. Similarly, a broken chain of trust from commit to deploy can directly affect the integrity of deployed applications.

    Binu Ramakrishnan highlights current security risks and CI/CD threat modeling and presents security patterns-based techniques to mitigate these risks, including a novel idea called auth events to delegate user privileges to CI/CD workflow jobs. The auth events form the foundation of the chain of trust from commit to deploy and extend it to the deployed application.

    Photo of Binu Ramakrishnan

    Binu Ramakrishnan


    Binu Ramakrishnan is a principal security engineer at Yahoo with over a decade of experience in building Internet-scale systems and anti-abuse and application security. He currently leads security engagements in Yahoo mail, working closely with product engineers and leaders to help define and implement strategic security programs. Binu is an active participant in the industry-wide initiative to secure mail-delivery infrastructure and contributed to the recent SMTP STS efforts. He is also the author of a few open source tools.