Deploying Shared Pulumi Resources with Azure DevOps.

In my last post, I showed how to generate Pulumi projects that share resources between stacks. In this post, I will show how to deploy these projects with Azure DevOps pipelines, using yaml.

Deploying the Pulumi Stack

To deploy a Pulumi stack, I created a template for a job that can be reused for each stack I want to deploy (development, staging, production etc.).

parameters:
- name: pulumiFilesDir
  type: string
- name: pulumiStack
  type: string

jobs:
    - job: DeployInfrastructure
      displayName: Deploy Infrastructure
      steps:

      - task: NodeTool@0
        displayName: Install Node.js
        inputs:
          versionSpec: "13.11.0"

      - task: Npm@1
        displayName: Install Pulumi Node Packages
        inputs:
          command: 'install'
          workingDir: ${{ parameters.pulumiFilesDir }}

      - task: Pulumi@1
        displayName: Deploy Pulumi Changes
        inputs:
          azureSubscription: $(AzureSubscription)
          command: up
          cwd: ${{ parameters.pulumiFilesDir }}
          stack: ${{ parameters.pulumiStack }}
          args: "--yes"

This template accepts 2 parameters

• pulumiFilesDirpoints to the directory containing the Pulumi project files
• pulumiStackis the name of the stack to deploy.

The tasks in the job are

1. Install Node.js on the build machine. Note that tools that are commonly used to build, test, and run JavaScript apps such as npm, Node, Yarn, and Gulp are preinstalled on Microsoft-hosted agents in Azure Pipelines and so this step may not be required. For the exact version of Node.js and npm that is preinstalled, refer to Microsoft-hosted agents.
2. Install the Pulumi packages from npm. Yarn could be used here if preferred. Dep1. loy the Pulumi stack to the specified Azure Subscription. This environment uses the Pulumi Azure task extension for Azure Pipelines which you can install from the marketplace. To use this extension, you must obtain a Pulumi access token from your account. Since the task runs in a CI environment, you can specify the access token using a build variable PULUMI_ACCESS_TOKEN for logging into your account non-interactively. Make sure to set this variable as sensitive so that it is saved as a secret. I also use a build variable for the Azure subscription, which will look something like Pay-As-You-Go(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)

From my main deployment file, I can then reference this template, to deploy both the shared infrastructure common to both stacks, as well as each individual stack.

trigger:
  - master

pool:
  vmImage: "ubuntu-latest"

stages:
  - stage: DeploySharedInfrastructure
    displayName: Deploy Shared Infrastructure
    jobs:
    - template: 'templates/azure-cd-deploy-pulumi-job.yml'
      parameters:
        pulumiFilesDir: 'infra/shared/'
        pulumiStack: shared
        
  - stage: DeployDevelopmentInfrastructure
    displayName: Deploy Development Infrastructure
    jobs:
    - template: 'templates/azure-cd-deploy-pulumi-job.yml'
      parameters:
        pulumiFilesDir: 'infra'
        pulumiStack: development  
        
  - stage: DeployProductionInfrastructure
    displayName: Deploy Production Infrastructure
    jobs:
    - template: 'templates/azure-cd-deploy-pulumi-job.yml'
      parameters:
        pulumiFilesDir: 'infra'
        pulumiStack: production  

This deployment has multiple stages

1. Deploy the infrastructure shared between development and production environments. The Pulumi project for this is stored in my git repository at infra/shared.
2. Deploy the infrastructure required for the dev environment. The Pulumi project for this is stored in my git repository at infra/.
3. Deploy the infrastructure required for the production environment. This is the same Pulumi project as step 2, just a different stack.

Note here that I have created the initial template as a job, and then used this as the only job in the stage. This is because in my project, I have more jobs in my stages, which I have not shown here. You can convert the initial template into a stage if you do not need other jobs to run.

Summary

In this post, I have shown how to deploy Pulumi stacks with shared resources using Azure DevOps pipelines.

A future enhancement to make is to utilise Manual Intervention/Approvals before deploying to production, so that the changes can be signed off by a gatekeeper before deployment.

Further reading on using Pulumi with Azure DevOps can be found in the Pulumi documentation