Run CumulusCI from Github Actions

CumulusCI can be used to run continuous integration builds with GitHub Actions.

In order to follow along, you should already have a repository that is hosted on GitHub and configured as a CumulusCI project. In other words, we’re assuming your project already has a cumulusci.yml and that you are successfully running CumulusCI flows locally.

There is also a template repository that is setup to run CumulusCI Flow with GitHub actions. This repository can be used as a starting point for implementing your own project or as a reference for the following material.

Note

GitHub Actions are free for open source (public) repositories. Check with GitHub about pricing for private repositories.

Create a GitHub Action Workflow

In GitHub Actions, you can define workflows which run automatically in response to events in the repository. We’re going to create an action called Apex Tests which runs whenever commits are pushed to a target GitHub repository.

Workflows are defined using files in YAML format in the .github/workflows folder within the repository. To set up the Apex Tests workflow, use your editor to create a file named apex_tests.yml in this folder and add the following contents:

name: Apex Tests

on: [push]

env:
  CUMULUSCI_KEYCHAIN_CLASS: cumulusci.core.keychain.EnvironmentProjectKeychain
  CUMULUSCI_SERVICE_github: ${{ secrets.CUMULUSCI_SERVICE_github }}

jobs:
  unit_tests:
    name: "Run Apex tests"
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Install Salesforce CLI
      run: |
        mkdir sfdx
        wget -qO- https://developer.salesforce.com/media/salesforce-cli/sfdx/channels/stable/sfdx-linux-x64.tar.xz | tar xJ -C sfdx --strip-components 1
        echo $(realpath sfdx/bin) >> $GITHUB_PATH
    - name: Authenticate Dev Hub
      run: |
        echo ${{ secrets.SFDX_AUTH_URL }} > sfdx_auth
        sfdx force:auth:sfdxurl:store -f sfdx_auth -d
    - name: Set up Python
      uses: actions/setup-python@v1
      with:
        python-version: "3.8"
    - name: Install CumulusCI
      run: |
        python -m pip install -U pip
        pip install cumulusci
    - run: |
        cci flow run ci_feature --org dev --delete-org

This workflow defines a job named Run Apex Tests which will run these steps in the CI environment after any commits are pushed:

  1. Check out the repository at the commit that was pushed

  2. Install the Salesforce CLI and authorize a Dev Hub user

  3. Install Python 3.8 and CumulusCI

  4. Run the ci_feature flow in CumulusCI in the dev scratch org, and then delete the org. The ci_feature flow deploys the package and then runs its Apex tests.

It also configures CumulusCI to use a special keychain, the EnvironmentProjectKeychain, which will load org and service configuration from environment variables instead of from files.

Configure Secrets

You may have noticed that the workflow refers to a couple of “secrets”: CUMULUSCI_SERVICE_github and SFDX_AUTH_URL. You need to add these secrets to the repository settings before you can use this workflow.

To find the settings for Secrets, open your repository in GitHub. Click the Settings tab. Then click the Secrets link on the left.

CUMULUSCI_SERVICE_github

CumulusCI may need access to the GitHub API in order to do things like look up information about dependency packages. To set this up, we’ll set a secret to configure the CumulusCI github service.

First, follow GitHub’s instructions to create a Personal Access Token.

Now, in your repository’s Secrets settings, click the “Add a new secret” link. Enter CUMULUSCI_SERVICE_github as the Name of the secret. For the Value, enter the following JSON:

{"username": "USERNAME", "password": "TOKEN", "email": "EMAIL"}

Click the “Add secret” button to save the secret.

Replace USERNAME with your GitHub username, TOKEN with the Personal Access Token you just created, and EMAIL with your email address.

SFDX_AUTH_URL

CumulusCI needs to be able to access a Salesforce org with the Dev Hub feature enabled in order to create scratch orgs. The easiest way to do this is to set up this connection locally, then copy its SFDX auth URL to a secret on GitHub.

Since you already have CumulusCI working locally, you should be able to run sfdx force:org:list to identify the username that is configured as the default Dev Hub username (it is marked with (D)).

Now run sfdx force:org:display --verbose -u [username], replacing [username] with your Dev Hub username. Look for the Sfdx Auth Url and copy it.

Attention

Treat this URL like a password. It provides access to log in as this user!

Now in your repository’s Secrets settings, click the ‘Add a new secret’ link. Enter SFDX_AUTH_URL as the Name of the secret, and the URL from above as the Value. Click the ‘Add secret’ button to save the secret.

Advanced Note

These instructions connect sfdx to your Dev Hub using the standard Salesforce CLI Connected App and a refresh token. It is also possible to authenticate sfdx using the force:auth:jwt:grant command with a custom Connected App client id and private key.

Your Secrets should look like this:

Screenshot showing the CUMULUSCI_SERVICE_github and SFDX_AUTH_URL secrets

Test the Workflow

Now you should be able to try out the workflow. Commit the new .github/workflows/apex_tests.yml file to the repository and push the commit to GitHub. You should be able to watch the status of this workflow in the repository’s Actions tab:

Screenshot showing a running GitHub Action workflow

If you open a pull request for a branch that includes the workflow, you will find a section at the bottom of the pull request that shows the results of the checks that were performed by the workflow:

Screenshot showing a successful check on a GitHub pull request

It is possible to configure the repository’s main branch as a protected branch so that changes can only be merged to it if these checks are passing.

See GitHub’s documentation for instructions to configure protected branches and enable required status checks.

Run Headless Browser Tests

It is possible to run Robot Framework tests that control a real browser as long as the CI environment has the necessary software installed. For Chrome, it must have Chrome and chromedriver. For Firefox, it must have Firefox and geckodriver.

Fortunately GitHub Actions comes preconfigured with an image that includes these browsers. However it is necessary to run the browser in headless mode. When using CumulusCI’s robot task, this can be done by passing the -o vars BROWSER:headlesschrome option.

Here is a complete workflow to run Robot Framework tests for any commit:

name: Robot Tests

on: [push]

env:
  CUMULUSCI_KEYCHAIN_CLASS: cumulusci.core.keychain.EnvironmentProjectKeychain
  CUMULUSCI_SERVICE_github: ${{ secrets.CUMULUSCI_SERVICE_github }}

jobs:
  unit_tests:
    name: "Run Robot Framework tests"
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Install Salesforce CLI
      run: |
        mkdir sfdx
        wget -qO- https://developer.salesforce.com/media/salesforce-cli/sfdx/channels/stable/sfdx-linux-x64.tar.xz | tar xJ -C sfdx --strip-components 1
        echo $(realpath sfdx/bin) >> $GITHUB_PATH
    - name: Authenticate Dev Hub
      run: |
        echo ${{ secrets.SFDX_AUTH_URL }} > sfdx_auth
        sfdx force:auth:sfdxurl:store -f sfdx_auth -d
    - name: Set up Python
      uses: actions/setup-python@v1
      with:
        python-version: "3.8"
    - name: Install CumulusCI
      run: |
        python -m pip install -U pip
        pip install cumulusci
    - run: |
        cci task run robot --org dev -o vars BROWSER:headlesschrome
    - name: Store robot results
      uses: actions/upload-artifact@v1
      with:
        name: robot
        path: robot/CumulusCI-Test/results
    - name: Delete scratch org
      if: always()
      run: |
        cci org scratch_delete dev

Connect a Persistent Org

Using the JWT flow for authentication is the recommended approach when running CumulusCI in a non-interactive environment for continuous integration with an existing org.

First, you need a Connected App that is configured with a certificate in the “Use digital signatures” setting in its OAuth settings. You can follow the Salesforce DX Developer Guide to get this set up:

Once the Connected App has been created, you can configure CumulusCI to use this Connected App to login to a persistent org by setting the following environment variables.

  • CUMULUSCI_KEYCHAIN_CLASS

  • CUMULUSCI_ORG_orgName

  • SFDX_CLIENT_ID

  • SFDX_HUB_KEY

See the below entries for the values to use with each.

Important

Setting the above environment variables negates the need to use the cci org connect command. You can simply run a cci command and pass the --org orgName option, where orgName corresponds to the name used in the CUMULUSCI_ORG_* environment variable.

In the context of GitHub Actions, all of these environment variables would be declared under the env section of a workflow. Below is an example of what this would look like:

env:
    CUMULUSCI_KEYCHAIN_CLASS: cumulusci.core.keychain.EnvironmentProjectKeychain
    CUMULUSCI_ORG_sandbox: {"username": "just.in@salesforce.org", "instance_url": "https://sfdo--sbxname.my.salesforce.com"}
    SFDX_CLIENT_ID: {{ $secrets.client_id }}
    SFDX_HUB_KEY: {{ $secrets.server_key }}

The above assumes that you have client_id and server_key setup in your GitHub encrypted secrets

CUMULUSCI_KEYCHAIN_CLASS

Set this equal to EnvironmentProjectKeychain. This instructs CumulusCI to look for org configurations in environment variables instead of files.

CUMULUSCI_ORG_orgName

The name of this environment variable dictates what name to use for the value of the --org option. For example, a value of CUMULUSCI_ORG_mySandbox would mean you use --org mySandbox to use this org in a cci command.

Set this variable equal to the following json string:

{
    "username": "USERNAME",
    "instance_url": "INSTANCE_URL"
}

  • USERNAME - The username of the user you will login to the org as.

  • INSTANCE_URL - The instance URL for the org. Should begin with the https:// schema.

You can see an example of setting this environment variable in a GitHub actions workflow in our demo repository.

Wizard Note

If the target org’s instance URL is instanceless (i.e. does not contain a segment like cs46 identifying the instance), then for sandboxes it is also necessary to set SFDX_AUDIENCE_URL to https://test.salesforce.com". This instructs CumulusCI to set the correct aud value in the JWT (which is normally determined from the instance URL).

SFDX_CLIENT_ID

Set this to your Connected App’s client id. This, combined with the SFDX_HUB_KEY variable instructs CumulusCI to authenticate to the org using the JWT Bearer Flow instead of the Web Server Flow.

SFDX_HUB_KEY

Set this to the private key associated with your Connected App (this is the contents of your server.key file). This combined with the SFDX_CLIENT_ID variable instructs CumulusCI to authenticate to the org using the JWT Bearer Flow instead of the Web Server Flow.

Deploy to a Persistent Org

The final step in a CI pipeline is often deploying newly-verified changes into a production environment. In the context of a Salesforce project, this could mean a couple of different things. It could mean that you want to deploy changes in a managed package project into a packaging org. It could also mean that you want to deploy changes in a project to a production org.

The following sections cover which tasks and flows you would want to consider based on your project’s particular needs.

Deploy to a Packaging Org

When working on a managed package project, there are two standard library flows that are generally of interest when deploying to a packaging org: deploy_packaging and ci_master.

The deploy_packaging flow deploys the package’s metadata to the packaging org.

The ci_master flow includes the deploy_packaging flow, but also takes care of:

  1. Updating any dependencies in the packaging org

  2. Deploying any unpackaged Metadata under unpackaged/pre

  3. Sets up the System Administrator profile with full FLS permissions on all objects/fields.

Deploy to a Production Org

Deployments to a Production org environment will typically want to utilize either the deploy_unmanaged flow or the deploy task.

In most cases, deploy_unmanaged will have the desired outcome. This will deploy metadata, but also unschedule Scheduled Apex and uninstall previously-deployed components that have been removed from the source repository. If you do not want incremental component removal or Apex unscheduling, use the deploy task.

Build Managed Package Versions

Once new metadata has been added to the packaging org, it is often desirable to create a new beta version for your managed package so that it can be tested. We can use the release_beta flow to accomplish this. The following shows a snippet from the main <https://github.com/SFDO-Tooling/CumulusCI-CI-Demo/blob/main/.github/workflows/main.yml> workflow in our demo repository.

release_beta:
  name: "Upload Managed Beta"
  runs-on: ubuntu-latest
  needs: deploy_packaging
  steps:
    - uses: actions/checkout@v2
    - name: Install Salesforce CLI
      run: |
        mkdir sfdx
        wget -qO- https://developer.salesforce.com/media/salesforce-cli/sfdx/channels/stable/sfdx-linux-x64.tar.xz | tar xJ -C sfdx --strip-components 1
        echo $(realpath sfdx/bin) >> $GITHUB_PATH
    - name: Authenticate Dev Hub
      run: |
       echo ${{ secrets.SFDX_AUTH_URL }} > sfdx_auth
       sfdx force:auth:sfdxurl:store -f sfdx_auth -d
    - name: Set up Python
      uses: actions/setup-python@v1
      with:
        python-version: "3.8"
    - name: Install CumulusCI
      run: |
        python -m pip install -U pip
        pip install cumulusci
    - run: |
        cci flow run release_beta --org packaging

After installing sfdx, Python, and CumulusCI, the workflow executes the release_beta flow against the packaging org. This flow does several things:

  • Uploads a new Beta Version of the package in the packaging org

  • Creates a GitHub release for the beta version

  • Generates sample release notes for the beta version

  • Merges the latest commit on the main branch into all open feature branches

Important

CumulusCI is able to connect to the packaging org via CUMULUSCI_ORG_packaging environment variable defined at the top of the workflow.