Carvel and Crossplane packages

We store the code and configuration for the reference packages in the packages folder, in a structure like the following:

packages
├── <PROVIDER>
│   └── <PACKAGING>
│       └── <NAME>
│           ├── ...
│           ├── ...
┆           ┆

• PROVIDER: the provider we build the package for (i.e., aws, azure, gcp, multicloud).
• PACKAGING: the packaging system used (currently, it must be either carvel or crossplane).
• NAME: the package name.

The following diagram shows that multiple packages can be built and tested in parallel via the publish-packages.yml workflow.

graph LR
  A(List packages) --> B(Bump version)
  B --> C
  B --> D

  subgraph Crossplane
    D(Publish Crossplane packages) --> G(Test Crossplane packages)
  end

  subgraph Carvel
    C(Publish Carvel packages) --> E(Publish Carvel repository)
    E --> F(Test Carvel packages)
  end

  click A "#list-packages"
  click B "#bump-version"
  click C "#publish-carvel-packages"
  click E "#publish-carvel-repository"
  click F "#test-carvel-packages"
  click D "#publish-crossplane-packages"
  click G "#test-crossplane-packages"

This workflow runs upon a push event to either main or develop branches and pull request events targeting the main branch. Only the files matching the packages/*/*/*/** glob pattern trigger the workflow: this reflects the packages structure described before and prevents changes to files outside actual package directories from triggering it.

List packages¶

This job evaluates the differences since the previous commit and builds two lists of packages, for Carvel and Crossplane, to feed the following GitHub Actions matrix jobs.

Bump version¶

The repository is tagged with a new version, bumping the latest one by either the major, minor, or patch fields. The git comment determines which section of the version we increment. Depending on the occurrence of #major, #minor, #patch (default is patch). You can find more information here.

Publish Carvel packages¶

We build the changed Carvel packages in parallel and publish them to GitHub Packages using Carvel's kctrl CLI. kctrl also produces package metadata files needed to build the Carvel repository. We store these metadata files in a separate (temporary) git branch for each package.

Publish Carvel repository¶

We collect the packages' metadata files produced in the previous job into one branch. Finally, we use kctrl to author the pre-release Carvel repository and publish it to GitHub Packages in OCI format. The artifact produced by kctrl is a manifest file for the PackageRepository resource, useful for quickly deploying the repository on top of a kapp-enabled Kubernetes cluster. This file is attached to a pre-release; we use this file in the following job for testing the packages.

Test Carvel packages¶

This phase aims at testing the Carvel packages by spawning one (GitHub Workflow) job per package and running them in parallel on separate GitHub Actions runners. Each job performs the following steps:

1. install the Carvel suite
2. create a kind cluster
3. install kapp-controller
4. install secretgen-controller
5. install the Carvel repository published in the previous job

6. run a test script that adheres to a specific naming convention (scripts/carvel-e2e-$PACKAGE_PROVIDER-$PACKAGE_NAME.sh), to:

   1. install the cloud operator specific to the current package (i.e., ACK, ASO, ...)
   2. deploy the package with sample input data (need to consider also firewall rules to allow network traffic)
   3. deploy an application to consume the service provided via the package
   4. ingest data into the application
   5. assert the ingested data

7. clean up everything

Publish Crossplane packages¶

We build Crossplane packages leveraging ytt, for dealing with templating, and the up CLI from Upbound, for building the packages and pushing them to an OCI registry (GitHub Packages).

Test Crossplane packages¶

The test phase is similar to Carvel's, which means:

1. create a kind cluster
2. install Upbound's Universal Crossplane

3. run a test script that adheres to a specific naming convention (scripts/crossplane-e2e-$PACKAGE_PROVIDER-$PACKAGE_NAME.sh), to:

   1. install the Crossplane provider(s) needed by the package
   2. install the Crossplane package
   3. create a claim to trigger the build of the final managed resources
   4. deploy and test an application

4. clean up everything