Functionally testing pipeline stage steps (build, run, publish, deploy,
exports) is tricky on account of either having to mock all calls to
workflow script and/or pipeline runner methods, or somehow setup
dependent backend systems such as k8s for systems tests.
The lack of coverage for these methods has resulted in a number of found
bugs and incremental fixes that could have been found through testing,
so the value of heavy mocking and system-test complexity in these cases
seems to outweigh the cost.
By installing a stubbed version of the `docker-pusher` script into the
Jenkins container, we can run a system test for the `publish` step, and
by stubbing out some of the previous stage/step context, we can more
effectively test `PipelineStage` step methods.
Change-Id: Ibed4f781e6c46dcbf5741eaf8adb79ebe4ef2d39
Importing PipelineStage into PipelineBuilder was causing an unknown
class `ExecutionContext.NodeContext` exception but only when importing
within a Jenkins CPS context. Instead of pulling out my hair trying to
debug anything related to CPS, I opted to remove the explicit type on
the PipelineStage.context property instead.
Change-Id: If33cb029b9d585bf2c6ac9754d46f64ac14143a9
Fixes a bug where any set value of `deploy.test` was overwritten with
null.
Refactored tests for default configuration to be clearer and included
tests for deploy step.
Change-Id: I2ecda922d7ddc2d873e4c31513f92a12e6ebfcf7
Refactored `ExecutionContext#getAll` to return a map of `stageName:
value` which allows one to retrieve multiple bounds and know which
happened in the same stage.
Refactored `PipelineStage#teardown` to include all image tags in the
Gerrit report.
Change-Id: I723ce2edcf1201be5eb1a726920395b24abadd9f
The previous implementation of `PipelineStage#getDefaultNodeLabels`
caused a `groovy.lang.MissingPropertyException` as it was assuming a
previously implemented but since refactored structure for the `publish`
configuration.
Tests were added to cover `Pipeline#getRequiredNodeLabels` which
transitively covers `PipelineStage#getRequiredNodeLabels` and seemed
like a better "user" facing interface to test.
Change-Id: Iced37fe8ab4482f7931f91c3a0d6255cbeda68ba
Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and
mapping user-defined pipelines, stages, and execution graphs to actual
Jenkins Pipeline stage definitions.
Provides `Pipeline` class that constructs a "stack" of `PipelineStage`
objects from the user-provided configs, each with its own `NodeContext`
for binding output values to names and consuming bound values from
previous stages.
Provides `PipelineStage` that contains core stage step implementations
based on the existing `service-pipeline` JJB job definition in
`integration/config`. A closure is returned by each stage for passing
off to Jenkins Pipeline stage definitions by the builder.
Steps have a fixed order within a given stage: build, run, publish,
deploy, exports. This allows for concise definition of a stage that
performs multiple steps, and deterministic behavior of default
configuration that references locally bound output values (e.g. the
default configuration of `image:` for an `publish: { type: image }`
publish entry is `${.imageID}`, referencing the image built in the
current stage's `build` step.) If the user needs to change ordering,
they can simply break the stage out into multiple stages.
See the `Pipeline` class for currently supported configuration. Note
that the aforementioned context system allows for user's to make use of
the same value bindings that step implementations use internally. They
can also use the `exports` configuration field to bind new values.
To illustrate the minimally required configuration, the following would
approximate the current `service-pipeline-test-and-publish` JJB job for
a project named "foo".
pipelines:
foo:
directory: src/foo
stages:
- name: test # builds/runs "test" variant
- name: candidate
build: production
publish:
image: true
deploy: # currently only the "ci" cluster
chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz
test: true
And to illustrate how the "candidate" stage in this example could be
expressed as multiple stages using references to the output names that
steps bind/export:
pipelines:
foo:
directory: src/foo
stages:
- name: tested
- name: built
build: production
- name: published
publish:
image:
id: '${built.imageID}'
exports:
image: '${.imageFullName}:${.imageTag}'
- name: staged
deploy:
image: '${published.image}'
chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz
test: true
Bug: T210267
Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
Dan Duvall