You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

422 lines
9.7 KiB

pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
pipeline: Builder and stage implementation Provides `PipelineBuilder` for reading `.pipeline/config.yaml` and mapping user-defined pipelines, stages, and execution graphs to actual Jenkins Pipeline stage definitions. Provides `Pipeline` class that constructs a "stack" of `PipelineStage` objects from the user-provided configs, each with its own `NodeContext` for binding output values to names and consuming bound values from previous stages. Provides `PipelineStage` that contains core stage step implementations based on the existing `service-pipeline` JJB job definition in `integration/config`. A closure is returned by each stage for passing off to Jenkins Pipeline stage definitions by the builder. Steps have a fixed order within a given stage: build, run, publish, deploy, exports. This allows for concise definition of a stage that performs multiple steps, and deterministic behavior of default configuration that references locally bound output values (e.g. the default configuration of `image:` for an `publish: { type: image }` publish entry is `${.imageID}`, referencing the image built in the current stage's `build` step.) If the user needs to change ordering, they can simply break the stage out into multiple stages. See the `Pipeline` class for currently supported configuration. Note that the aforementioned context system allows for user's to make use of the same value bindings that step implementations use internally. They can also use the `exports` configuration field to bind new values. To illustrate the minimally required configuration, the following would approximate the current `service-pipeline-test-and-publish` JJB job for a project named "foo". pipelines: foo: directory: src/foo stages: - name: test # builds/runs "test" variant - name: candidate build: production publish: image: true deploy: # currently only the "ci" cluster chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true And to illustrate how the "candidate" stage in this example could be expressed as multiple stages using references to the output names that steps bind/export: pipelines: foo: directory: src/foo stages: - name: tested - name: built build: production - name: published publish: image: id: '${built.imageID}' exports: image: '${.imageFullName}:${.imageTag}' - name: staged deploy: image: '${published.image}' chart: https://releases.wikimedia.org/charts/foo-0.0.1.tgz test: true Bug: T210267 Change-Id: I5a41d0d33ed7e9174db6178ab7921f5143296c75
5 years ago
  1. import groovy.mock.interceptor.MockFor
  2. import static groovy.test.GroovyAssert.*
  3. import groovy.util.GroovyTestCase
  4. import org.wikimedia.integration.Pipeline
  5. import org.wikimedia.integration.PipelineRunner
  6. import org.wikimedia.integration.PipelineStage
  7. import org.wikimedia.integration.ExecutionGraph
  8. import org.wikimedia.integration.ExecutionContext
  9. class PipelineStageTest extends GroovyTestCase {
  10. private class WorkflowScript {} // Mock for Jenkins Pipeline workflow context
  11. void testDefaultConfig_shortHand() {
  12. // shorthand with just name is: build and run a variant
  13. def cfg = [name: "foo"]
  14. assert PipelineStage.defaultConfig(cfg) == [
  15. name: "foo",
  16. build: '${.stage}',
  17. run: [
  18. image: '${.imageID}',
  19. arguments: [],
  20. ],
  21. ]
  22. }
  23. void testDefaultConfig_run() {
  24. def cfg = [
  25. name: "foo",
  26. build: "foo",
  27. run: true,
  28. ]
  29. assert PipelineStage.defaultConfig(cfg) == [
  30. name: "foo",
  31. build: "foo",
  32. // run: true means run the built image
  33. run: [
  34. image: '${.imageID}',
  35. arguments: [],
  36. ],
  37. ]
  38. }
  39. void testDefaultConfig_publish() {
  40. def cfg = [
  41. publish: [
  42. image: true,
  43. ],
  44. ]
  45. assert PipelineStage.defaultConfig(cfg) == [
  46. publish: [
  47. image: [
  48. // defaults to the previously built image
  49. id: '${.imageID}',
  50. // defaults to the project name
  51. name: '${setup.project}',
  52. // defaults to the pipeline start timestamp and this stage name
  53. tag: '${setup.timestamp}-${.stage}',
  54. // defaults to []
  55. tags: [],
  56. ],
  57. ],
  58. ]
  59. }
  60. void testDefaultConfig_deploy() {
  61. def cfg = [
  62. deploy: [
  63. chart: "chart.tar.gz",
  64. ],
  65. ]
  66. assert PipelineStage.defaultConfig(cfg) == [
  67. deploy: [
  68. chart: "chart.tar.gz",
  69. // defaults to the previously published image
  70. image: '${.publishedImage}',
  71. // defaults to "ci"
  72. cluster: "ci",
  73. // defaults to true
  74. test: true,
  75. ],
  76. ]
  77. cfg = [
  78. deploy: [
  79. chart: "chart.tar.gz",
  80. image: "fooimage",
  81. cluster: "foocluster",
  82. test: true,
  83. ],
  84. ]
  85. assert PipelineStage.defaultConfig(cfg) == [
  86. deploy: [
  87. chart: "chart.tar.gz",
  88. image: "fooimage",
  89. cluster: "foocluster",
  90. test: true,
  91. ],
  92. ]
  93. }
  94. void testBuild() {
  95. def pipeline = new Pipeline("foo", [
  96. stages: [
  97. [
  98. name: "foo",
  99. build: "foovariant",
  100. ],
  101. ]
  102. ])
  103. def mockRunner = new MockFor(PipelineRunner)
  104. def stack = pipeline.stack()
  105. def stage = stack[1][0]
  106. stubStageContexts(stack, [
  107. setup: { ctx ->
  108. ctx["imageLabels"] = [foo: "foolabel", bar: "barlabel"]
  109. },
  110. ])
  111. mockRunner.demand.build { variant, labels ->
  112. assert variant == "foovariant"
  113. assert labels == [foo: "foolabel", bar: "barlabel"]
  114. "foo-image-id"
  115. }
  116. mockRunner.use {
  117. def ws = new WorkflowScript()
  118. def runner = new PipelineRunner(ws)
  119. stage.build(ws, runner)
  120. }
  121. assert stage.context["imageID"] == "foo-image-id"
  122. }
  123. void testDeploy_withTest() {
  124. def pipeline = new Pipeline("foo", [
  125. stages: [
  126. [
  127. name: "foo",
  128. build: "foovariant",
  129. publish: [
  130. image: true,
  131. ],
  132. ],
  133. [
  134. name: "bar",
  135. deploy: [
  136. image: '${foo.publishedImage}',
  137. chart: 'https://an.example/chart/foo-${.stage}.tar.gz',
  138. tag: '${.stage}-tag',
  139. test: true,
  140. ],
  141. ],
  142. ]
  143. ])
  144. def mockRunner = new MockFor(PipelineRunner)
  145. def stack = pipeline.stack()
  146. def stage = stack[2][0]
  147. stubStageContexts(stack, [
  148. foo: { ctx ->
  149. ctx["imageID"] = "foo-image-id"
  150. ctx["imageName"] = "foo-project"
  151. ctx["imageFullName"] = "registry.example/bar/foo-project"
  152. ctx["imageTag"] = "0000-00-00-000000-foo"
  153. ctx["imageTags"] = ["0000-00-00-000000-foo"]
  154. ctx["publishedImage"] = "registry.example/bar/foo-project:0000-00-00-000000-foo"
  155. },
  156. ])
  157. mockRunner.demand.deployWithChart { chart, image, tag ->
  158. assert chart == "https://an.example/chart/foo-bar.tar.gz"
  159. assert image == "registry.example/bar/foo-project:0000-00-00-000000-foo"
  160. assert tag == "bar-tag"
  161. "bar-release-name"
  162. }
  163. mockRunner.demand.testRelease { releaseName ->
  164. assert releaseName == "bar-release-name"
  165. }
  166. mockRunner.use {
  167. def ws = new WorkflowScript()
  168. def runner = new PipelineRunner(ws)
  169. stage.deploy(ws, runner)
  170. }
  171. assert stage.context["releaseName"] == "bar-release-name"
  172. }
  173. void testDeploy_withoutTest() {
  174. def pipeline = new Pipeline("foo", [
  175. stages: [
  176. [
  177. name: "foo",
  178. build: "foovariant",
  179. publish: [
  180. image: true,
  181. ],
  182. ],
  183. [
  184. name: "bar",
  185. deploy: [
  186. image: '${foo.publishedImage}',
  187. chart: 'https://an.example/chart/foo-${.stage}.tar.gz',
  188. tag: '${.stage}-tag',
  189. test: false,
  190. ],
  191. ],
  192. ]
  193. ])
  194. def mockRunner = new MockFor(PipelineRunner)
  195. def stack = pipeline.stack()
  196. def stage = stack[2][0]
  197. stubStageContexts(stack, [
  198. foo: { ctx ->
  199. ctx["imageID"] = "foo-image-id"
  200. ctx["imageName"] = "foo-project"
  201. ctx["imageFullName"] = "registry.example/bar/foo-project"
  202. ctx["imageTag"] = "0000-00-00-000000-foo"
  203. ctx["imageTags"] = ["0000-00-00-000000-foo"]
  204. ctx["publishedImage"] = "registry.example/bar/foo-project:0000-00-00-000000-foo"
  205. },
  206. ])
  207. mockRunner.demand.deployWithChart { chart, image, tag ->
  208. assert chart == "https://an.example/chart/foo-bar.tar.gz"
  209. assert image == "registry.example/bar/foo-project:0000-00-00-000000-foo"
  210. assert tag == "bar-tag"
  211. "bar-release-name"
  212. }
  213. mockRunner.use {
  214. def ws = new WorkflowScript()
  215. def runner = new PipelineRunner(ws)
  216. stage.deploy(ws, runner)
  217. }
  218. assert stage.context["releaseName"] == "bar-release-name"
  219. }
  220. void testExports() {
  221. def pipeline = new Pipeline("foo", [
  222. stages: [
  223. [
  224. name: "foo",
  225. build: "foovariant",
  226. exports: [
  227. thing: '${.imageID}-${setup.timestamp}',
  228. ],
  229. ],
  230. ]
  231. ])
  232. def mockRunner = new MockFor(PipelineRunner)
  233. def mockWS = new MockFor(WorkflowScript)
  234. def stack = pipeline.stack()
  235. def stage = stack[1][0]
  236. stubStageContexts(stack, [
  237. setup: { ctx ->
  238. ctx["timestamp"] = "0000-00-00-000000"
  239. },
  240. foo: { ctx ->
  241. ctx["imageID"] = "foo-image-id"
  242. },
  243. ])
  244. mockWS.demand.echo { msg ->
  245. assert msg == "exported foo.thing='foo-image-id-0000-00-00-000000'"
  246. }
  247. mockWS.use {
  248. mockRunner.use {
  249. def ws = new WorkflowScript()
  250. def runner = new PipelineRunner(ws)
  251. stage.exports(ws, runner)
  252. }
  253. }
  254. assert stage.context["thing"] == "foo-image-id-0000-00-00-000000"
  255. }
  256. void testPublish() {
  257. def pipeline = new Pipeline("foopipeline", [
  258. stages: [
  259. [
  260. name: "built",
  261. build: "foovariant",
  262. ],
  263. [
  264. name: "published",
  265. publish: [
  266. image: [
  267. id: '${built.imageID}',
  268. ],
  269. ],
  270. ]
  271. ]
  272. ])
  273. def mockRunner = new MockFor(PipelineRunner)
  274. def stack = pipeline.stack()
  275. def stage = stack[2][0]
  276. stubStageContexts(stack, [
  277. setup: { ctx ->
  278. ctx["project"] = "foo-project"
  279. ctx["timestamp"] = "0000-00-00-000000"
  280. ctx["imageLabels"] = []
  281. },
  282. built: { ctx ->
  283. ctx["imageID"] = "foo-image-id"
  284. },
  285. ])
  286. mockRunner.demand.registerAs { imageID, imageName, tag ->
  287. assert imageID == "foo-image-id"
  288. assert imageName == "foo-project"
  289. assert tag == "0000-00-00-000000-published"
  290. }
  291. mockRunner.demand.qualifyRegistryPath { imageName ->
  292. assert imageName == "foo-project"
  293. "registry.example/bar/foo-project"
  294. }
  295. mockRunner.use {
  296. def ws = new WorkflowScript()
  297. def runner = new PipelineRunner(ws)
  298. stage.publish(ws, runner)
  299. }
  300. assert stage.context["imageName"] == "foo-project"
  301. assert stage.context["imageFullName"] == "registry.example/bar/foo-project"
  302. assert stage.context["imageTag"] == "0000-00-00-000000-published"
  303. assert stage.context["imageTags"] == ["0000-00-00-000000-published"]
  304. assert stage.context["publishedImage"] == "registry.example/bar/foo-project:0000-00-00-000000-published"
  305. }
  306. void testRun() {
  307. def pipeline = new Pipeline("foo", [
  308. stages: [
  309. [
  310. name: "foo",
  311. build: "foovariant",
  312. ],
  313. [
  314. name: "bar",
  315. run: [
  316. image: '${foo.imageID}',
  317. arguments: ['${.stage}arg'],
  318. ],
  319. ],
  320. ]
  321. ])
  322. def mockRunner = new MockFor(PipelineRunner)
  323. def stack = pipeline.stack()
  324. def stage = stack[2][0]
  325. stubStageContexts(stack, [
  326. foo: { ctx ->
  327. ctx["imageID"] = "foo-image-id"
  328. },
  329. ])
  330. mockRunner.demand.run { image, args ->
  331. assert image == "foo-image-id"
  332. assert args == ["bararg"]
  333. }
  334. mockRunner.use {
  335. def ws = new WorkflowScript()
  336. def runner = new PipelineRunner(ws)
  337. stage.run(ws, runner)
  338. }
  339. }
  340. private void stubStageContexts(stack, stubs) {
  341. stack.each { stages ->
  342. stages.each { stage ->
  343. stage.context["stage"] = stage.name
  344. stubs.each { stageName, stub ->
  345. if (stage.name == stageName) {
  346. stub(stage.context)
  347. }
  348. }
  349. }
  350. }
  351. }
  352. }