We've changed the config to be more consistent about "copies" and
"artifacts". This means that "copies" no longer accepts a string, but a
slice of strings. We should bump the config version and blubber version
to make it clear that we're not longer backwards compatible.
Change-Id: Ib68f623cbdcdf0aed74f4fdaec062835748e8eb9
Refactored `copies` configuration to allow for greater control over when
and which both local build context files and variant artifacts are
copied into the target image.
The new configuration introduces a "local" keyword to signify when a
`copies` entry should apply to the files from the local build context as
opposed to files from another variant's image during a multi-stage
build.
variants:
build:
copies:
- from: local
source: ./src
destination: .
Note that with this change, the user must now explicitly define whether
_any_ files should be copied in from the local build context. None will
be copied in by default.
To help keep configurations succinct, especially considering this new
requirement, and to approximate the old `copies: variant`, a shorthand
format and sane defaults for `source` and `destination` (depending whether
`from` is "local" or a variant name) were implemented.
variants:
build:
copies: [local]
development:
copies:
- from: build
- from: local
source: ./config.dev.yaml
destination: ./config.yaml
The shorthand:
copies: [ref, ...]
# is equivalent to
copies: [{ from: ref }, ...]
And the following defaults are used when no `source` and `destination`
are specified.
copies:
- from: local
# defaults to
- from: local
source: .
destination: .
copies:
- from: variant
# defaults to two entries
- from: variant
source: /srv/app # the lives.in dir
destination: /srv/app
- from: variant
source: /opt/local # the shared lib dir
destination: /opt/local
Bug: T211625
Change-Id: I4c4217905afc0762b6bd66ed594d43cc0486e3e2
Uses the github.com/ghodss/yaml library to convert YAML to JSON before
unmarshaling for the purposes of supporting YAML and JSON input while
converting to only support JSON internally.
Bug: T207694
Change-Id: I00668014907e9ea54917f5d5067cac08d0668053
All import paths and other references to the previously Blubber repo in
Phabricator have been changed to point to the new Gerrit project URL.
Note that building or testing against this change will require you to
move your working directory to `$GOPATH/src/gerrit.wikimedia.org/r/blubber`.
Contribution documentation was updated to reflect the changes, and the
`.arcvendor` submodule has been removed.
Bug: T200452
Change-Id: I3ab23e420b2825e86e2bd7c9b3da9d4de23edaa1
Summary:
Introduced a `version` config field that must be specified and match
`config.CurrentVersion`.
Changed `config.ReadConfig` to use `yaml.UnmarshalStrict` to ensure that
errors are surfaced when unknown/bad fields are present in the given
YAML config. A smaller `config.VersionConfig` is now unmarshaled first
to prevalidate the new `version` field before the entire config is
parsed.
Fixes T191460
Test Plan:
Run `go test ./...`. Run `blubber` against some configuration containing
invalid fields and ensure that it surfaces a YAML error.
Reviewers: thcipriani, demon, hashar, mmodell, mobrovac, #release-engineering-team
Reviewed By: thcipriani, #release-engineering-team
Tags: #release-engineering-team
Maniphest Tasks: T191460
Differential Revision: https://phabricator.wikimedia.org/D1021
Summary:
Certain meta data including the Blubber version and variant used at
invocation may be useful in downstream tracking.
Implemented abstract and Docker-specific instructions for labels and
modified the Docker compiler to include these instructions at the end of
the output for the final stage.
Depends on D816
Fixes T178022
Test Plan:
Run unit tests. Run `make && bin/blubber blubber.example.yaml production` and
verify that both `blubber.version` and `blubber.variant` labels are present
and accurate.
Reviewers: thcipriani, hashar, #release-engineering-team
Reviewed By: thcipriani, #release-engineering-team
Tags: #release-engineering-team
Maniphest Tasks: T178022
Differential Revision: https://phabricator.wikimedia.org/D818
Summary:
Rather than calling `log.Fatal` in `Compile` bubble the error up to
`main` and handle it there.
Test Plan: go test ./...
Reviewers: dduvall, #release-engineering-team
Reviewed By: dduvall, #release-engineering-team
Tags: #release-engineering-team
Differential Revision: https://phabricator.wikimedia.org/D784
Summary:
Debian only provides a package for the GitHub version so it's either
this or we'd have to maintain a patch to change the import paths at
packaging, a moving target for sure.
Since it's only a development dependency, we're not too worried about
versioning it and this is a much more simple solution.
Refs T175609
Test Plan: Run `go get ./... && go test ./...`.
Reviewers: thcipriani, #release-engineering-team
Reviewed By: thcipriani, #release-engineering-team
Tags: #release-engineering-team
Maniphest Tasks: T175609
Differential Revision: https://phabricator.wikimedia.org/D782
Summary:
Now that our Phabricator instance handles `go get` requests, we can use
a path without the `.git` suffix as the canonical import path.
Test Plan: Run `go test ./...` and `go build`.
Reviewers: thcipriani, #release-engineering-team
Reviewed By: thcipriani, #release-engineering-team
Tags: #release-engineering-team
Differential Revision: https://phabricator.wikimedia.org/D780
Summary:
The previous implementation for iterating over defined artifacts did not
take into account the possibility for multiple artifacts with the same
`from` variant name and would compile stages with duplicate names. This
fixes that behavior by iterating over a unique set of names.
Test Plan: Run `go test ./...` or `arc unit`.
Reviewers: thcipriani, mmodell, #release-engineering-team
Reviewed By: thcipriani, #release-engineering-team
Tags: #release-engineering-team
Differential Revision: https://phabricator.wikimedia.org/D758
Summary:
One of Golang's "advantages" is not quibbling over style. To this end
the `go fmt` command exists. This is the result of me running:
find . -name '*.go' -exec go fmt {} \;
Test Plan: built package, ran tests
Reviewers: dduvall, mmodell, #release-engineering-team
Reviewed By: dduvall, mmodell, #release-engineering-team
Subscribers: hashar
Tags: #release-engineering-team
Differential Revision: https://phabricator.wikimedia.org/D694
Summary:
Added support for definition of environment variables under
`runs.environment`. Corresponding `ENV` instructions will be added to
Dockerfile output for the unprivileged build phase.
Fixes T168425
Test Plan: Run `go test ./...`.
Reviewers: thcipriani, mobrovac, hashar, Jrbranaa, mmodell, #release-engineering-team
Reviewed By: mobrovac
Tags: #release-engineering-team
Maniphest Tasks: T168425
Differential Revision: https://phabricator.wikimedia.org/D691
Summary:
Fixes build issues around home permissions by setting `HOME` to the
unprivileged user's home directory once the "privileges dropped" build
phase has been reached.
Test Plan: Run `go test ./...`.
Reviewers: thcipriani, mobrovac, hashar, Jrbranaa, mmodell, #release-engineering-team
Reviewed By: mobrovac
Tags: #release-engineering-team
Differential Revision: https://phabricator.wikimedia.org/D686
Summary:
Refactored the Docker compiler to omit the stage name (`AS <stage>`)
from the `FROM` instruction when no dependent artifacts are declared.
This allows for use with older versions of Docker that don't support
multi-stage builds.
Test Plan: Run `go test` under the `docker` directory.
Reviewers: thcipriani, mobrovac, hashar, Jrbranaa, mmodell, #release-engineering-team
Reviewed By: mobrovac
Tags: #release-engineering-team
Differential Revision: https://phabricator.wikimedia.org/D684
Tyler Cipriani
Dan Duvall