Simple unit tests for ensuring basic functionality of these utility
functions. Exported from the "config" package now for testing.

Change-Id: I5f7133d16c14f99ff0b3b6392e4f210357678a88

We attempt to install dependencies from pip to a custom location using
the --target flag.  This flag is broken in Debian/Ubuntu due to a
distro-specific --user install default in versions prior to 18.1.1
(including the Stretch version).

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830892

This behavior can be switched off by specifying the --system flag in the
install command. This patch adds support for a use-system-flag directive
that allows doing so.

Bug: T227919
Change-Id: Ifacc3d96be3a89135187e46450d524a7d083f885

Uses the github.com/ghodss/yaml library to convert YAML to JSON before
unmarshaling for the purposes of supporting YAML and JSON input while
converting to only support JSON internally.

Bug: T207694
Change-Id: I00668014907e9ea54917f5d5067cac08d0668053

The builder configuration has proven useful for supporting generic
pre-entrypoint commands such as dependency managers not otherwise
supported by specific Blubber configuration. Adding additional
`builder.requirements` config expands support for such commands by
allowing the user to specify files that should be copied into the image
before the builder command runs.

To support this extra configuration, `builder` had to be changed from a
simple string to a mapping. The builder command must now by given as
`builder.command`.

The pattern of creating parent directories, copying files, and executing
one or more commands prior to the entrypoint has become a common
one. Some of the implementation of this pattern was moved from
`PythonConfig` into shared build macros `build.SortFilesByDir` and
`build.SyncFiles`. All config types that must have requirements files
copied over independently of the entire source tree (`PythonConfig`,
`BuilderConfig`, `NodeConfig`) now delegate to these functions.

Change-Id: I67f33034f22cee2851ec866cfb07ab20c23eba8c

All import paths and other references to the previously Blubber repo in
Phabricator have been changed to point to the new Gerrit project URL.

Note that building or testing against this change will require you to
move your working directory to `$GOPATH/src/gerrit.wikimedia.org/r/blubber`.

Contribution documentation was updated to reflect the changes, and the
`.arcvendor` submodule has been removed.

Bug: T200452
Change-Id: I3ab23e420b2825e86e2bd7c9b3da9d4de23edaa1

Summary:
We want to also install the wheels, not just generate them. To do that
we add a pip install -t <target> command. For this to work however, we
need to install into /opt/lib/python and amend the environment
accordingly to set PYTHONPATH and PATH in order for the libraries and
binaries to be accessible.

Reviewers: dduvall, thcipriani, #release-engineering-team

Tags: #release-engineering-team

Differential Revision: https://phabricator.wikimedia.org/D1057

Summary:
A new root and variant `python` config field is provided with two new
fields below, `version` and `requirements`.

The former, `version`, should specify the Python executable to use when
executing related package installation commands and ostensibly the same
executable that will be used to run the application.

The latter, `requirements`, should specify all pip requirements files
such that a compiler that supports layered filesystems (e.g. Docker) can
output separate instructions that will invalidate cache layers for
changes to those files independently of changes to the rest of the
codebase.

Python related instructions will be generated only if either `version`
or `requirements` are given.

Fixes T186545

Test Plan: Run `go test ./...`.

Reviewers: thcipriani, hashar, demon, #release-engineering-team

Reviewed By: thcipriani, #release-engineering-team

Tags: #release-engineering-team

Maniphest Tasks: T186545

Differential Revision: https://phabricator.wikimedia.org/D976