You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

245 lines
6.8 KiB

Unify `copies` and `artifacts` configuration Refactored `copies` configuration to allow for greater control over when and which both local build context files and variant artifacts are copied into the target image. The new configuration introduces a "local" keyword to signify when a `copies` entry should apply to the files from the local build context as opposed to files from another variant's image during a multi-stage build. variants: build: copies: - from: local source: ./src destination: . Note that with this change, the user must now explicitly define whether _any_ files should be copied in from the local build context. None will be copied in by default. To help keep configurations succinct, especially considering this new requirement, and to approximate the old `copies: variant`, a shorthand format and sane defaults for `source` and `destination` (depending whether `from` is "local" or a variant name) were implemented. variants: build: copies: [local] development: copies: - from: build - from: local source: ./config.dev.yaml destination: ./config.yaml The shorthand: copies: [ref, ...] # is equivalent to copies: [{ from: ref }, ...] And the following defaults are used when no `source` and `destination` are specified. copies: - from: local # defaults to - from: local source: . destination: . copies: - from: variant # defaults to two entries - from: variant source: /srv/app # the lives.in dir destination: /srv/app - from: variant source: /opt/local # the shared lib dir destination: /opt/local Bug: T211625 Change-Id: I4c4217905afc0762b6bd66ed594d43cc0486e3e2
6 years ago
Unify `copies` and `artifacts` configuration Refactored `copies` configuration to allow for greater control over when and which both local build context files and variant artifacts are copied into the target image. The new configuration introduces a "local" keyword to signify when a `copies` entry should apply to the files from the local build context as opposed to files from another variant's image during a multi-stage build. variants: build: copies: - from: local source: ./src destination: . Note that with this change, the user must now explicitly define whether _any_ files should be copied in from the local build context. None will be copied in by default. To help keep configurations succinct, especially considering this new requirement, and to approximate the old `copies: variant`, a shorthand format and sane defaults for `source` and `destination` (depending whether `from` is "local" or a variant name) were implemented. variants: build: copies: [local] development: copies: - from: build - from: local source: ./config.dev.yaml destination: ./config.yaml The shorthand: copies: [ref, ...] # is equivalent to copies: [{ from: ref }, ...] And the following defaults are used when no `source` and `destination` are specified. copies: - from: local # defaults to - from: local source: . destination: . copies: - from: variant # defaults to two entries - from: variant source: /srv/app # the lives.in dir destination: /srv/app - from: variant source: /opt/local # the shared lib dir destination: /opt/local Bug: T211625 Change-Id: I4c4217905afc0762b6bd66ed594d43cc0486e3e2
6 years ago
Unify `copies` and `artifacts` configuration Refactored `copies` configuration to allow for greater control over when and which both local build context files and variant artifacts are copied into the target image. The new configuration introduces a "local" keyword to signify when a `copies` entry should apply to the files from the local build context as opposed to files from another variant's image during a multi-stage build. variants: build: copies: - from: local source: ./src destination: . Note that with this change, the user must now explicitly define whether _any_ files should be copied in from the local build context. None will be copied in by default. To help keep configurations succinct, especially considering this new requirement, and to approximate the old `copies: variant`, a shorthand format and sane defaults for `source` and `destination` (depending whether `from` is "local" or a variant name) were implemented. variants: build: copies: [local] development: copies: - from: build - from: local source: ./config.dev.yaml destination: ./config.yaml The shorthand: copies: [ref, ...] # is equivalent to copies: [{ from: ref }, ...] And the following defaults are used when no `source` and `destination` are specified. copies: - from: local # defaults to - from: local source: . destination: . copies: - from: variant # defaults to two entries - from: variant source: /srv/app # the lives.in dir destination: /srv/app - from: variant source: /opt/local # the shared lib dir destination: /opt/local Bug: T211625 Change-Id: I4c4217905afc0762b6bd66ed594d43cc0486e3e2
6 years ago
Unify `copies` and `artifacts` configuration Refactored `copies` configuration to allow for greater control over when and which both local build context files and variant artifacts are copied into the target image. The new configuration introduces a "local" keyword to signify when a `copies` entry should apply to the files from the local build context as opposed to files from another variant's image during a multi-stage build. variants: build: copies: - from: local source: ./src destination: . Note that with this change, the user must now explicitly define whether _any_ files should be copied in from the local build context. None will be copied in by default. To help keep configurations succinct, especially considering this new requirement, and to approximate the old `copies: variant`, a shorthand format and sane defaults for `source` and `destination` (depending whether `from` is "local" or a variant name) were implemented. variants: build: copies: [local] development: copies: - from: build - from: local source: ./config.dev.yaml destination: ./config.yaml The shorthand: copies: [ref, ...] # is equivalent to copies: [{ from: ref }, ...] And the following defaults are used when no `source` and `destination` are specified. copies: - from: local # defaults to - from: local source: . destination: . copies: - from: variant # defaults to two entries - from: variant source: /srv/app # the lives.in dir destination: /srv/app - from: variant source: /opt/local # the shared lib dir destination: /opt/local Bug: T211625 Change-Id: I4c4217905afc0762b6bd66ed594d43cc0486e3e2
6 years ago
Unify `copies` and `artifacts` configuration Refactored `copies` configuration to allow for greater control over when and which both local build context files and variant artifacts are copied into the target image. The new configuration introduces a "local" keyword to signify when a `copies` entry should apply to the files from the local build context as opposed to files from another variant's image during a multi-stage build. variants: build: copies: - from: local source: ./src destination: . Note that with this change, the user must now explicitly define whether _any_ files should be copied in from the local build context. None will be copied in by default. To help keep configurations succinct, especially considering this new requirement, and to approximate the old `copies: variant`, a shorthand format and sane defaults for `source` and `destination` (depending whether `from` is "local" or a variant name) were implemented. variants: build: copies: [local] development: copies: - from: build - from: local source: ./config.dev.yaml destination: ./config.yaml The shorthand: copies: [ref, ...] # is equivalent to copies: [{ from: ref }, ...] And the following defaults are used when no `source` and `destination` are specified. copies: - from: local # defaults to - from: local source: . destination: . copies: - from: variant # defaults to two entries - from: variant source: /srv/app # the lives.in dir destination: /srv/app - from: variant source: /opt/local # the shared lib dir destination: /opt/local Bug: T211625 Change-Id: I4c4217905afc0762b6bd66ed594d43cc0486e3e2
6 years ago
Unify `copies` and `artifacts` configuration Refactored `copies` configuration to allow for greater control over when and which both local build context files and variant artifacts are copied into the target image. The new configuration introduces a "local" keyword to signify when a `copies` entry should apply to the files from the local build context as opposed to files from another variant's image during a multi-stage build. variants: build: copies: - from: local source: ./src destination: . Note that with this change, the user must now explicitly define whether _any_ files should be copied in from the local build context. None will be copied in by default. To help keep configurations succinct, especially considering this new requirement, and to approximate the old `copies: variant`, a shorthand format and sane defaults for `source` and `destination` (depending whether `from` is "local" or a variant name) were implemented. variants: build: copies: [local] development: copies: - from: build - from: local source: ./config.dev.yaml destination: ./config.yaml The shorthand: copies: [ref, ...] # is equivalent to copies: [{ from: ref }, ...] And the following defaults are used when no `source` and `destination` are specified. copies: - from: local # defaults to - from: local source: . destination: . copies: - from: variant # defaults to two entries - from: variant source: /srv/app # the lives.in dir destination: /srv/app - from: variant source: /opt/local # the shared lib dir destination: /opt/local Bug: T211625 Change-Id: I4c4217905afc0762b6bd66ed594d43cc0486e3e2
6 years ago
  1. package config
  2. import (
  3. "bytes"
  4. "context"
  5. "fmt"
  6. "path"
  7. "reflect"
  8. "regexp"
  9. "strings"
  10. "text/template"
  11. "github.com/docker/distribution/reference"
  12. "gopkg.in/go-playground/validator.v9"
  13. )
  14. var (
  15. // See Debian Policy
  16. // https://www.debian.org/doc/debian-policy/#s-f-source
  17. // https://www.debian.org/doc/debian-policy/#s-f-version
  18. debianPackageName = `[a-z0-9][a-z0-9+.-]+`
  19. debianVersionSpec = `(?:[0-9]+:)?[0-9]+[a-zA-Z0-9\.\+\-~]*`
  20. debianReleaseName = `[a-zA-Z](?:[a-zA-Z0-9\-]*[a-zA-Z0-9]+)?`
  21. debianPackageRegexp = regexp.MustCompile(fmt.Sprintf(
  22. `^%s(?:=%s|/%s)?$`, debianPackageName, debianVersionSpec, debianReleaseName))
  23. // See IEEE Std 1003.1-2008 (http://pubs.opengroup.org/onlinepubs/9699919799/)
  24. environmentVariableRegexp = regexp.MustCompile(`^[a-zA-Z_][a-zA-Z0-9_]+$`)
  25. // Pattern for valid variant names
  26. variantNameRegexp = regexp.MustCompile(`^[a-zA-Z][a-zA-Z0-9\-\.]+[a-zA-Z0-9]$`)
  27. humanizedErrors = map[string]string{
  28. "abspath": `{{.Field}}: "{{.Value}}" is not a valid absolute non-root path`,
  29. "baseimage": `{{.Field}}: "{{.Value}}" is not a valid base image reference`,
  30. "currentversion": `{{.Field}}: config version "{{.Value}}" is unsupported`,
  31. "debianpackage": `{{.Field}}: "{{.Value}}" is not a valid Debian package name`,
  32. "envvars": `{{.Field}}: contains invalid environment variable names`,
  33. "nodeenv": `{{.Field}}: "{{.Value}}" is not a valid Node environment name`,
  34. "relativelocal": `{{.Field}}: path must be relative when "from" is "local"`,
  35. "required": `{{.Field}}: is required`,
  36. "requiredwith": `{{.Field}}: is required if "{{.Param}}" is also set`,
  37. "unique": `{{.Field}}: cannot contain duplicates`,
  38. "username": `{{.Field}}: "{{.Value}}" is not a valid user name`,
  39. "variantref": `{{.Field}}: references an unknown variant "{{.Value}}"`,
  40. "variants": `{{.Field}}: contains a bad variant name`,
  41. }
  42. validatorAliases = map[string]string{
  43. "currentversion": "eq=" + CurrentVersion,
  44. "nodeenv": "alphanum",
  45. "username": "hostname,ne=root",
  46. }
  47. validatorFuncs = map[string]validator.FuncCtx{
  48. "abspath": isAbsNonRootPath,
  49. "baseimage": isBaseImage,
  50. "debianpackage": isDebianPackage,
  51. "envvars": isEnvironmentVariables,
  52. "isfalse": isFalse,
  53. "istrue": isTrue,
  54. "relativelocal": isRelativePathForLocalArtifact,
  55. "requiredwith": isSetIfOtherFieldIsSet,
  56. "variantref": isVariantReference,
  57. "variants": hasVariantNames,
  58. }
  59. )
  60. type ctxKey uint8
  61. const rootCfgCtx ctxKey = iota
  62. // newValidator returns a validator instance for which our custom aliases and
  63. // functions are registered.
  64. //
  65. func newValidator() *validator.Validate {
  66. validate := validator.New()
  67. validate.RegisterTagNameFunc(resolveJSONTagName)
  68. for name, tags := range validatorAliases {
  69. validate.RegisterAlias(name, tags)
  70. }
  71. for name, f := range validatorFuncs {
  72. validate.RegisterValidationCtx(name, f)
  73. }
  74. return validate
  75. }
  76. // Validate runs all validations defined for config fields against the given
  77. // Config value. If the returned error is not nil, it will contain a
  78. // user-friendly message describing all invalid field values.
  79. //
  80. func Validate(config interface{}) error {
  81. validate := newValidator()
  82. ctx := context.WithValue(context.Background(), rootCfgCtx, config)
  83. return validate.StructCtx(ctx, config)
  84. }
  85. // HumanizeValidationError transforms the given validator.ValidationErrors
  86. // into messages more likely to be understood by human beings.
  87. //
  88. func HumanizeValidationError(err error) string {
  89. var message bytes.Buffer
  90. if err == nil {
  91. return ""
  92. } else if !IsValidationError(err) {
  93. return err.Error()
  94. }
  95. templates := map[string]*template.Template{}
  96. for name, tmplString := range humanizedErrors {
  97. if tmpl, err := template.New(name).Parse(tmplString); err == nil {
  98. templates[name] = tmpl
  99. }
  100. }
  101. for _, ferr := range err.(validator.ValidationErrors) {
  102. if tmpl, ok := templates[ferr.Tag()]; ok {
  103. tmpl.Execute(&message, ferr)
  104. } else if trueErr, ok := err.(error); ok {
  105. message.WriteString(trueErr.Error())
  106. }
  107. message.WriteString("\n")
  108. }
  109. return strings.TrimSpace(message.String())
  110. }
  111. // IsValidationError tests whether the given error is a
  112. // validator.ValidationErrors and can be safely iterated over as such.
  113. //
  114. func IsValidationError(err error) bool {
  115. if err == nil {
  116. return false
  117. } else if _, ok := err.(*validator.InvalidValidationError); ok {
  118. return false
  119. } else if _, ok := err.(validator.ValidationErrors); ok {
  120. return true
  121. }
  122. return false
  123. }
  124. func hasVariantNames(_ context.Context, fl validator.FieldLevel) bool {
  125. for _, name := range fl.Field().MapKeys() {
  126. if !variantNameRegexp.MatchString(name.String()) {
  127. return false
  128. }
  129. }
  130. return true
  131. }
  132. func isAbsNonRootPath(_ context.Context, fl validator.FieldLevel) bool {
  133. value := fl.Field().String()
  134. return path.IsAbs(value) && path.Base(path.Clean(value)) != "/"
  135. }
  136. func isBaseImage(_ context.Context, fl validator.FieldLevel) bool {
  137. value := fl.Field().String()
  138. return reference.ReferenceRegexp.MatchString(value)
  139. }
  140. func isDebianPackage(_ context.Context, fl validator.FieldLevel) bool {
  141. value := fl.Field().String()
  142. return debianPackageRegexp.MatchString(value)
  143. }
  144. func isEnvironmentVariables(_ context.Context, fl validator.FieldLevel) bool {
  145. for _, key := range fl.Field().MapKeys() {
  146. if !environmentVariableRegexp.MatchString(key.String()) {
  147. return false
  148. }
  149. }
  150. return true
  151. }
  152. func isFalse(_ context.Context, fl validator.FieldLevel) bool {
  153. val, ok := fl.Field().Interface().(bool)
  154. return ok && val == false
  155. }
  156. func isRelativePathForLocalArtifact(_ context.Context, fl validator.FieldLevel) bool {
  157. value := fl.Field().String()
  158. from := fl.Parent().FieldByName("From").String()
  159. if value == "" || from != LocalArtifactKeyword {
  160. return true
  161. }
  162. // path must be relative and do no "../" funny business
  163. return !(path.IsAbs(value) || strings.HasPrefix(path.Clean(value), ".."))
  164. }
  165. func isSetIfOtherFieldIsSet(_ context.Context, fl validator.FieldLevel) bool {
  166. if otherField, err := ResolveJSONPath(fl.Param(), fl.Parent().Interface()); err == nil {
  167. return isZeroValue(reflect.ValueOf(otherField)) || !isZeroValue(fl.Field())
  168. }
  169. return false
  170. }
  171. func isTrue(_ context.Context, fl validator.FieldLevel) bool {
  172. val, ok := fl.Field().Interface().(bool)
  173. return ok && val == true
  174. }
  175. func isVariantReference(ctx context.Context, fl validator.FieldLevel) bool {
  176. cfg := ctx.Value(rootCfgCtx).(Config)
  177. ref := fl.Field().String()
  178. if ref == LocalArtifactKeyword {
  179. return true
  180. }
  181. for name := range cfg.Variants {
  182. if name == ref {
  183. return true
  184. }
  185. }
  186. return false
  187. }
  188. func isZeroValue(v reflect.Value) bool {
  189. return reflect.DeepEqual(v.Interface(), reflect.Zero(v.Type()).Interface())
  190. }
  191. func resolveJSONTagName(field reflect.StructField) string {
  192. return strings.SplitN(field.Tag.Get("json"), ",", 2)[0]
  193. }