kustomize API 使用手册

使用kustomize应该遵循应用程序代码 和k8s资源清单分开，服务镜像，配置和环境所有一切都是通过k8s资源清单声明，并且根据git commit版本控制。应用程序部署和生命周期的管理都是自动化的，不需要人为参与，而且可审核的且易于理解的

当我们的微服务数量很多或者环境数量很多时，我们就有许多套的配置，这些配置只有细微的差别，而在很大程度上都一样，kustomize可以通过继承模版的方式，然后个性化覆盖模版生成最终部署应用所需的描述文件。

kustomize 的入门使用网上有的很多帖子，我就不在这里班门弄斧了。直接上干货😃

kustomization API使用

kustomize 版本v3.0.0 以上

字段类型描述bases[]string此列表中的每个条目都应该解析为包含kustomization.yaml文件的目录commonAnnotationsmap[string]string添加到所有资源的注解commonLabelsmap[string]string要添加到所有资源和选择器的标签components创造中，未来会发布configMapGenerator[][ConfigMapArgs][]ConfigMapArgs此列表中的每个条目都生成一个ConfigMapcrds[]string增加对 CRD 的支持。generatorOptionsgeneratorOptions修改生成所有ConfigMap和Secret Generator的行为images修改镜像的名称、tag 或 image digest。namePrefixstring此字段的值为所有资源的名称前缀namespacestring为所有资源添加 namespace。nameSuffixstring为所有资源和引用的名称添加后缀。patches[]stringPatches 在资源上添加或覆盖字段，非常实用patchesJson6902json通过json格式生成kubernetes 对象类型与patchespatchesStrategicMerge[]string通过文件定义的方式进行增加补丁或者修改字段 属性replicasstring修改资源的副本数。resources[]string每个条目必须解析为现有的资源配置文件secretGeneratorsecretGenerator生成 Secret 资源vars[]string通过变量引用其他文件的资源和属性-反射功能

bases

bases在v2.1.0中已弃用该字段

bases移到resonrces字段中。这使得基础（仍然是一个资源中心概念）相对于其他输入资源进行排序。

commonAnnotations

为所有资源添加注释，如果资源上已经存在注解键，该值将被覆盖。

cat <<EOF >./deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: EOF cat <<EOF >./kustomization.yaml namespace: my-namespace commonAnnotations: oncallPager: 800-555-1212 resources: - deployment.yaml EOF

运行kubectl kustomize ./查看Deployment资源中设置的所有字段:

apiVersion: apps/v1 kind: Deployment metadata: annotations: oncallPager: 800-555-1212 labels: app: bingo name: dev-nginx-deployment-001 namespace: my-namespace spec: ....

commonLabels

为所有资源和 selectors 增加标签。如果资源上已经存在注解键，该值将被覆盖。

cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx ports: - containerPort: 80 EOF # Create a kustomization.yaml cat <<EOF >./kustomization.yaml namePrefix: dev- commonLabels: app: my-nginx resources: - deployment.yaml EOF

运行kubectl kustomize ./查看Deployment资源中设置的所有字段:

apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx app: my-nginx replicas: 2 template: metadata: labels: run: my-nginx app: my-nginx spec: ....

components

创造中，未来会发布

configMapGenerator

简单来说就是生成对应configmap文件。

下面有几个例子：

1.通过文件生成configmap

configMapGenerator中的files列表 中添加对应文件

cat <<EOF >application.properties FOO=Bar EOF cat <<EOF >./kustomization.yaml configMapGenerator: - name: example-configmap-1 files: - application.properties EOF

运行kubectl kustomize ./查看：

生成的ConfigMap是：

apiVersion: v1 data: application.properties: | FOO=Bar kind: ConfigMap metadata: name: example-configmap-1-8mbdf7882g

2.通过在kustomization 文件内自定义文本键值对生成ConfigMap

configMapGenerator的literals列表中添加对应文本键值

cat <<EOF >./kustomization.yaml configMapGenerator: - name: example-configmap-2 literals: - FOO=Bar EOF

生成的ConfigMap是：

apiVersion: v1 data: FOO: Bar kind: ConfigMap metadata: name: example-configmap-2-g2hdhfc6tk

3.env环境文件生成创建ConfigMap

cat <<EOF >tracing.env ENABLE_TRACING=true SAMPLER_TYPE=probabilistic SAMPLER_PARAMETERS=0.1 EOF cat <<EOF >./kustomization.yaml configMapGenerator: - name: tracing-options envs: - tracing.env EOF

运行kubectl kustomize ./查看：

生成的ConfigMap是：

apiVersion: v1 kind: ConfigMap metadata: name: tracing-options-6bh8gkdf7k data: ENABLE_TRACING: "true" SAMPLER_TYPE: "probabilistic" SAMPLER_PARAMETERS: "0.1"

4.为生成的configmap 添加标签和（或）注释

options 字段允许用户为生成的实例添加标签和（或）注释，或者分别禁用该实例名称的哈希后缀。此处添加的标签和注释不会被 kustomization 文件 generatorOptions 字段关联的全局选项覆盖。但是如果全局 generatorOptions 字段指定 disableNameSuffixHash: true，其他 options 的设置将无法将其覆盖。

cat <<EOF >./kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization configMapGenerator: - name: special-config namespace: default options: annotations: dashboard: "1" labels: app.kubernetes.io/name: "app1" envs: - as.env EOF

运行kubectl kustomize ./查看：

生成的ConfigMap是：

apiVersion: v1 data: nq: nqkj.nq.lan kind: ConfigMap metadata: annotations: dashboard: "1" labels: app.kubernetes.io/name: app1 fruit: apple name: special-config-82682bthg5 namespace: default

crds

此列表中的每个条目都应该是自定义资源定义（CRD）文件的相对路径。

该字段的存在是为了让 kustomize 识别用户自定义的 CRD ，并对这些类型中的对象应用适当的转换。

典型用例：CRD 引用 ConfigMap 对象

在 kustomization 中，ConfigMap 对象名称可能会通过 namePrefix 、nameSuffix 或 hashing 来更改 CRD 对象中该 ConfigMap 对象的名称， 引用时需要以相同的方式使用 namePrefix 、 nameSuffix 或 hashing 来进行更新。

Annotations 可以放入 openAPI 的定义中：

“x-kubernetes-annotation”: "”“x-kubernetes-label-selector”: "”“x-kubernetes-identity”: "”“x-kubernetes-object-ref-api-version”: “v1”,“x-kubernetes-object-ref-kind”: “Secret”,“x-kubernetes-object-ref-name-key”: “name”, apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization crds: - crds/typeA.yaml - crds/typeB.yaml

generatorOptions

生成的ConfigMap和Secret通过散列内容添加后缀。这将确保在更改内容时生成新的ConfigMap或Secret

disableNameSuffixHash: true 禁用hash 生成name后缀

cat <<EOF >./kustomization.yaml configMapGenerator: - name: example-configmap-3 literals: - FOO=Bar generatorOptions: disableNameSuffixHash: true labels: type: generated annotations: note: generated EOF

运行kubectl kustomize ./查看：

查看生成的ConfigMap:

apiVersion: v1 data: FOO: Bar kind: ConfigMap metadata: annotations: note: generated labels: type: generated name: example-configmap-3

images

修改镜像的名称、tag 或 image digest。

通过在kustomize .yaml的images字段中指定新的镜像来更改容器内使用的镜像。

cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: template: spec: containers: - name: my-nginx image: nginx ports: - containerPort: 80 EOF cat <<EOF >./kustomization.yaml resources: - deployment.yaml images: - name: nginx newName: my.image.registry/nginx newTag: 1.4.0 EOF

运行kubectl kustomize ./查看正在使用的镜像是否更新:

apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: template: spec: containers: - image: my.image.registry/nginx:1.4.0 name: my-nginx ports: - containerPort: 80

namePrefix

为所有资源和引用的名称添加前缀。

服务名称可能会发生更改。不建议在命令参数中硬编码服务名称。对于这种用法，Kustomize可以通过vars将服务名称注入容器

cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx command: ["start", "--host", "\$(MY_SERVICE_NAME)"] EOF # Create a service.yaml file cat <<EOF > service.yaml apiVersion: v1 kind: Service metadata: name: my-nginx labels: run: my-nginx spec: ports: - port: 80 protocol: TCP selector: run: my-nginx EOF cat <<EOF >./kustomization.yaml namePrefix: dev- nameSuffix: "-001" resources: - deployment.yaml - service.yaml

运行kubectl kustomize ./查看注入容器的服务名称是dev-my-nginx-001:

apiVersion: apps/v1 kind: Deployment metadata: name: dev-my-nginx-001 spec: replicas: 2 selector: matchLabels: run: my-nginx template: metadata: labels: run: my-nginx spec: containers: - command: - start - --host - dev-my-nginx-001 image: nginx name: my-nginx

namespace

为所有资源添加 namespace。

如果在资源上设置了现有 namespace，则将覆盖现有 namespace；如果在资源上未设置现有 namespace，则使用现有 namespace。

cat <<EOF >./deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx EOF cat <<EOF >./kustomization.yaml namespace: my-namespace resources: - deployment.yaml EOF

运行kubectl kustomize ./查看Deployment

apiVersion: apps/v1 kind: Deployment metadata: labels: app: nginx name: dev-nginx-deployment namespace: my-namespace

nameSuffix

为所有资源和引用的名称添加后缀。

cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: EOF # Create a service.yaml file cat <<EOF > service.yaml apiVersion: v1 kind: Service metadata: name: my-nginx spec: EOF cat <<EOF >./kustomization.yaml nameSuffix: "-001" resources: - deployment.yaml - service.yaml

运行kubectl kustomize ./查看注入容器的服务名称是my-nginx-001:

apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx-001 spec: --- apiVersion: v1 kind: Service metadata: name: my-nginx-001 spec:

patches

通过Patches 进行替换/新增资源清单的属性值

Patches 在资源上添加或覆盖字段，Kustomization 使用 patches 字段来提供该功能。

patches 字段包含要按指定顺序应用的 patch 列表。

patch 可以:

是一个 strategic merge patch，或者是一个 JSON patch。也可以是 patch 文件或 inline string针对单个资源或多个资源

目标选择器可以通过 group、version、kind、name、namespace、标签选择器和注释选择器来选择资源，选择一个或多个匹配所有指定字段的资源来应用 patch。

op: [add,replace]

替换 /新增的方式有三种

直接在Kustomization 通过 | - op: replace path: /metadata/name value: beautiful-country-bigdata

进行操作

2.通过yaml进行操作

3.通过j son文件进行操作

apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization patches: - path: patch.yaml target: group: apps version: v1 kind: Deployment name: deploy.* labelSelector: "env=dev" annotationSelector: "zone=west" - patch: |- - op: replace path: /some/existing/path value: new value target: kind: MyKind labelSelector: "env=dev"

首先我们记住deployment.yaml和service.yaml文件长什么样

deployment.yaml

apiVersion: apps/v1 kind: Deployment metadata: labels: app: java name: java spec: selector: matchLabels: app: java template: metadata: labels: app: java spec: containers: - image: java name: java ports: - containerPort: 8080 name: web apiVersion: v1 kind: Service metadata: name: java spec: selector: app: java ports: - name: http port: 8001 targetPort: 8001

下面案例都是基于上面的deployment.yaml和service.yaml 进行操作

1.通过Kustomization 直接编写替换/新增资源清单属性

apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - ./deployment.yaml - ./service.yaml commonLabels: app: bigdata images: - name: java newName: registry.cn-qingdao.aliyuncs.com/nqkj-snapshot/sky-bigdata newTag: develop-be43cc32 patches: - patch: | - op: replace path: /metadata/name value: bigdata - op: replace path: /spec/template/spec/containers/0/name value: bigdata - op: replace path: /spec/template/spec/containers/0/ports/0/containerPort value: 8001 target: group: apps kind: Deployment version: v1 - patch: | - op: replace path: /metadata/name value: bigdata target: kind: Service

运行kubectl kustomize ./查看：

生成的deployment.yaml和service.yaml 为:

apiVersion: v1 kind: Service metadata: labels: app: bigdata name: bigdata spec: ports: - name: http port: 8001 targetPort: 8001 selector: app: bigdata --- apiVersion: apps/v1 kind: Deployment metadata: labels: app: bigdata name: bigdata spec: selector: matchLabels: app: bigdata template: metadata: labels: app: bigdata spec: containers: image: registry.cn-qingdao.aliyuncs.com/nqkj-snapshot/sky-bigdata:develop-ad8ed411 name: bigdata ports: - containerPort: 8001 name: web

注意： 当资源清单属性为数组的时候：比如containers: 下面有多个镜像或者env: 下面有多个环境变量时，我们要通过下标的方式进行编写path

下标为0 的镜像下的prot下标为0 的value为8001

参考：

- op: replace path: /spec/template/spec/containers/0/ports/0/containerPort value: 8001

2.如果通过.yaml 文件进行修改/新增资源清单属性

准备patch.yaml,内容如下

- op: replace path: /metadata/name value: oauth-server - op: replace path: /spec/template/spec/containers/0/name value: oauth-server

Kustomization

resources: - ./deployment.yaml - ./service.yaml commonLabels: # 标签 app: oauth-server images: - name: java newName: my-registry/my-postgres patches: - path: patch.yaml target: group: apps kind: Deployment version: v1

运行kubectl kustomize ./查看：

生成的deployment.yaml和service.yaml 为:

apiVersion: v1 kind: Service metadata: labels: app: oauth-server name: java spec: ports: - name: http port: 8001 targetPort: 8001 selector: app: oauth-server --- apiVersion: apps/v1 kind: Deployment metadata: labels: app: oauth-server name: oauth-server spec: selector: matchLabels: app: oauth-server template: metadata: labels: app: oauth-server spec: containers: - image: my-registry/my-postgres name: oauth-server ports: - containerPort: 8080 name: web

与1 的生成的文件一样，效果也是一样，.json文件这里就不演示，与下面的patchesJson6902 是一样

patchesJson6902

通过json 文件的方式替换/新增资源清单的属性值

patchesJson6902 不光可以使用json文件也是可以使用yaml文件与上面的patches如出一辙

注意： 在通过target: 匹配资源清单的同时必须加入target>name: 属性，属性值为Deployment 的name，这是与patches的区别

我们还是使用patches所使用的deployment.yaml和service.yaml 作为基础模版文件

准备 patch.json 和 patch-svc.json

patch.json

[ { "op": "replace", "path": "/metadata/name", "value": "oauth-server" }, { "op": "add", "path": "/spec/template/spec/containers/0/name", "value": "oauth-server"} ]

patch-svc.json

[ { "op": "replace", "path": "/metadata/name", "value": "oauth-server" } ]

Kustomization

resources: - ../../../template commonLabels: # 标签 app: oauth-server images: - name: java newName: my-registry/my-postgres patchesJson6902: - path: patch.json target: group: apps kind: Deployment version: v1 name: java - path: patch-svc.json # 指定json target: version: v1 kind: Service name: java

运行kubectl kustomize ./查看：

生成的deployment.yaml和service.yaml 为:

apiVersion: v1 kind: Service metadata: labels: app: oauth-server name: oauth-server spec: ports: - name: http port: 8001 targetPort: 8001 selector: app: oauth-server --- apiVersion: apps/v1 kind: Deployment metadata: labels: app: oauth-server name: oauth-server spec: selector: matchLabels: app: oauth-server template: metadata: labels: app: oauth-server spec: containers: - image: my-registry/my-postgres name: oauth-server ports: - containerPort: 8080 name: web

patchesStrategicMerge

通过.yaml 文件的方式为要生成的资源清单定义补丁

注意补丁的yaml 文件的name 要跟模版清单一致，下面是deployment.yaml模版清单，increase_replicas.yaml和set_memory.yaml为补丁。在kustomization声明引用补丁文件，最后合并成带补丁的资源清单。

cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx ports: - containerPort: 80 EOF # Create a patch increase_replicas.yaml cat <<EOF > increase_replicas.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: replicas: 3 EOF # Create another patch set_memory.yaml cat <<EOF > set_memory.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: template: spec: containers: - name: my-nginx resources: limits: memory: 512Mi EOF cat <<EOF >./kustomization.yaml resources: - deployment.yaml patchesStrategicMerge: - increase_replicas.yaml - set_memory.yaml EOF

运行kubectl kustomize ./查看Deployment:

apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: replicas: 3 selector: matchLabels: run: my-nginx template: metadata: labels: run: my-nginx spec: containers: - image: nginx limits: memory: 512Mi name: my-nginx ports: - containerPort: 80

replicas

修改资源的副本数。

对于如下 kubernetes Deployment 片段：

# deployment.yaml kind: Deployment metadata: name: deployment-name spec: replicas: 3

在 kustomization 中添加以下内容，将副本数更改为 5：

apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization replicas: - name: deployment-name count: 5

运行kubectl kustomize ./查看：

kind: Deployment metadata: name: deployment-name spec: replicas: 5

secretGenerator

生成 Secret 资源。

还可以从文件或文本键值对生成Secret。要从文件生成秘密，请在secretGenerator中的files列表中添加一个条目。下面是一个用文件中的数据项生成秘密的示例。

# Create a password.txt file cat <<EOF >./password.txt username=admin password=secret EOF cat <<EOF >./kustomization.yaml secretGenerator: - name: example-secret-1 files: - password.txt EOF

生成的Secret如下:

apiVersion: v1 data: password.txt: dXNlcm5hbWU9YWRtaW4KcGFzc3dvcmQ9c2VjcmV0Cg== kind: Secret metadata: name: example-secret-1-t2kt65hgtb type: Opaque

vars

通过变量引用其他文件的资源和属性-反射功能

Kustomize可以通过vars将服务名称注入容器

下面例子中 有deployment.yaml 和service.yaml。在deployment.yaml 文件中command 属性里面的脚本中引用了一个环境变量MY_SERVICE_NAME 。这个环境变量在kustomization 文件中通过vars 声明，并且环境变量MY_SERVICE_NAME 的value 引用的是service.yaml 的name。

cat <<EOF > deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx command: ["start", "--host", "\$(MY_SERVICE_NAME)"] EOF cat <<EOF > service.yaml apiVersion: v1 kind: Service metadata: name: my-nginx-001 labels: run: my-nginx spec: ports: - port: 80 protocol: TCP selector: run: my-nginx EOF cat <<EOF >./kustomization.yaml resources: - deployment.yaml - service.yaml vars: - name: MY_SERVICE_NAME objref: kind: Service name: my-nginx apiVersion: v1 EOF

运行kubectl kustomize ./查看注入容器的服务名称是dev-my-nginx-001:

apiVersion: apps/v1 kind: Deployment metadata: name: dev-my-nginx-001 spec: replicas: 2 selector: matchLabels: run: my-nginx template: metadata: labels: run: my-nginx spec: containers: - command: - start - --host - dev-my-nginx-001 image: nginx name: my-nginx

综合案例使用

定义基础模版

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Kt9tO0Jb-1603329585345)(/Users/zcj/Library/Application Support/typora-user-images/image-20201021120207783.png)]

Deployment.yaml

apiVersion: apps/v1 kind: Deployment metadata: labels: app: java name: java spec: selector: matchLabels: app: java template: metadata: labels: app: java spec: containers: - image: java name: java ports: - containerPort: 8080 name: web - containerPort: 8081 name: actuator

进行个性化定义清单

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-XoKBAQc2-1603329585347)(/Users/zcj/Library/Application Support/typora-user-images/image-20201021120341712.png)]

kustomization.yaml

resources: - ../../../template commonLabels: app: sky-bigdata commonAnnotations: prometheus.io/path: /actuator/prometheus prometheus.io/port: "8081" prometheus.io/scrape: "true" traffic.sidecar.istio.io/excludeOutboundPorts: "3101" images: - name: java newName: registry.cn-qingdao.aliyuncs.com/bigdata newTag: develop-be43cc32 patches: - patch: | - op: replace path: /metadata/name value: sky-bigdata - op: replace path: /spec/template/spec/containers/0/name value: sky-bigdata - op: replace path: /spec/template/spec/containers/0/ports/0/containerPort value: 8001 target: group: apps kind: Deployment version: v1 - patch: | - op: replace path: /metadata/name value: sky-bigdata target: kind: Service patchesStrategicMerge: - env.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization

运行kubectl kustomize ./查看：

apiVersion: v1 kind: Service metadata: annotations: prometheus.io/path: /actuator/prometheus prometheus.io/port: "8081" prometheus.io/scrape: "true" traffic.sidecar.istio.io/excludeOutboundPorts: "3101" labels: app: sky-bigdata name: sky-bigdata spec: ports: - name: http port: 8001 targetPort: 8001 selector: app: sky-bigdata --- apiVersion: apps/v1 kind: Deployment metadata: annotations: prometheus.io/path: /actuator/prometheus prometheus.io/port: "8081" prometheus.io/scrape: "true" traffic.sidecar.istio.io/excludeOutboundPorts: "3101" labels: app: sky-bigdata name: sky-bigdata spec: selector: matchLabels: app: sky-bigdata template: metadata: annotations: prometheus.io/path: /actuator/prometheus prometheus.io/port: "8081" prometheus.io/scrape: "true" traffic.sidecar.istio.io/excludeOutboundPorts: "3101" labels: app: sky-bigdata spec: containers: - env: - name: ACTIVE value: pro image: registry.cn-qingdao.aliyuncs.com/nqkj-snapshot/sky-bigdata:develop-be43cc32 name: sky-bigdata ports: - containerPort: 8001 name: web - containerPort: 8081 name: actuator