Kubernetes准入控制器指南

作者: Malte Isberner(StackRox)

如今,Kubernetes极大地提高了后端集群在生产中的速度和可管理性。由于其灵活性,可扩展性和易用性,Kubernetes已成为容器协调器中的事实上的标准。 Kubernetes还提供了确保生产工作负载安全的一系列功能。安全功能的最新介绍是一组名为“ 准入控制器。”必须启用准入控制器才能使用Kubernetes的一些更高级的安全功能,例如 吊舱安全政策 在整个名称空间上强制执行安全配置基准。以下必知的技巧将帮助您利用准入控制器充分利用Kubernetes中的这些安全功能。

什么是Kubernetes准入控制器?

简而言之,Kubernetes准入控制器是控制和强制使用集群的插件。可以将它们视为拦截(已认证)API请求的网守,并且可以更改请求对象或完全拒绝该请求。准入控制过程分为两个阶段: 变异 首先执行阶段,然后执行 证实 相。因此,准入控制器可以充当变异或验证控制器或两者的组合。例如, 极限危险 准入控制器可以使用默认资源请求和限制(更改阶段)来扩展Pod,并验证具有明确设置的资源要求的Pod不会超过 LimitRange对象 (验证阶段)。

录取控制者阶段

录取控制者阶段

It is worth noting that some aspects of Kubernetes’ operation that many users would consider built-in are in fact governed by 准入控制器. For example, when a namespace is deleted and subsequently enters the Terminating state, the NamespaceLifecycle 接纳控制器是防止在此名称空间中创建任何新对象的工具。

Among the more than 30 准入控制器 shipped with Kubernetes, two take a special role because of their nearly limitless flexibility - ValidatingAdmissionWebhooks and MutatingAdmissionWebhooks, both of which are in beta status as of Kubernetes 1.13. We will examine these two 准入控制器 closely, as they do not implement any policy decision logic themselves. Instead, the respective action is obtained from a REST endpoint (a 网络挂钩)在群集中运行的服务。这种方法将准入控制器逻辑与Kubernetes API服务器解耦,从而使用户能够实现在Kubernetes集群中创建,更新或删除资源时执行自定义逻辑。

The difference between the two kinds of admission controller 网络挂钩s is pretty much self-explanatory: 变异 admission 网络挂钩s may mutate the objects, while 证实 admission 网络挂钩s may not. However, even a 变异 admission 网络挂钩 can reject requests and thus act in a 证实 fashion. Validating admission 网络挂钩s have two main advantages over 变异 ones: first, for security reasons it might be desirable to disable the MutatingAdmissionWebhook admission controller (or apply stricter RBAC restrictions as to who may create MutatingWebhookConfiguration objects) because of its potentially confusing or even dangerous side effects. Second, as shown in the previous diagram, 证实 准入控制器 (and thus 网络挂钩s) are run after any 变异 ones. As a result, whatever request object a 证实 网络挂钩 sees is the final version that would be persisted to etcd.

通过将标志传递给Kubernetes API服务器来配置启用的准入控制器集。注意旧 -入场控制 标记在1.10中已弃用,并替换为 --enable-admission-plugins.

--enable-admission-plugins=ValidatingAdmissionWebhook,MutatingAdmissionWebhook

Kubernetes建议默认启用以下准入控制器。

--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,Priority,ResourceQuota,PodSecurityPolicy

可以找到准入控制器及其说明的完整列表 在Kubernetes官方参考中。该讨论将仅集中在基于Webhook的准入控制器上。

为什么需要入场控制者?

  • 安全: Admission controllers can increase security by mandating a reasonable security baseline across an entire namespace or cluster. The built-in PodSecurityPolicy admission controller is perhaps the most prominent example; it can be used for disallowing containers from running as root or making sure the container’s root filesystem is always mounted read-only, for example. Further use cases that can be realized by custom, 网络挂钩-based 准入控制器 include:
  • 仅允许从企业已知的特定注册表中提取图像,同时拒绝未知的图像注册表。
  • Reject deployments that do not meet security standards. For example, containers using the privileged flag can circumvent a lot of security checks. This risk could be mitigated by a 网络挂钩-based admission controller that either rejects such deployments (validating) or overrides the privileged flag, setting it to false.
  • 治理: 准入控制器允许您强制遵守某些惯例,例如具有良好的标签,注释,资源限制或其他设置。一些常见方案包括:
  • 对不同的对象强制执行标签验证,以确保将正确的标签用于各种对象,例如,分配给团队或项目的每个对象,或指定应用程序标签的每个部署。
  • 自动为对象添加注释,例如为“开发”部署资源分配正确的成本中心。
  • 配置管理: 准入控制器使您可以验证集群中运行的对象的配置,并防止任何明显的错误配置击中您的集群。准入控制器在检测和修复不带语义标签的映像时非常有用,例如:
  • 自动添加资源限制或验证资源限制,
  • 确保将合理的标签添加到吊舱,或者
  • ensuring image references used in production deployments are not using the latest tags, or tags with a -dev suffix.

这样,准入控制器和策略管理有助于确保应用程序在不断变化的控件范围内保持合规性。

示例:编写和部署准入控制器Webhook

To illustrate how admission controller 网络挂钩s can be leveraged to establish custom security policies, let’s consider an example that addresses one of the shortcomings of Kubernetes: a lot of its defaults are optimized for ease of use and reducing friction, sometimes at the expense of security. One of these settings is that containers are by default allowed to run as root (and, without further configuration and no USER directive in the Dockerfile, will also do so). Even though containers are isolated from the underlying host to a certain extent, running containers as root does increase the risk profile of your deployment— and should be avoided as one of many 安全最佳实践。的 最近暴露的runC漏洞 (CVE-2019-5736),例如,只有在容器以root身份运行时才能被利用。

您可以使用自定义变异准入控制器Webhook来应用更安全的默认值:除非明确要求,否则我们的Webhook将确保Pod以非root用户身份运行(如果未进行显式分配,我们将分配用户ID 1234)。请注意,此设置不会阻止您在群集中部署任何工作负载,包括合法需要以root用户身份运行的工作负载。它仅要求您在部署配置中显式启用此风险较高的操作模式,而对于所有其他工作负载,默认为非root用户模式。

完整的代码以及部署说明可在我们的随附文件中找到 GitHub资料库。在这里,我们将重点介绍有关Webhooks工作原理的一些更微妙的方面。

突变Webhook配置

A 变异 admission controller 网络挂钩 is defined by creating a MutatingWebhookConfiguration object in Kubernetes. In our example, we use the following configuration:

apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
  name: demo-webhook
webhooks:
  - name: 网络挂钩-server.webhook-demo.svc
    clientConfig:
      service:
        name: 网络挂钩-server
        namespace: 网络挂钩-demo
        path: "/mutate"
      caBundle: ${CA_PEM_B64}
    rules:
      - operations: [ "CREATE" ]
        apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods"]

This configuration defines a 网络挂钩 网络挂钩-server.webhook-demo.svc, and instructs the Kubernetes API server to consult the service 网络挂钩-server in namespace 网络挂钩-demo whenever a pod is created by making a HTTP POST request to the /mutate URL. For this configuration to work, several prerequisites have to be met.

Webhook REST API

Kubernetes API服务器使用JSON编码向给定的服务和URL路径发出HTTPS POST请求 AdmissionReview (with the Request field set) in the request body. The response should in turn be a JSON-encoded AdmissionReview, this time with the Response field set.

我们的演示存储库包含一个 功能 that takes care of the serialization/deserialization boilerplate code and allows you to focus on implementing the logic operating on Kubernetes API objects. In our example, the 功能 implementing the admission controller logic is called applySecurityDefaults, and an HTTPS server serving this 功能 under the /mutate URL can be set up as follows:

mux := http.NewServeMux()
mux.Handle("/mutate", admitFuncHandler(applySecurityDefaults))
server := &http.Server{
  Addr:    ":8443",
  Handler: mux,
}
log.Fatal(server.ListenAndServeTLS(certPath, keyPath))

请注意,要使服务器在没有提升特权的情况下运行,我们的HTTP服务器将在端口8443上进行侦听。Kubernetes不允许在webhook配置中指定端口。它始终采用HTTPS端口443。但是,由于仍然需要服务对象,因此我们可以轻松地将服务的端口443映射到容器上的端口8443:

apiVersion: v1
kind: Service
metadata:
  name: 网络挂钩-server
  namespace: 网络挂钩-demo
spec:
  selector:
    app: 网络挂钩-server  # specified by the deployment/pod
  ports:
    - port: 443
      targetPort: 网络挂钩-api  # name of port 8443 of the container

对象修改逻辑

在变异接纳控制器Webhook中,变异是通过 JSON补丁。尽管JSON修补程序标准包含许多复杂的内容,这些内容远远超出了本讨论的范围,但示例中的Go数据结构及其用法应为用户提供有关JSON修补程序工作原理的良好初步概览:

type patchOperation struct {
  Op    string      `json:"op"`
  Path  string      `json:"path"`
  Value interface{} `json:"value,omitempty"`
}

For setting the field .spec.securityContext.runAsNonRoot of a pod to true, we construct the following patchOperation object:

patches = append(patches, patchOperation{
  Op:    "add",
  Path:  "/spec/securityContext/runAsNonRoot",
  Value: true,
})

TLS证书

Since a 网络挂钩 must be served via HTTPS, we need proper certificates for the server. These certificates can be self-signed (rather: signed by a self-signed CA), but we need Kubernetes to instruct the respective CA certificate when talking to the 网络挂钩 server. In addition, the common name (CN) of the certificate must match the server name used by the Kubernetes API server, which for internal services is <service-name>.<namespace>.svc, i.e., 网络挂钩-server.webhook-demo.svc in our case. Since the generation of self-signed TLS certificates is well documented across the Internet, we simply refer to the respective 外壳脚本 在我们的例子中。

The 网络挂钩 configuration shown previously contains a placeholder ${CA_PEM_B64}. Before we can create this configuration, we need to replace this portion with the Base64-encoded PEM certificate of the CA. The openssl base64 -A command can be used for this purpose.

测试Webhook

在部署了Webhook服务器并对其进行配置之后(可以通过从存储库中调用./deploy.sh脚本来完成),现在该测试并验证Webhook确实能够完成其工作了。存储库包含 三个例子:

  • A pod that does not specify a security context (pod-with-defaults). We expect this pod to be run as non-root with user id 1234.
  • A pod that does specify a security context, explicitly allowing it to run as root (pod-with-override).
  • A pod with a conflicting configuration, specifying it must run as non-root but with a user id of 0 (pod-with-conflict). To showcase the rejection of object creation requests, we have augmented our admission controller logic to reject such obvious misconfigurations.

Create one of these pods by running kubectl create -f examples/<name>.yaml. In the first two examples, you can verify the user id under which the pod ran by inspecting the logs, for example:

$ kubectl create -f examples/pod-with-defaults.yaml
$ kubectl logs pod-with-defaults
I am running as user 1234

在第三个示例中,应使用适当的错误消息拒绝对象创建:

$ kubectl create -f examples/pod-with-conflict.yaml
Error from server (InternalError): error when creating "examples/pod-with-conflict.yaml": Internal error occurred: admission 网络挂钩 "webhook-server.webhook-demo.svc" denied the request: runAsNonRoot specified, but runAsUser set to 0 (the root user)

也可以使用自己的工作负载进行测试。当然,您还可以通过更改Webhook的逻辑进行进一步的试验,并查看更改如何影响对象的创建。有关如何进行此类更改的实验的更多信息,请参见 仓库的自述文件.

概要

Kubernetes准入控制器在安全性方面具有明显优势。深入研究两个强大的示例以及随附的可用代码,将帮助您开始利用这些强大的功能。

参考文献: