centos8 + kubernetes 1.24 master/node 节点

目标

使用 centos8 安装 kubernetes 1.24 版本
注意， kubernetes 1.24 已经不再支持 dockerd

软件版本

所需镜像

所有镜像都需要存放在 k8s.io namespace 中

安装过程

系统包

yum -y install yum-utils device-mapper-persistent-data lvm2 libtool-ltdl libcgroup langpacks-en glibc-all-langpacks tc

contraind

yum install -y containerd.io.x86_64

contrainer 模块

echo -e "overlay\nbr_netfilter" > /etc/modules-load.d/containerd.conf
modprobe overlay
modprobe br_netfilter

修改 contrainer 配置

mkdir -p /etc/containerd
containerd config default | tee /etc/containerd/config.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml

如果需要在 contrainerd 中使用到 proxy

vim /usr/lib/systemd/system/containerd.service

[Service]
Environment="HTTP_PROXY=http://proxy_ip:port"
Environment="HTTPS_PROXY=http://proxy_ip:port"
Environment="NO_PROXY=.svr,127.0.0.1,节点1，节点2"

添加内核模块

echo -e  "net.bridge.bridge-nf-call-iptables = 1\nnet.ipv4.ip_forward = 1\nnet.bridge.bridge-nf-call-ip6tables = 1" > /etc/sysctl.d/99-kubernetes-cri.conf
sysctl --system

selinux 关闭

setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

安装 kubelet

yum install -y kubelet kubeadm kubectl

镜像管理工具

https://github.com/kubernetes-sigs/cri-tools/releases/tag/v1.26.0

配置

cat /etc/crictl.yaml
runtime-endpoint: unix:///var/run/containerd/containerd.sock  <- 使用 contrainred 作为 cri 
image-endpoint: unix:///var/run/containerd/containerd.sock    <- 使用 contrainerd 进行惊险管理
timeout: 2
debug: false
pull-image-on-create: false
disable-pull-on-run: false

master 配置

swapoff

sed -i /swap/d  /etc/fstab
swapoff -a

检查需要的镜像版本

# kubeadm config images list
I0206 17:09:57.566434   40157 version.go:255] remote version is much newer: v1.26.1; falling back to: stable-1.24
k8s.gcr.io/kube-apiserver:v1.24.10
k8s.gcr.io/kube-controller-manager:v1.24.10
k8s.gcr.io/kube-scheduler:v1.24.10
k8s.gcr.io/kube-proxy:v1.24.10
k8s.gcr.io/pause:3.7
k8s.gcr.io/etcd:3.5.5-0
k8s.gcr.io/coredns/coredns:v1.8.6

安装详细日志过程

# kubeadm init  --pod-network-cidr=10.244.0.0/16  --v=9
I0207 08:21:56.379975   46264 initconfiguration.go:117] detected and using CRI socket: unix:///var/run/containerd/containerd.sock
I0207 08:21:56.380241   46264 interface.go:432] Looking for default routes with IPv4 addresses
I0207 08:21:56.380274   46264 interface.go:437] Default route transits interface "bond0.20"
I0207 08:21:56.380733   46264 interface.go:209] Interface bond0.20 is up
I0207 08:21:56.380793   46264 interface.go:257] Interface "bond0.20" has 2 addresses :[10.189.20.64/24 fe80::ee38:8fff:fe79:26fe/64].
I0207 08:21:56.380829   46264 interface.go:224] Checking addr  10.189.20.64/24.
I0207 08:21:56.380843   46264 interface.go:231] IP found 10.189.20.64
I0207 08:21:56.380854   46264 interface.go:263] Found valid IPv4 address 10.189.20.64 for interface "bond0.20".
I0207 08:21:56.380865   46264 interface.go:443] Found active IP 10.189.20.64
I0207 08:21:56.380900   46264 kubelet.go:218] the value of KubeletConfiguration.cgroupDriver is empty; setting it to "systemd"
I0207 08:21:56.391103   46264 version.go:186] fetching Kubernetes version from URL: https://dl.k8s.io/release/stable-1.txt
W0207 08:21:56.624125   46264 version.go:103] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get "https://dl.k8s.io/release/stable-1.txt": x509: certificate signed by unknown authority
W0207 08:21:56.624154   46264 version.go:104] falling back to the local client version: v1.24.8
[init] Using Kubernetes version: v1.24.8
[preflight] Running pre-flight checks
I0207 08:21:56.624441   46264 checks.go:570] validating Kubernetes and kubeadm version
I0207 08:21:56.624468   46264 checks.go:170] validating if the firewall is enabled and active
I0207 08:21:56.645248   46264 checks.go:205] validating availability of port 6443
I0207 08:21:56.645424   46264 checks.go:205] validating availability of port 10259
I0207 08:21:56.645472   46264 checks.go:205] validating availability of port 10257
I0207 08:21:56.645511   46264 checks.go:282] validating the existence of file /etc/kubernetes/manifests/kube-apiserver.yaml
I0207 08:21:56.645539   46264 checks.go:282] validating the existence of file /etc/kubernetes/manifests/kube-controller-manager.yaml
I0207 08:21:56.645555   46264 checks.go:282] validating the existence of file /etc/kubernetes/manifests/kube-scheduler.yaml
I0207 08:21:56.645569   46264 checks.go:282] validating the existence of file /etc/kubernetes/manifests/etcd.yaml
I0207 08:21:56.645583   46264 checks.go:432] validating if the connectivity type is via proxy or direct
I0207 08:21:56.645605   46264 checks.go:471] validating http connectivity to first IP address in the CIDR
I0207 08:21:56.645627   46264 checks.go:471] validating http connectivity to first IP address in the CIDR
I0207 08:21:56.645654   46264 checks.go:106] validating the container runtime
I0207 08:21:56.673959   46264 checks.go:331] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I0207 08:21:56.674021   46264 checks.go:331] validating the contents of file /proc/sys/net/ipv4/ip_forward
I0207 08:21:56.674052   46264 checks.go:646] validating whether swap is enabled or not
I0207 08:21:56.674088   46264 checks.go:372] validating the presence of executable crictl
I0207 08:21:56.674113   46264 checks.go:372] validating the presence of executable conntrack
I0207 08:21:56.674130   46264 checks.go:372] validating the presence of executable ip
I0207 08:21:56.674146   46264 checks.go:372] validating the presence of executable iptables
I0207 08:21:56.674163   46264 checks.go:372] validating the presence of executable mount
I0207 08:21:56.674180   46264 checks.go:372] validating the presence of executable nsenter
I0207 08:21:56.674198   46264 checks.go:372] validating the presence of executable ebtables
I0207 08:21:56.674214   46264 checks.go:372] validating the presence of executable ethtool
I0207 08:21:56.674230   46264 checks.go:372] validating the presence of executable socat
I0207 08:21:56.674247   46264 checks.go:372] validating the presence of executable tc
I0207 08:21:56.674274   46264 checks.go:372] validating the presence of executable touch
I0207 08:21:56.674299   46264 checks.go:518] running all checks
I0207 08:21:56.681808   46264 checks.go:403] checking whether the given node name is valid and reachable using net.LookupHost
I0207 08:21:56.681838   46264 checks.go:612] validating kubelet version
I0207 08:21:56.737501   46264 checks.go:132] validating if the "kubelet" service is enabled and active
I0207 08:21:56.754844   46264 checks.go:205] validating availability of port 10250
I0207 08:21:56.754937   46264 checks.go:205] validating availability of port 2379
I0207 08:21:56.754981   46264 checks.go:205] validating availability of port 2380
I0207 08:21:56.755023   46264 checks.go:245] validating the existence and emptiness of directory /var/lib/etcd
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0207 08:21:56.755130   46264 checks.go:834] using image pull policy: IfNotPresent
I0207 08:21:56.782170   46264 checks.go:843] image exists: k8s.gcr.io/kube-apiserver:v1.24.8
I0207 08:21:56.808358   46264 checks.go:843] image exists: k8s.gcr.io/kube-controller-manager:v1.24.8
I0207 08:21:56.834233   46264 checks.go:843] image exists: k8s.gcr.io/kube-scheduler:v1.24.8
I0207 08:21:56.859979   46264 checks.go:843] image exists: k8s.gcr.io/kube-proxy:v1.24.8
I0207 08:21:56.885745   46264 checks.go:843] image exists: k8s.gcr.io/pause:3.7
I0207 08:21:56.911946   46264 checks.go:843] image exists: k8s.gcr.io/etcd:3.5.5-0
I0207 08:21:56.937651   46264 checks.go:843] image exists: k8s.gcr.io/coredns/coredns:v1.8.6
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I0207 08:21:56.937724   46264 certs.go:112] creating a new certificate authority for ca
[certs] Generating "ca" certificate and key
I0207 08:21:57.071752   46264 certs.go:522] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local ns-yun-020064.vclound.com] and IPs [10.96.0.1 10.189.20.64]
[certs] Generating "apiserver-kubelet-client" certificate and key
I0207 08:21:57.659884   46264 certs.go:112] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I0207 08:21:57.794444   46264 certs.go:522] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I0207 08:21:57.965192   46264 certs.go:112] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I0207 08:21:58.086396   46264 certs.go:522] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost ns-yun-020064.vclound.com] and IPs [10.189.20.64 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost ns-yun-020064.vclound.com] and IPs [10.189.20.64 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I0207 08:21:59.067506   46264 certs.go:78] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0207 08:21:59.328104   46264 kubeconfig.go:103] creating kubeconfig file for admin.conf
[kubeconfig] Writing "admin.conf" kubeconfig file
I0207 08:21:59.455912   46264 kubeconfig.go:103] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I0207 08:21:59.992365   46264 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0207 08:22:00.424942   46264 kubeconfig.go:103] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
I0207 08:22:00.646836   46264 kubelet.go:65] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0207 08:22:00.859398   46264 manifests.go:99] [control-plane] getting StaticPodSpecs
I0207 08:22:00.859759   46264 certs.go:522] validating certificate period for CA certificate
I0207 08:22:00.859872   46264 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0207 08:22:00.859889   46264 manifests.go:125] [control-plane] adding volume "etc-pki" for component "kube-apiserver"
I0207 08:22:00.859903   46264 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0207 08:22:00.863969   46264 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0207 08:22:00.863995   46264 manifests.go:99] [control-plane] getting StaticPodSpecs
I0207 08:22:00.864249   46264 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0207 08:22:00.864275   46264 manifests.go:125] [control-plane] adding volume "etc-pki" for component "kube-controller-manager"
I0207 08:22:00.864286   46264 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0207 08:22:00.864295   46264 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0207 08:22:00.864305   46264 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0207 08:22:00.865129   46264 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0207 08:22:00.865149   46264 manifests.go:99] [control-plane] getting StaticPodSpecs
I0207 08:22:00.865393   46264 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I0207 08:22:00.865952   46264 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0207 08:22:00.866794   46264 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0207 08:22:00.866811   46264 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy
I0207 08:22:00.867582   46264 loader.go:372] Config loaded from file:  /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0207 08:22:00.868334   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:00.868540   46264 round_trippers.go:508] HTTP Trace: Dial to tcp:10.189.20.64:6443 failed: dial tcp 10.189.20.64:6443: connect: connection refused
I0207 08:22:00.868568   46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s  in 0 milliseconds
I0207 08:22:00.868582   46264 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 0 ms Duration 0 ms
I0207 08:22:00.868594   46264 round_trippers.go:577] Response Headers:
I0207 08:22:01.369456   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:01.369753   46264 round_trippers.go:508] HTTP Trace: Dial to tcp:10.189.20.64:6443 failed: dial tcp 10.189.20.64:6443: connect: connection refused
I0207 08:22:01.369794   46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s  in 0 milliseconds
I0207 08:22:01.369813   46264 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 0 ms Duration 0 ms
I0207 08:22:01.369834   46264 round_trippers.go:577] Response Headers:
I0207 08:22:01.869586   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:01.869932   46264 round_trippers.go:508] HTTP Trace: Dial to tcp:10.189.20.64:6443 failed: dial tcp 10.189.20.64:6443: connect: connection refused
I0207 08:22:01.869970   46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s  in 0 milliseconds
I0207 08:22:01.869994   46264 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 0 ms Duration 0 ms
I0207 08:22:01.870010   46264 round_trippers.go:577] Response Headers:
I0207 08:22:02.369678   46264 round_trippers.go:466] curl -v -XGET  -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:02.370100   46264 round_trippers.go:508] HTTP Trace: Dial to tcp:10.189.20.64:6443 failed: dial tcp 10.189.20.64:6443: connect: connection refused
I0207 08:22:02.370142   46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s  in 0 milliseconds
I0207 08:22:02.370168   46264 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 0 ms Duration 0 ms
I0207 08:22:02.370188   46264 round_trippers.go:577] Response Headers:
I0207 08:22:02.869837   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:02.870127   46264 round_trippers.go:510] HTTP Trace: Dial to tcp:10.189.20.64:6443 succeed
I0207 08:22:06.099994   46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s 500 Internal Server Error in 3230 milliseconds
I0207 08:22:06.100044   46264 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 3225 ms ServerProcessing 3 ms Duration 3230 ms
I0207 08:22:06.100062   46264 round_trippers.go:577] Response Headers:
I0207 08:22:06.100080   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid:
I0207 08:22:06.100096   46264 round_trippers.go:580]     Content-Length: 1263
I0207 08:22:06.100112   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:06 GMT
I0207 08:22:06.100127   46264 round_trippers.go:580]     Audit-Id: 70822272-9f2a-4c44-97c9-6b816800ee08
I0207 08:22:06.100143   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:06.100159   46264 round_trippers.go:580]     Content-Type: text/plain; charset=utf-8
I0207 08:22:06.100174   46264 round_trippers.go:580]     X-Content-Type-Options: nosniff
I0207 08:22:06.100189   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid:
I0207 08:22:06.100255   46264 request.go:1154] Response Body: [+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/start-apiextensions-informers ok
[-]poststarthook/start-apiextensions-controllers failed: reason withheld
[-]poststarthook/crd-informer-synced failed: reason withheld
[-]poststarthook/bootstrap-controller failed: reason withheld
[-]poststarthook/rbac/bootstrap-roles failed: reason withheld
[-]poststarthook/scheduling/bootstrap-system-priority-classes failed: reason withheld
[-]poststarthook/priority-and-fairness-config-producer failed: reason withheld
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[-]poststarthook/apiservice-registration-controller failed: reason withheld
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[-]autoregister-completion failed: reason withheld
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0207 08:22:06.368953   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:06.370374   46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s 500 Internal Server Error in 1 milliseconds
I0207 08:22:06.370396   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:06.370414   46264 round_trippers.go:577] Response Headers:
I0207 08:22:06.370427   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:06.370440   46264 round_trippers.go:580]     Content-Type: text/plain; charset=utf-8
I0207 08:22:06.370450   46264 round_trippers.go:580]     X-Content-Type-Options: nosniff
I0207 08:22:06.370461   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:06.370471   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:06.370483   46264 round_trippers.go:580]     Content-Length: 1137
I0207 08:22:06.370493   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:06 GMT
I0207 08:22:06.370504   46264 round_trippers.go:580]     Audit-Id: c360ea9a-d67c-4017-8327-f1951446880b
I0207 08:22:06.370542   46264 request.go:1154] Response Body: [+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/bootstrap-controller ok
[-]poststarthook/rbac/bootstrap-roles failed: reason withheld
[-]poststarthook/scheduling/bootstrap-system-priority-classes failed: reason withheld
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0207 08:22:06.869063   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:06.870317   46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s 500 Internal Server Error in 1 milliseconds
I0207 08:22:06.870339   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:06.870352   46264 round_trippers.go:577] Response Headers:
I0207 08:22:06.870364   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:06.870376   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:06.870387   46264 round_trippers.go:580]     Content-Length: 1137
I0207 08:22:06.870397   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:06 GMT
I0207 08:22:06.870408   46264 round_trippers.go:580]     Audit-Id: 8b0409a8-1fd4-4247-ad17-a8ee0b50828c
I0207 08:22:06.870419   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:06.870429   46264 round_trippers.go:580]     Content-Type: text/plain; charset=utf-8
I0207 08:22:06.870440   46264 round_trippers.go:580]     X-Content-Type-Options: nosniff
I0207 08:22:06.870468   46264 request.go:1154] Response Body: [+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/bootstrap-controller ok
[-]poststarthook/rbac/bootstrap-roles failed: reason withheld
[-]poststarthook/scheduling/bootstrap-system-priority-classes failed: reason withheld
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0207 08:22:07.369063   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:07.370350   46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s 500 Internal Server Error in 1 milliseconds
I0207 08:22:07.370377   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.370389   46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.370402   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:07.370413   46264 round_trippers.go:580]     Content-Type: text/plain; charset=utf-8
I0207 08:22:07.370423   46264 round_trippers.go:580]     X-Content-Type-Options: nosniff
I0207 08:22:07.370434   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.370445   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.370456   46264 round_trippers.go:580]     Content-Length: 1116
I0207 08:22:07.370466   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.370477   46264 round_trippers.go:580]     Audit-Id: f2718e1e-acd1-4896-8b48-cf23392703e1
I0207 08:22:07.370514   46264 request.go:1154] Response Body: [+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/bootstrap-controller ok
[-]poststarthook/rbac/bootstrap-roles failed: reason withheld
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0207 08:22:07.869107   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:07.870718   46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s 200 OK in 1 milliseconds
I0207 08:22:07.870740   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.870752   46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.870765   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:07.870776   46264 round_trippers.go:580]     Content-Type: text/plain; charset=utf-8
I0207 08:22:07.870787   46264 round_trippers.go:580]     X-Content-Type-Options: nosniff
I0207 08:22:07.870797   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.870808   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.870818   46264 round_trippers.go:580]     Content-Length: 2
I0207 08:22:07.870829   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.870840   46264 round_trippers.go:580]     Audit-Id: bc001798-04ca-4a38-8a77-8e36c065f40d
I0207 08:22:07.870885   46264 request.go:1154] Response Body: ok
[apiclient] All control plane components are healthy after 7.002728 seconds
I0207 08:22:07.870954   46264 uploadconfig.go:110] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0207 08:22:07.871630   46264 request.go:1154] Request Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubeadm-config","namespace":"kube-system","creationTimestamp":null},"data":{"ClusterConfiguration":"apiServer:\n  extraArgs:\n    authorization-mode: Node,RBAC\n  timeoutForControlPlane: 4m0s\napiVersion: kubeadm.k8s.io/v1beta3\ncertificatesDir: /etc/kubernetes/pki\nclusterName: kubernetes\ncontrollerManager: {}\ndns: {}\netcd:\n  local:\n    dataDir: /var/lib/etcd\nimageRepository: k8s.gcr.io\nkind: ClusterConfiguration\nkubernetesVersion: v1.24.8\nnetworking:\n  dnsDomain: cluster.local\n  podSubnet: 10.244.0.0/16\n  serviceSubnet: 10.96.0.0/12\nscheduler: {}\n"}}
I0207 08:22:07.871703   46264 round_trippers.go:466] curl -v -XPOST  -H "Content-Type: application/json" -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s'
I0207 08:22:07.873785   46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 2 milliseconds
I0207 08:22:07.873807   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 2 ms
I0207 08:22:07.873823   46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.873835   46264 round_trippers.go:580]     Audit-Id: fc5476cb-f980-4faa-bb09-28fd8c5298cd
I0207 08:22:07.873846   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:07.873857   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:07.873867   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.873878   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.873889   46264 round_trippers.go:580]     Content-Length: 910
I0207 08:22:07.873899   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.873928   46264 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubeadm-config","namespace":"kube-system","uid":"ea94739b-1e8c-4a2f-ad2b-0b9c1769816d","resourceVersion":"204","creationTimestamp":"2023-02-07T00:22:07Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:ClusterConfiguration":{}}}}]},"data":{"ClusterConfiguration":"apiServer:\n  extraArgs:\n    authorization-mode: Node,RBAC\n  timeoutForControlPlane: 4m0s\napiVersion: kubeadm.k8s.io/v1beta3\ncertificatesDir: /etc/kubernetes/pki\nclusterName: kubernetes\ncontrollerManager: {}\ndns: {}\netcd:\n  local:\n    dataDir: /var/lib/etcd\nimageRepository: k8s.gcr.io\nkind: ClusterConfiguration\nkubernetesVersion: v1.24.8\nnetworking:\n  dnsDomain: cluster.local\n  podSubnet: 10.244.0.0/16\n  serviceSubnet: 10.96.0.0/12\nscheduler: {}\n"}}
I0207 08:22:07.874313   46264 request.go:1154] Request Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:nodes-kubeadm-config","namespace":"kube-system","creationTimestamp":null},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["kubeadm-config"]}]}
I0207 08:22:07.874385   46264 round_trippers.go:466] curl -v -XPOST  -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" -H "Content-Type: application/json" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s'
I0207 08:22:07.876285   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:07.876305   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.876316   46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.876327   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:07.876338   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.876349   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.876360   46264 round_trippers.go:580]     Content-Length: 541
I0207 08:22:07.876370   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.876381   46264 round_trippers.go:580]     Audit-Id: 6756eb4f-54d6-4045-8d90-a0611df7d9ad
I0207 08:22:07.876391   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:07.876413   46264 request.go:1154] Response Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:nodes-kubeadm-config","namespace":"kube-system","uid":"0956bc31-f01d-43b1-af7c-0d94e93a6da9","resourceVersion":"205","creationTimestamp":"2023-02-07T00:22:07Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:rules":{}}}]},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["kubeadm-config"]}]}
I0207 08:22:07.876649   46264 request.go:1154] Request Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:nodes-kubeadm-config","namespace":"kube-system","creationTimestamp":null},"subjects":[{"kind":"Group","name":"system:bootstrappers:kubeadm:default-node-token"},{"kind":"Group","name":"system:nodes"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kubeadm:nodes-kubeadm-config"}}
I0207 08:22:07.876715   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s'
I0207 08:22:07.878732   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:07.878753   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.878765   46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.878777   46264 round_trippers.go:580]     Audit-Id: 6eeda9a6-693d-45a1-9095-1295d6a2794b
I0207 08:22:07.878788   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:07.878798   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:07.878810   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.878821   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.878832   46264 round_trippers.go:580]     Content-Length: 766
I0207 08:22:07.878843   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.878869   46264 request.go:1154] Response Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:nodes-kubeadm-config","namespace":"kube-system","uid":"c4cb7f9b-0235-405b-b521-d318a4012638","resourceVersion":"206","creationTimestamp":"2023-02-07T00:22:07Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:bootstrappers:kubeadm:default-node-token"},{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:nodes"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kubeadm:nodes-kubeadm-config"}}
I0207 08:22:07.879035   46264 uploadconfig.go:124] [upload-config] Uploading the kubelet component config to a ConfigMap
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0207 08:22:07.879585   46264 request.go:1154] Request Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubelet-config","namespace":"kube-system","creationTimestamp":null,"annotations":{"kubeadm.kubernetes.io/component-config.hash":"sha256:53821686a8ddaca8403df63803a726d2ab31b145c7da69ed7b78c45d2e223a18"}},"data":{"kubelet":"apiVersion: kubelet.config.k8s.io/v1beta1\nauthentication:\n  anonymous:\n    enabled: false\n  webhook:\n    cacheTTL: 0s\n    enabled: true\n  x509:\n    clientCAFile: /etc/kubernetes/pki/ca.crt\nauthorization:\n  mode: Webhook\n  webhook:\n    cacheAuthorizedTTL: 0s\n    cacheUnauthorizedTTL: 0s\ncgroupDriver: systemd\nclusterDNS:\n- 10.96.0.10\nclusterDomain: cluster.local\ncpuManagerReconcilePeriod: 0s\nevictionPressureTransitionPeriod: 0s\nfileCheckFrequency: 0s\nhealthzBindAddress: 127.0.0.1\nhealthzPort: 10248\nhttpCheckFrequency: 0s\nimageMinimumGCAge: 0s\nkind: KubeletConfiguration\nlogging:\n  flushFrequency: 0\n  options:\n    json:\n      infoBufferSize: \"0\"\n  verbosity: 0\nmemorySwap: {}\nnodeStatusReportFrequency: 0s\nnodeStatusUpdateFrequency: 0s\nrotateCertificates: true\nruntimeRequestTimeout: 0s\nshutdownGracePeriod: 0s\nshutdownGracePeriodCriticalPods: 0s\nstaticPodPath: /etc/kubernetes/manifests\nstreamingConnectionIdleTimeout: 0s\nsyncFrequency: 0s\nvolumeStatsAggPeriod: 0s\n"}}
I0207 08:22:07.879657   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s'
I0207 08:22:07.881675   46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:07.881699   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.881711   46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.881722   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.881734   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.881744   46264 round_trippers.go:580]     Content-Length: 1657
I0207 08:22:07.881755   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.881765   46264 round_trippers.go:580]     Audit-Id: 96f8fa2c-8899-4cb8-8c27-2719e5f3cc0b
I0207 08:22:07.881776   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:07.881786   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:07.881814   46264 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubelet-config","namespace":"kube-system","uid":"c6277399-bbe5-4b2d-ad78-94a120465f7e","resourceVersion":"207","creationTimestamp":"2023-02-07T00:22:07Z","annotations":{"kubeadm.kubernetes.io/component-config.hash":"sha256:53821686a8ddaca8403df63803a726d2ab31b145c7da69ed7b78c45d2e223a18"},"managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:kubelet":{}},"f:metadata":{"f:annotations":{".":{},"f:kubeadm.kubernetes.io/component-config.hash":{}}}}}]},"data":{"kubelet":"apiVersion: kubelet.config.k8s.io/v1beta1\nauthentication:\n  anonymous:\n    enabled: false\n  webhook:\n    cacheTTL: 0s\n    enabled: true\n  x509:\n    clientCAFile: /etc/kubernetes/pki/ca.crt\nauthorization:\n  mode: Webhook\n  webhook:\n    cacheAuthorizedTTL: 0s\n    cacheUnauthorizedTTL: 0s\ncgroupDriver: systemd\nclusterDNS:\n- 10.96.0.10\nclusterDomain: cluster.local\ncpuManagerReconcilePeriod: 0s\nevictionPressureTransitionPeriod: 0s\nfileCheckFrequency: 0s\nhealthzBindAddress: 127.0.0.1\nhealthzPort: 10248\nhttpCheckFrequency: 0s\nimageMinimumGCAge: 0s\nkind: KubeletConfiguration\nlogging:\n  flushFrequency: 0\n  options:\n    json:\n      infoBufferSize: \"0\"\n  verbosity: 0\nmemorySwap: {}\nnodeStatusReportFrequency: 0s\nnodeStatusUpdateFrequency: 0s\nrotateCertificates: true\nruntimeRequestTimeout: 0s\nshutdownGracePeriod: 0s\nshutdownGracePeriodCriticalPods: 0s\nstaticPodPath: /etc/kubernetes/manifests\nstreamingConnectionIdleTimeout: 0s\nsyncFrequency: 0s\nvolumeStatsAggPeriod: 0s\n"}}
I0207 08:22:07.881985   46264 request.go:1154] Request Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:kubelet-config","namespace":"kube-system","creationTimestamp":null},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["kubelet-config"]}]}
I0207 08:22:07.882061   46264 round_trippers.go:466] curl -v -XPOST  -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s'
I0207 08:22:07.883908   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:07.883930   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.883941   46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.883953   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.883964   46264 round_trippers.go:580]     Audit-Id: 29f655fb-97ca-41fa-9987-59bf5f73fc0b
I0207 08:22:07.883974   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:07.883985   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:07.883995   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.884006   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.884017   46264 round_trippers.go:580]     Content-Length: 535
I0207 08:22:07.884038   46264 request.go:1154] Response Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:kubelet-config","namespace":"kube-system","uid":"eb145f80-3a93-4f2c-aa45-6824597f4757","resourceVersion":"208","creationTimestamp":"2023-02-07T00:22:07Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:rules":{}}}]},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["kubelet-config"]}]}
I0207 08:22:07.884161   46264 request.go:1154] Request Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:kubelet-config","namespace":"kube-system","creationTimestamp":null},"subjects":[{"kind":"Group","name":"system:nodes"},{"kind":"Group","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kubeadm:kubelet-config"}}
I0207 08:22:07.884227   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s'
I0207 08:22:07.886011   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:07.886032   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.886044   46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.886056   46264 round_trippers.go:580]     Audit-Id: 81c58bd1-2cc8-4658-bc14-a3f4b1fa2ff2
I0207 08:22:07.886067   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:07.886077   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:07.886088   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.886099   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.886109   46264 round_trippers.go:580]     Content-Length: 754
I0207 08:22:07.886120   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.886161   46264 request.go:1154] Response Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:kubelet-config","namespace":"kube-system","uid":"97be87e9-37a6-44c4-b137-6d529ce4aef2","resourceVersion":"209","creationTimestamp":"2023-02-07T00:22:07Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:nodes"},{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kubeadm:kubelet-config"}}
I0207 08:22:07.886285   46264 uploadconfig.go:129] [upload-config] Preserving the CRISocket information for the control-plane node
I0207 08:22:07.886302   46264 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///var/run/containerd/containerd.sock" to the Node API object "ns-yun-020064.vclound.com" as an annotation
I0207 08:22:08.387124   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020064.vclound.com?timeout=10s'
I0207 08:22:08.389204   46264 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020064.vclound.com?timeout=10s 200 OK in 2 milliseconds
I0207 08:22:08.389226   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 2 ms
I0207 08:22:08.389238   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.389250   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.389297   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.389317   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.389334   46264 round_trippers.go:580]     Content-Length: 4046
I0207 08:22:08.389354   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.389370   46264 round_trippers.go:580]     Audit-Id: 8cf06c9f-dd99-4bbd-ba0f-3cda64fdacfb
I0207 08:22:08.389388   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.389438   46264 request.go:1154] Response Body: {"kind":"Node","apiVersion":"v1","metadata":{"name":"ns-yun-020064.vclound.com","uid":"a6575610-3c4c-468b-8511-a5d23196927c","resourceVersion":"56","creationTimestamp":"2023-02-07T00:22:06Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"ns-yun-020064.vclound.com","kubernetes.io/os":"linux"},"annotations":{"volumes.kubernetes.io/controller-managed-attach-detach":"true"},"managedFields":[{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:06Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:06Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{}}}}},"subresource":"status"}]},"spec":{"taints":[{"key":"node.kubernetes.io/not-ready","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"24","ephemeral-storage":"314107036Ki","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131850144Ki","pods":"110"},"allocatable":{"cpu":"24","ephemeral-storage":"289481043899","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131747744Ki","pods":"110"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletNotReady","message":"[container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized, CSINode is not yet initialized]"}],"addresses":[{"type":"InternalIP","address":"10.189.20.64"},{"type":"Hostname","address":"ns-yun-020064.vclound.com"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"3889867cda874b7f8bba55f6137923a8","systemUUID":"20d821d7-e8a2-e511-8a39-0027798f38ec","bootID":"750034a5-560d-4a66-9d9e-198aed158f5f","kernelVersion":"4.18.0-305.3.1.el8.x86_64","osImage":"CentOS Linux 8","containerRuntimeVersion":"containerd://1.6.15","kubeletVersion":"v1.24.8","kubeProxyVersion":"v1.24.8","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["k8s.gcr.io/etcd:3.5.5-0"],"sizeBytes":301522620},{"names":["k8s.gcr.io/kube-apiserver:v1.24.10"],"sizeBytes":131733971},{"names":["k8s.gcr.io/kube-apiserver:v1.24.8"],"sizeBytes":131113427},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.10"],"sizeBytes":121342265},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.8"],"sizeBytes":120746809},{"names":["k8s.gcr.io/kube-proxy:v1.24.10"],"sizeBytes":112211511},{"names":["k8s.gcr.io/kube-proxy:v1.24.8"],"sizeBytes":111867447},{"names":["k8s.gcr.io/kube-scheduler:v1.24.10"],"sizeBytes":52751160},{"names":["k8s.gcr.io/kube-scheduler:v1.24.8"],"sizeBytes":52339512},{"names":["k8s.gcr.io/coredns/coredns:v1.8.6","k8s.gcr.io/coredns:v1.8.6"],"sizeBytes":46957023},{"names":["k8s.gcr.io/pause:3.7"],"sizeBytes":714605},{"names":["registry.k8s.io/pause:3.6"],"sizeBytes":685866}]}}
I0207 08:22:08.391975   46264 request.go:1154] Request Body: {"metadata":{"annotations":{"kubeadm.alpha.kubernetes.io/cri-socket":"unix:///var/run/containerd/containerd.sock"}}}
I0207 08:22:08.392047   46264 round_trippers.go:466] curl -v -XPATCH  -H "Accept: application/json, */*" -H "Content-Type: application/strategic-merge-patch+json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020064.vclound.com?timeout=10s'
I0207 08:22:08.395890   46264 round_trippers.go:553] PATCH https://10.189.20.64:6443/api/v1/nodes/ns-yun-020064.vclound.com?timeout=10s 200 OK in 3 milliseconds
I0207 08:22:08.395914   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 3 ms Duration 3 ms
I0207 08:22:08.395925   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.395938   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.395949   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.395960   46264 round_trippers.go:580]     Audit-Id: 4f153554-54b8-4f35-ad57-18049f53b776
I0207 08:22:08.395972   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.395982   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.395991   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.396063   46264 request.go:1154] Response Body: {"kind":"Node","apiVersion":"v1","metadata":{"name":"ns-yun-020064.vclound.com","uid":"a6575610-3c4c-468b-8511-a5d23196927c","resourceVersion":"211","creationTimestamp":"2023-02-07T00:22:06Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"ns-yun-020064.vclound.com","kubernetes.io/os":"linux"},"annotations":{"kubeadm.alpha.kubernetes.io/cri-socket":"unix:///var/run/containerd/containerd.sock","volumes.kubernetes.io/controller-managed-attach-detach":"true"},"managedFields":[{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:06Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:06Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{}}}}},"subresource":"status"},{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}}]},"spec":{"taints":[{"key":"node.kubernetes.io/not-ready","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"24","ephemeral-storage":"314107036Ki","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131850144Ki","pods":"110"},"allocatable":{"cpu":"24","ephemeral-storage":"289481043899","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131747744Ki","pods":"110"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletNotReady","message":"[container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized, CSINode is not yet initialized]"}],"addresses":[{"type":"InternalIP","address":"10.189.20.64"},{"type":"Hostname","address":"ns-yun-020064.vclound.com"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"3889867cda874b7f8bba55f6137923a8","systemUUID":"20d821d7-e8a2-e511-8a39-0027798f38ec","bootID":"750034a5-560d-4a66-9d9e-198aed158f5f","kernelVersion":"4.18.0-305.3.1.el8.x86_64","osImage":"CentOS Linux 8","containerRuntimeVersion":"containerd://1.6.15","kubeletVersion":"v1.24.8","kubeProxyVersion":"v1.24.8","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["k8s.gcr.io/etcd:3.5.5-0"],"sizeBytes":301522620},{"names":["k8s.gcr.io/kube-apiserver:v1.24.10"],"sizeBytes":131733971},{"names":["k8s.gcr.io/kube-apiserver:v1.24.8"],"sizeBytes":131113427},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.10"],"sizeBytes":121342265},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.8"],"sizeBytes":120746809},{"names":["k8s.gcr.io/kube-proxy:v1.24.10"],"sizeBytes":112211511},{"names":["k8s.gcr.io/kube-proxy:v1.24.8"],"sizeBytes":111867447},{"names":["k8s.gcr.io/kube-scheduler:v1.24.10"],"sizeBytes":52751160},{"names":["k8s.gcr.io/kube-scheduler:v1.24.8"],"sizeBytes":52339512},{"names":["k8s.gcr.io/coredns/coredns:v1.8.6","k8s.gcr.io/coredns:v1.8.6"],"sizeBytes":46957023},{"names":["k8s.gcr.io/pause:3.7"],"sizeBytes":714605},{"names":["registry.k8s.io/pause:3.6"],"sizeBytes":685866}]}}
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node ns-yun-020064.vclound.com as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node ns-yun-020064.vclound.com as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]
I0207 08:22:08.896813   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020064.vclound.com?timeout=10s'
I0207 08:22:08.898726   46264 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020064.vclound.com?timeout=10s 200 OK in 1 milliseconds
I0207 08:22:08.898757   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:08.898774   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.898795   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.898810   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.898823   46264 round_trippers.go:580]     Audit-Id: a9237724-f668-47bb-9be7-7cc7f28d60b0
I0207 08:22:08.898834   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.898844   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.898855   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.898943   46264 request.go:1154] Response Body: {"kind":"Node","apiVersion":"v1","metadata":{"name":"ns-yun-020064.vclound.com","uid":"a6575610-3c4c-468b-8511-a5d23196927c","resourceVersion":"211","creationTimestamp":"2023-02-07T00:22:06Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"ns-yun-020064.vclound.com","kubernetes.io/os":"linux"},"annotations":{"kubeadm.alpha.kubernetes.io/cri-socket":"unix:///var/run/containerd/containerd.sock","volumes.kubernetes.io/controller-managed-attach-detach":"true"},"managedFields":[{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:06Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:06Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{}}}}},"subresource":"status"},{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}}]},"spec":{"taints":[{"key":"node.kubernetes.io/not-ready","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"24","ephemeral-storage":"314107036Ki","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131850144Ki","pods":"110"},"allocatable":{"cpu":"24","ephemeral-storage":"289481043899","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131747744Ki","pods":"110"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletNotReady","message":"[container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized, CSINode is not yet initialized]"}],"addresses":[{"type":"InternalIP","address":"10.189.20.64"},{"type":"Hostname","address":"ns-yun-020064.vclound.com"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"3889867cda874b7f8bba55f6137923a8","systemUUID":"20d821d7-e8a2-e511-8a39-0027798f38ec","bootID":"750034a5-560d-4a66-9d9e-198aed158f5f","kernelVersion":"4.18.0-305.3.1.el8.x86_64","osImage":"CentOS Linux 8","containerRuntimeVersion":"containerd://1.6.15","kubeletVersion":"v1.24.8","kubeProxyVersion":"v1.24.8","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["k8s.gcr.io/etcd:3.5.5-0"],"sizeBytes":301522620},{"names":["k8s.gcr.io/kube-apiserver:v1.24.10"],"sizeBytes":131733971},{"names":["k8s.gcr.io/kube-apiserver:v1.24.8"],"sizeBytes":131113427},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.10"],"sizeBytes":121342265},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.8"],"sizeBytes":120746809},{"names":["k8s.gcr.io/kube-proxy:v1.24.10"],"sizeBytes":112211511},{"names":["k8s.gcr.io/kube-proxy:v1.24.8"],"sizeBytes":111867447},{"names":["k8s.gcr.io/kube-scheduler:v1.24.10"],"sizeBytes":52751160},{"names":["k8s.gcr.io/kube-scheduler:v1.24.8"],"sizeBytes":52339512},{"names":["k8s.gcr.io/coredns/coredns:v1.8.6","k8s.gcr.io/coredns:v1.8.6"],"sizeBytes":46957023},{"names":["k8s.gcr.io/pause:3.7"],"sizeBytes":714605},{"names":["registry.k8s.io/pause:3.6"],"sizeBytes":685866}]}}
I0207 08:22:08.899866   46264 request.go:1154] Request Body: {"metadata":{"labels":{"node-role.kubernetes.io/control-plane":"","node.kubernetes.io/exclude-from-external-load-balancers":""}},"spec":{"taints":[{"effect":"NoSchedule","key":"node-role.kubernetes.io/master"},{"effect":"NoSchedule","key":"node-role.kubernetes.io/control-plane"},{"effect":"NoSchedule","key":"node.kubernetes.io/not-ready"}]}}
I0207 08:22:08.899935   46264 round_trippers.go:466] curl -v -XPATCH  -H "Accept: application/json, */*" -H "Content-Type: application/strategic-merge-patch+json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020064.vclound.com?timeout=10s'
I0207 08:22:08.903835   46264 round_trippers.go:553] PATCH https://10.189.20.64:6443/api/v1/nodes/ns-yun-020064.vclound.com?timeout=10s 200 OK in 3 milliseconds
I0207 08:22:08.903858   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 3 ms Duration 3 ms
I0207 08:22:08.903869   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.903881   46264 round_trippers.go:580]     Audit-Id: d83c38b0-5d2d-49b8-b2d1-ead161e4d086
I0207 08:22:08.903906   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.903929   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.903949   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.903969   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.903991   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.904098   46264 request.go:1154] Response Body: {"kind":"Node","apiVersion":"v1","metadata":{"name":"ns-yun-020064.vclound.com","uid":"a6575610-3c4c-468b-8511-a5d23196927c","resourceVersion":"215","creationTimestamp":"2023-02-07T00:22:06Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"ns-yun-020064.vclound.com","kubernetes.io/os":"linux","node-role.kubernetes.io/control-plane":"","node.kubernetes.io/exclude-from-external-load-balancers":""},"annotations":{"kubeadm.alpha.kubernetes.io/cri-socket":"unix:///var/run/containerd/containerd.sock","volumes.kubernetes.io/controller-managed-attach-detach":"true"},"managedFields":[{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:06Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:06Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{}}}}},"subresource":"status"},{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/control-plane":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{}}},"f:spec":{"f:taints":{}}}}]},"spec":{"taints":[{"key":"node-role.kubernetes.io/master","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/control-plane","effect":"NoSchedule"},{"key":"node.kubernetes.io/not-ready","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"24","ephemeral-storage":"314107036Ki","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131850144Ki","pods":"110"},"allocatable":{"cpu":"24","ephemeral-storage":"289481043899","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131747744Ki","pods":"110"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"False","lastHeartbeatTime":"2023-02-07T00:22:06Z","lastTransitionTime":"2023-02-07T00:22:02Z","reason":"KubeletNotReady","message":"[container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized, CSINode is not yet initialized]"}],"addresses":[{"type":"InternalIP","address":"10.189.20.64"},{"type":"Hostname","address":"ns-yun-020064.vclound.com"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"3889867cda874b7f8bba55f6137923a8","systemUUID":"20d821d7-e8a2-e511-8a39-0027798f38ec","bootID":"750034a5-560d-4a66-9d9e-198aed158f5f","kernelVersion":"4.18.0-305.3.1.el8.x86_64","osImage":"CentOS Linux 8","containerRuntimeVersion":"containerd://1.6.15","kubeletVersion":"v1.24.8","kubeProxyVersion":"v1.24.8","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["k8s.gcr.io/etcd:3.5.5-0"],"sizeBytes":301522620},{"names":["k8s.gcr.io/kube-apiserver:v1.24.10"],"sizeBytes":131733971},{"names":["k8s.gcr.io/kube-apiserver:v1.24.8"],"sizeBytes":131113427},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.10"],"sizeBytes":121342265},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.8"],"sizeBytes":120746809},{"names":["k8s.gcr.io/kube-proxy:v1.24.10"],"sizeBytes":112211511},{"names":["k8s.gcr.io/kube-proxy:v1.24.8"],"sizeBytes":111867447},{"names":["k8s.gcr.io/kube-scheduler:v1.24.10"],"sizeBytes":52751160},{"names":["k8s.gcr.io/kube-scheduler:v1.24.8"],"sizeBytes":52339512},{"names":["k8s.gcr.io/coredns/coredns:v1.8.6","k8s.gcr.io/coredns:v1.8.6"],"sizeBytes":46957023},{"names":["k8s.gcr.io/pause:3.7"],"sizeBytes":714605},{"names":["registry.k8s.io/pause:3.6"],"sizeBytes":685866}]}}
[bootstrap-token] Using token: 8foyh9.qjblri5jcbkvbfp7
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I0207 08:22:08.904565   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/secrets/bootstrap-token-8foyh9?timeout=10s'
I0207 08:22:08.906069   46264 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/namespaces/kube-system/secrets/bootstrap-token-8foyh9?timeout=10s 404 Not Found in 1 milliseconds
I0207 08:22:08.906091   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:08.906102   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.906117   46264 round_trippers.go:580]     Audit-Id: bdf94b00-8292-427a-a217-53b04076d06b
I0207 08:22:08.906140   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.906161   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.906181   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.906203   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.906224   46264 round_trippers.go:580]     Content-Length: 218
I0207 08:22:08.906246   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.906291   46264 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \"bootstrap-token-8foyh9\" not found","reason":"NotFound","details":{"name":"bootstrap-token-8foyh9","kind":"secrets"},"code":404}
I0207 08:22:08.906580   46264 request.go:1154] Request Body: {"kind":"Secret","apiVersion":"v1","metadata":{"name":"bootstrap-token-8foyh9","namespace":"kube-system","creationTimestamp":null},"data":{"auth-extra-groups":"c3lzdGVtOmJvb3RzdHJhcHBlcnM6a3ViZWFkbTpkZWZhdWx0LW5vZGUtdG9rZW4=","description":"VGhlIGRlZmF1bHQgYm9vdHN0cmFwIHRva2VuIGdlbmVyYXRlZCBieSAna3ViZWFkbSBpbml0Jy4=","expiration":"MjAyMy0wMi0wOFQwMDoyMjowOFo=","token-id":"OGZveWg5","token-secret":"cWpibHJpNWpjYmt2YmZwNw==","usage-bootstrap-authentication":"dHJ1ZQ==","usage-bootstrap-signing":"dHJ1ZQ=="},"type":"bootstrap.kubernetes.io/token"}
I0207 08:22:08.906672   46264 round_trippers.go:466] curl -v -XPOST  -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/secrets?timeout=10s'
I0207 08:22:08.908850   46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-system/secrets?timeout=10s 201 Created in 2 milliseconds
I0207 08:22:08.908869   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 08:22:08.908879   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.908889   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.908910   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.908933   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.908952   46264 round_trippers.go:580]     Content-Length: 979
I0207 08:22:08.908973   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.908994   46264 round_trippers.go:580]     Audit-Id: 7a813034-b444-4cce-b0ab-59d5d8f9630c
I0207 08:22:08.909015   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.909053   46264 request.go:1154] Response Body: {"kind":"Secret","apiVersion":"v1","metadata":{"name":"bootstrap-token-8foyh9","namespace":"kube-system","uid":"279a0e94-8134-437d-92dc-31bfe5d03ebc","resourceVersion":"216","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:auth-extra-groups":{},"f:description":{},"f:expiration":{},"f:token-id":{},"f:token-secret":{},"f:usage-bootstrap-authentication":{},"f:usage-bootstrap-signing":{}},"f:type":{}}}]},"data":{"auth-extra-groups":"c3lzdGVtOmJvb3RzdHJhcHBlcnM6a3ViZWFkbTpkZWZhdWx0LW5vZGUtdG9rZW4=","description":"VGhlIGRlZmF1bHQgYm9vdHN0cmFwIHRva2VuIGdlbmVyYXRlZCBieSAna3ViZWFkbSBpbml0Jy4=","expiration":"MjAyMy0wMi0wOFQwMDoyMjowOFo=","token-id":"OGZveWg5","token-secret":"cWpibHJpNWpjYmt2YmZwNw==","usage-bootstrap-authentication":"dHJ1ZQ==","usage-bootstrap-signing":"dHJ1ZQ=="},"type":"bootstrap.kubernetes.io/token"}
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I0207 08:22:08.909343   46264 request.go:1154] Request Body: {"kind":"ClusterRole","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:get-nodes","namespace":"kube-system","creationTimestamp":null},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["nodes"]}]}
I0207 08:22:08.909428   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s'
I0207 08:22:08.911388   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:08.911412   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:08.911429   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.911442   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.911453   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.911464   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.911475   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.911486   46264 round_trippers.go:580]     Content-Length: 471
I0207 08:22:08.911496   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.911507   46264 round_trippers.go:580]     Audit-Id: 6c16e53d-1dcf-4c88-a6b9-17fcfd0cb108
I0207 08:22:08.911529   46264 request.go:1154] Response Body: {"kind":"ClusterRole","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:get-nodes","uid":"da748614-9a75-4b3d-a4cd-edd1e0c57008","resourceVersion":"217","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:rules":{}}}]},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["nodes"]}]}
I0207 08:22:08.911753   46264 request.go:1154] Request Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:get-nodes","namespace":"kube-system","creationTimestamp":null},"subjects":[{"kind":"Group","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"kubeadm:get-nodes"}}
I0207 08:22:08.911817   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s'
I0207 08:22:08.913733   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:08.913755   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:08.913765   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.913781   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.913804   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.913824   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.913845   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.913866   46264 round_trippers.go:580]     Content-Length: 654
I0207 08:22:08.913887   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.913908   46264 round_trippers.go:580]     Audit-Id: 0654d9a3-eb22-4bab-80b5-5ffda9a37d6d
I0207 08:22:08.913945   46264 request.go:1154] Response Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:get-nodes","uid":"633438d1-9082-4ab6-bb6c-11b1a90840df","resourceVersion":"218","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"kubeadm:get-nodes"}}
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0207 08:22:08.914138   46264 request.go:1154] Request Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:kubelet-bootstrap","creationTimestamp":null},"subjects":[{"kind":"Group","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"system:node-bootstrapper"}}
I0207 08:22:08.914225   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s'
I0207 08:22:08.915896   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:08.915919   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:08.915933   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.915945   46264 round_trippers.go:580]     Audit-Id: 626f6508-f050-455f-8cfc-d02745445825
I0207 08:22:08.915956   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.915968   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.915979   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.915993   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.916009   46264 round_trippers.go:580]     Content-Length: 669
I0207 08:22:08.916020   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.916051   46264 request.go:1154] Response Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:kubelet-bootstrap","uid":"f3f353ea-4188-4039-bdcc-051856772a1b","resourceVersion":"219","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"system:node-bootstrapper"}}
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I0207 08:22:08.916178   46264 request.go:1154] Request Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:node-autoapprove-bootstrap","creationTimestamp":null},"subjects":[{"kind":"Group","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"system:certificates.k8s.io:certificatesigningrequests:nodeclient"}}
I0207 08:22:08.916243   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s'
I0207 08:22:08.918118   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:08.918141   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:08.918152   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.918167   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.918191   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.918212   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.918232   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.918252   46264 round_trippers.go:580]     Content-Length: 718
I0207 08:22:08.918278   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.918301   46264 round_trippers.go:580]     Audit-Id: 76579b13-ae10-48a1-8a3a-96e8926537dd
I0207 08:22:08.918342   46264 request.go:1154] Response Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:node-autoapprove-bootstrap","uid":"3acfcc99-5bde-4b5a-bfd5-5a8b2c7bcdf3","resourceVersion":"220","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"system:certificates.k8s.io:certificatesigningrequests:nodeclient"}}
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0207 08:22:08.918513   46264 request.go:1154] Request Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:node-autoapprove-certificate-rotation","creationTimestamp":null},"subjects":[{"kind":"Group","name":"system:nodes"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"system:certificates.k8s.io:certificatesigningrequests:selfnodeclient"}}
I0207 08:22:08.918598   46264 round_trippers.go:466] curl -v -XPOST  -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" -H "Content-Type: application/json" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s'
I0207 08:22:08.920411   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:08.920438   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:08.920448   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.920462   46264 round_trippers.go:580]     Audit-Id: fe6d8cda-25cd-4344-9998-54b44c8c3252
I0207 08:22:08.920486   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.920506   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.920525   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.920545   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.920565   46264 round_trippers.go:580]     Content-Length: 698
I0207 08:22:08.920587   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.920631   46264 request.go:1154] Response Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:node-autoapprove-certificate-rotation","uid":"299899c7-f7bd-409c-b431-641e9ab3d221","resourceVersion":"221","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:nodes"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"system:certificates.k8s.io:certificatesigningrequests:selfnodeclient"}}
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0207 08:22:08.920777   46264 clusterinfo.go:47] [bootstrap-token] loading admin kubeconfig
I0207 08:22:08.921393   46264 loader.go:372] Config loaded from file:  /etc/kubernetes/admin.conf
I0207 08:22:08.921413   46264 clusterinfo.go:58] [bootstrap-token] copying the cluster from admin.conf to the bootstrap kubeconfig
I0207 08:22:08.921736   46264 clusterinfo.go:70] [bootstrap-token] creating/updating ConfigMap in kube-public namespace
I0207 08:22:08.921806   46264 request.go:1154] Request Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"cluster-info","namespace":"kube-public","creationTimestamp":null},"data":{"kubeconfig":"apiVersion: v1\nclusters:\n- cluster:\n    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJek1ESXdOekF3TWpFMU4xb1hEVE16TURJd05EQXdNakUxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTHExCk9iRUdDR0JWbldldEpTNFNJU0J6R0s2bGthamVSczV5MmhCVnZVZVpEb3BWZUVrNjNycURNc3UwZDN5MXJqUzEKeUMxeG9hcjJIVVM4NWpvVmpmMzFsY2xCS3NGTWJkWkE2T2VBdk1oMGFjcjkrRVBYMzUzRGlNRndnT0poSjYrQwpWRm53MEpZeFlOeXB3MDFWUURuZ1ZGK0lKY29GRllZSEVmbGRWTVhFRldhN1RrYjVPRXhTVGZhMjhvaFdJQzdiClNPWjhvMmdXdGhWQ3k0SzBXb3pVYkdrQ3plYTAyU1lJTlJzNjhQTERZMmlkUUx6dG5QUWltWHJ1UzM2RFpOUFYKWXBLQzRpQlEvNC9UV0VOdnhqWFZUUldyckdDQ2NGVzdpS0lwMFIveTZBSTkwUUVJTW5uMVlVODNmb2s1ZmJ2YwppMWh0a3FsUXNhWXJ4bzJHbzBNQ0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOZXNmT1M1MmtvcFZ2aG9UbmdYeGRtai8zOStNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBQzQ3ZFRYV2YwRGJDS0JheG1qZwpyZk0zdU5POUJxR004TklrSjNjMkR6c2JoUHFVL25wTXlleVZEOEdZSzR6RGVEckl3TzFGSWlYRXBDYzdhM2RwCjFwSmhkekI0N0lPWVNIY2pGS05iY0IreUxXdDMzQ3JRd2NoRHNwdkVmOUdjSjk2VStRRFh5S0o5YmFGZmhxVTgKSkR4Y05UcTU5RjNUeUk1SDcxTHhhdkRDb1B2WlN6S1NXTGtOUGRjd3FJREoxdXVDVkpNNC94dWpHN2tSMVZUSwptWUdGb0pWVTA3TTRpQm8xNXFGd1U1RmkyM0NUU3dHY1h6eHVrOGViY1FWVDl3aTZMQjQyaGk1bHhXbVcrR3hEClVOaHhoK0laQTA3QzdtOUlnSDRXaUgzQjNTTXZZNGtEYlJMTzlmdVhvbGVBUDQreWlXY1BmM0NLT3NYdkRHbC8KZDBVPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==\n    server: https://10.189.20.64:6443\n  name: \"\"\ncontexts: null\ncurrent-context: \"\"\nkind: Config\npreferences: {}\nusers: null\n"}}
I0207 08:22:08.921909   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-public/configmaps?timeout=10s'
I0207 08:22:08.924003   46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-public/configmaps?timeout=10s 201 Created in 2 milliseconds
I0207 08:22:08.924032   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 2 ms
I0207 08:22:08.924056   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.924079   46264 round_trippers.go:580]     Content-Length: 2096
I0207 08:22:08.924100   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.924121   46264 round_trippers.go:580]     Audit-Id: 7367d04d-df17-4d86-a643-2d429fd0626e
I0207 08:22:08.924142   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.924161   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.924182   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.924202   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.924269   46264 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"cluster-info","namespace":"kube-public","uid":"c8d08668-6dbf-4cba-85a0-fee86d04140c","resourceVersion":"222","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:kubeconfig":{}}}}]},"data":{"kubeconfig":"apiVersion: v1\nclusters:\n- cluster:\n    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJek1ESXdOekF3TWpFMU4xb1hEVE16TURJd05EQXdNakUxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTHExCk9iRUdDR0JWbldldEpTNFNJU0J6R0s2bGthamVSczV5MmhCVnZVZVpEb3BWZUVrNjNycURNc3UwZDN5MXJqUzEKeUMxeG9hcjJIVVM4NWpvVmpmMzFsY2xCS3NGTWJkWkE2T2VBdk1oMGFjcjkrRVBYMzUzRGlNRndnT0poSjYrQwpWRm53MEpZeFlOeXB3MDFWUURuZ1ZGK0lKY29GRllZSEVmbGRWTVhFRldhN1RrYjVPRXhTVGZhMjhvaFdJQzdiClNPWjhvMmdXdGhWQ3k0SzBXb3pVYkdrQ3plYTAyU1lJTlJzNjhQTERZMmlkUUx6dG5QUWltWHJ1UzM2RFpOUFYKWXBLQzRpQlEvNC9UV0VOdnhqWFZUUldyckdDQ2NGVzdpS0lwMFIveTZBSTkwUUVJTW5uMVlVODNmb2s1ZmJ2YwppMWh0a3FsUXNhWXJ4bzJHbzBNQ0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOZXNmT1M1MmtvcFZ2aG9UbmdYeGRtai8zOStNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBQzQ3ZFRYV2YwRGJDS0JheG1qZwpyZk0zdU5POUJxR004TklrSjNjMkR6c2JoUHFVL25wTXlleVZEOEdZSzR6RGVEckl3TzFGSWlYRXBDYzdhM2RwCjFwSmhkekI0N0lPWVNIY2pGS05iY0IreUxXdDMzQ3JRd2NoRHNwdkVmOUdjSjk2VStRRFh5S0o5YmFGZmhxVTgKSkR4Y05UcTU5RjNUeUk1SDcxTHhhdkRDb1B2WlN6S1NXTGtOUGRjd3FJREoxdXVDVkpNNC94dWpHN2tSMVZUSwptWUdGb0pWVTA3TTRpQm8xNXFGd1U1RmkyM0NUU3dHY1h6eHVrOGViY1FWVDl3aTZMQjQyaGk1bHhXbVcrR3hEClVOaHhoK0laQTA3QzdtOUlnSDRXaUgzQjNTTXZZNGtEYlJMTzlmdVhvbGVBUDQreWlXY1BmM0NLT3NYdkRHbC8KZDBVPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==\n    server: https://10.189.20.64:6443\n  name: \"\"\ncontexts: null\ncurrent-context: \"\"\nkind: Config\npreferences: {}\nusers: null\n"}}
I0207 08:22:08.924429   46264 clusterinfo.go:84] creating the RBAC rules for exposing the cluster-info ConfigMap in the kube-public namespace
I0207 08:22:08.924489   46264 request.go:1154] Request Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:bootstrap-signer-clusterinfo","namespace":"kube-public","creationTimestamp":null},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["cluster-info"]}]}
I0207 08:22:08.924578   46264 round_trippers.go:466] curl -v -XPOST  -H "Content-Type: application/json" -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/roles?timeout=10s'
I0207 08:22:08.926538   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/roles?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:08.926560   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:08.926570   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.926581   46264 round_trippers.go:580]     Content-Length: 547
I0207 08:22:08.926594   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.926617   46264 round_trippers.go:580]     Audit-Id: 77c414d9-1489-475e-907f-49de2bdde8ff
I0207 08:22:08.926637   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.926658   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.926679   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.926700   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.926739   46264 request.go:1154] Response Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:bootstrap-signer-clusterinfo","namespace":"kube-public","uid":"dd3ac90c-3968-46f1-9b49-6f85a1495644","resourceVersion":"223","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:rules":{}}}]},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["cluster-info"]}]}
I0207 08:22:08.926888   46264 request.go:1154] Request Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:bootstrap-signer-clusterinfo","namespace":"kube-public","creationTimestamp":null},"subjects":[{"kind":"User","name":"system:anonymous"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kubeadm:bootstrap-signer-clusterinfo"}}
I0207 08:22:08.926976   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/rolebindings?timeout=10s'
I0207 08:22:08.928956   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/rolebindings?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:08.928979   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:08.928994   46264 round_trippers.go:577] Response Headers:
I0207 08:22:08.929010   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:08.929022   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:08.929032   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:08.929044   46264 round_trippers.go:580]     Content-Length: 672
I0207 08:22:08.929054   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:08 GMT
I0207 08:22:08.929065   46264 round_trippers.go:580]     Audit-Id: db1eff49-6e2e-45a0-9e44-3cb95c4eab5e
I0207 08:22:08.929075   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:08.929100   46264 request.go:1154] Response Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:bootstrap-signer-clusterinfo","namespace":"kube-public","uid":"b7d78cf8-77d6-45e3-a539-a033325e6ddd","resourceVersion":"224","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"User","apiGroup":"rbac.authorization.k8s.io","name":"system:anonymous"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kubeadm:bootstrap-signer-clusterinfo"}}
I0207 08:22:08.929224   46264 kubeletfinalize.go:90] [kubelet-finalize] Assuming that kubelet client certificate rotation is enabled: found "/var/lib/kubelet/pki/kubelet-client-current.pem"
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0207 08:22:08.929776   46264 loader.go:372] Config loaded from file:  /etc/kubernetes/kubelet.conf
I0207 08:22:08.930333   46264 kubeletfinalize.go:134] [kubelet-finalize] Restarting the kubelet to enable client certificate rotation
I0207 08:22:09.107029   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/apps/v1/namespaces/kube-system/deployments?labelSelector=k8s-app%3Dkube-dns'
I0207 08:22:09.111546   46264 round_trippers.go:553] GET https://10.189.20.64:6443/apis/apps/v1/namespaces/kube-system/deployments?labelSelector=k8s-app%3Dkube-dns 200 OK in 4 milliseconds
I0207 08:22:09.111574   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 4 ms Duration 4 ms
I0207 08:22:09.111587   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.111600   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.111612   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.111622   46264 round_trippers.go:580]     Content-Length: 97
I0207 08:22:09.111634   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.111645   46264 round_trippers.go:580]     Audit-Id: 32f9fb2d-29db-4471-a4b7-493e458b558e
I0207 08:22:09.111656   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.111667   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.111703   46264 request.go:1154] Response Body: {"kind":"DeploymentList","apiVersion":"apps/v1","metadata":{"resourceVersion":"225"},"items":[]}
I0207 08:22:09.114088   46264 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps/coredns?timeout=10s'
I0207 08:22:09.115644   46264 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps/coredns?timeout=10s 404 Not Found in 1 milliseconds
I0207 08:22:09.115669   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:09.115681   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.115693   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.115704   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.115715   46264 round_trippers.go:580]     Content-Length: 194
I0207 08:22:09.115729   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.115740   46264 round_trippers.go:580]     Audit-Id: 235910c2-011d-478e-8977-4ae7f2cf4cbe
I0207 08:22:09.115751   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.115762   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.115786   46264 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"configmaps \"coredns\" not found","reason":"NotFound","details":{"name":"coredns","kind":"configmaps"},"code":404}
I0207 08:22:09.115926   46264 request.go:1154] Request Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"coredns","namespace":"kube-system","creationTimestamp":null},"data":{"Corefile":".:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n"}}
I0207 08:22:09.115994   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s'
I0207 08:22:09.118288   46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 2 milliseconds
I0207 08:22:09.118309   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 08:22:09.118321   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.118333   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.118345   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.118356   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.118367   46264 round_trippers.go:580]     Content-Length: 769
I0207 08:22:09.118378   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.118389   46264 round_trippers.go:580]     Audit-Id: 529d2fce-9c6c-44aa-b7f3-cc459fb3725b
I0207 08:22:09.118400   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.118428   46264 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"coredns","namespace":"kube-system","uid":"c401b428-1897-4bfc-ac70-82dbd9739da5","resourceVersion":"226","creationTimestamp":"2023-02-07T00:22:09Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:Corefile":{}}}}]},"data":{"Corefile":".:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n"}}
I0207 08:22:09.118781   46264 request.go:1154] Request Body: {"kind":"ClusterRole","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"system:coredns","creationTimestamp":null},"rules":[{"verbs":["list","watch"],"apiGroups":[""],"resources":["endpoints","services","pods","namespaces"]},{"verbs":["get"],"apiGroups":[""],"resources":["nodes"]},{"verbs":["list","watch"],"apiGroups":["discovery.k8s.io"],"resources":["endpointslices"]}]}
I0207 08:22:09.118848   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s'
I0207 08:22:09.120915   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s 201 Created in 2 milliseconds
I0207 08:22:09.120943   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 2 ms
I0207 08:22:09.120955   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.120967   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.120978   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.120996   46264 round_trippers.go:580]     Content-Length: 660
I0207 08:22:09.121015   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.121033   46264 round_trippers.go:580]     Audit-Id: 853846e7-b535-467b-bab0-9fa56b9f82f0
I0207 08:22:09.121050   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.121066   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.121101   46264 request.go:1154] Response Body: {"kind":"ClusterRole","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"system:coredns","uid":"e542afaa-3c95-4d05-ad7a-affa16816372","resourceVersion":"227","creationTimestamp":"2023-02-07T00:22:09Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:rules":{}}}]},"rules":[{"verbs":["list","watch"],"apiGroups":[""],"resources":["endpoints","services","pods","namespaces"]},{"verbs":["get"],"apiGroups":[""],"resources":["nodes"]},{"verbs":["list","watch"],"apiGroups":["discovery.k8s.io"],"resources":["endpointslices"]}]}
I0207 08:22:09.121494   46264 request.go:1154] Request Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"system:coredns","creationTimestamp":null},"subjects":[{"kind":"ServiceAccount","name":"coredns","namespace":"kube-system"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"system:coredns"}}
I0207 08:22:09.121596   46264 round_trippers.go:466] curl -v -XPOST  -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" -H "Content-Type: application/json" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s'
I0207 08:22:09.123754   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 2 milliseconds
I0207 08:22:09.123777   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 2 ms
I0207 08:22:09.123789   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.123804   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.123816   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.123827   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.123840   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.123851   46264 round_trippers.go:580]     Content-Length: 604
I0207 08:22:09.123861   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.123872   46264 round_trippers.go:580]     Audit-Id: 52e608ba-f4b2-47f6-8f86-a2d90a65483f
I0207 08:22:09.123896   46264 request.go:1154] Response Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"system:coredns","uid":"259d908c-3bf1-4d9a-8213-4bba4fadff4c","resourceVersion":"228","creationTimestamp":"2023-02-07T00:22:09Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"ServiceAccount","name":"coredns","namespace":"kube-system"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"system:coredns"}}
I0207 08:22:09.124250   46264 request.go:1154] Request Body: {"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"coredns","namespace":"kube-system","creationTimestamp":null}}
I0207 08:22:09.124331   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s'
I0207 08:22:09.126550   46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s 201 Created in 2 milliseconds
I0207 08:22:09.126572   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 08:22:09.126584   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.126596   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.126608   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.126618   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.126629   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.126640   46264 round_trippers.go:580]     Content-Length: 212
I0207 08:22:09.126651   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.126662   46264 round_trippers.go:580]     Audit-Id: 93c237d7-a9b6-4758-8841-2eb8b94b5514
I0207 08:22:09.126685   46264 request.go:1154] Response Body: {"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"coredns","namespace":"kube-system","uid":"46ae662f-1560-4516-b139-ad43ab564ee6","resourceVersion":"229","creationTimestamp":"2023-02-07T00:22:09Z"}}
I0207 08:22:09.127705   46264 request.go:1154] Request Body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"coredns","namespace":"kube-system","creationTimestamp":null,"labels":{"k8s-app":"kube-dns"}},"spec":{"replicas":2,"selector":{"matchLabels":{"k8s-app":"kube-dns"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"k8s-app":"kube-dns"}},"spec":{"volumes":[{"name":"config-volume","configMap":{"name":"coredns","items":[{"key":"Corefile","path":"Corefile"}]}}],"containers":[{"name":"coredns","image":"k8s.gcr.io/coredns/coredns:v1.8.6","args":["-conf","/etc/coredns/Corefile"],"ports":[{"name":"dns","containerPort":53,"protocol":"UDP"},{"name":"dns-tcp","containerPort":53,"protocol":"TCP"},{"name":"metrics","containerPort":9153,"protocol":"TCP"}],"resources":{"limits":{"memory":"170Mi"},"requests":{"cpu":"100m","memory":"70Mi"}},"volumeMounts":[{"name":"config-volume","readOnly":true,"mountPath":"/etc/coredns"}],"livenessProbe":{"httpGet":{"path":"/health","port":8080,"scheme":"HTTP"},"initialDelaySeconds":60,"timeoutSeconds":5,"successThreshold":1,"failureThreshold":5},"readinessProbe":{"httpGet":{"path":"/ready","port":8181,"scheme":"HTTP"}},"imagePullPolicy":"IfNotPresent","securityContext":{"capabilities":{"add":["NET_BIND_SERVICE"],"drop":["all"]},"readOnlyRootFilesystem":true,"allowPrivilegeEscalation":false}}],"dnsPolicy":"Default","nodeSelector":{"kubernetes.io/os":"linux"},"serviceAccountName":"coredns","tolerations":[{"key":"CriticalAddonsOnly","operator":"Exists"},{"key":"node-role.kubernetes.io/master","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/control-plane","effect":"NoSchedule"}],"priorityClassName":"system-cluster-critical"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":1}}},"status":{}}
I0207 08:22:09.127782   46264 round_trippers.go:466] curl -v -XPOST  -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" 'https://10.189.20.64:6443/apis/apps/v1/namespaces/kube-system/deployments?timeout=10s'
I0207 08:22:09.133781   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/apps/v1/namespaces/kube-system/deployments?timeout=10s 201 Created in 5 milliseconds
I0207 08:22:09.133810   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 5 ms Duration 5 ms
I0207 08:22:09.133824   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.133836   46264 round_trippers.go:580]     Audit-Id: dd271f63-b904-4306-9e25-74bac667bca7
I0207 08:22:09.133847   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.133858   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.133868   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.133879   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.133890   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.133991   46264 request.go:1154] Response Body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"coredns","namespace":"kube-system","uid":"53649bcf-26f0-4bac-b147-180a7b6458b6","resourceVersion":"230","generation":1,"creationTimestamp":"2023-02-07T00:22:09Z","labels":{"k8s-app":"kube-dns"},"managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"apps/v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:k8s-app":{}}},"f:spec":{"f:progressDeadlineSeconds":{},"f:replicas":{},"f:revisionHistoryLimit":{},"f:selector":{},"f:strategy":{"f:rollingUpdate":{".":{},"f:maxSurge":{},"f:maxUnavailable":{}},"f:type":{}},"f:template":{"f:metadata":{"f:labels":{".":{},"f:k8s-app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"coredns\"}":{".":{},"f:args":{},"f:image":{},"f:imagePullPolicy":{},"f:livenessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:name":{},"f:ports":{".":{},"k:{\"containerPort\":53,\"protocol\":\"TCP\"}":{".":{},"f:containerPort":{},"f:name":{},"f:protocol":{}},"k:{\"containerPort\":53,\"protocol\":\"UDP\"}":{".":{},"f:containerPort":{},"f:name":{},"f:protocol":{}},"k:{\"containerPort\":9153,\"protocol\":\"TCP\"}":{".":{},"f:containerPort":{},"f:name":{},"f:protocol":{}}},"f:readinessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:resources":{".":{},"f:limits":{".":{},"f:memory":{}},"f:requests":{".":{},"f:cpu":{},"f:memory":{}}},"f:securityContext":{".":{},"f:allowPrivilegeEscalation":{},"f:capabilities":{".":{},"f:add":{},"f:drop":{}},"f:readOnlyRootFilesystem":{}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/etc/coredns\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}}}}},"f:dnsPolicy":{},"f:nodeSelector":{},"f:priorityClassName":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:serviceAccount":{},"f:serviceAccountName":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{},"f:volumes":{".":{},"k:{\"name\":\"config-volume\"}":{".":{},"f:configMap":{".":{},"f:defaultMode":{},"f:items":{},"f:name":{}},"f:name":{}}}}}}}}]},"spec":{"replicas":2,"selector":{"matchLabels":{"k8s-app":"kube-dns"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"k8s-app":"kube-dns"}},"spec":{"volumes":[{"name":"config-volume","configMap":{"name":"coredns","items":[{"key":"Corefile","path":"Corefile"}],"defaultMode":420}}],"containers":[{"name":"coredns","image":"k8s.gcr.io/coredns/coredns:v1.8.6","args":["-conf","/etc/coredns/Corefile"],"ports":[{"name":"dns","containerPort":53,"protocol":"UDP"},{"name":"dns-tcp","containerPort":53,"protocol":"TCP"},{"name":"metrics","containerPort":9153,"protocol":"TCP"}],"resources":{"limits":{"memory":"170Mi"},"requests":{"cpu":"100m","memory":"70Mi"}},"volumeMounts":[{"name":"config-volume","readOnly":true,"mountPath":"/etc/coredns"}],"livenessProbe":{"httpGet":{"path":"/health","port":8080,"scheme":"HTTP"},"initialDelaySeconds":60,"timeoutSeconds":5,"periodSeconds":10,"successThreshold":1,"failureThreshold":5},"readinessProbe":{"httpGet":{"path":"/ready","port":8181,"scheme":"HTTP"},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"capabilities":{"add":["NET_BIND_SERVICE"],"drop":["all"]},"readOnlyRootFilesystem":true,"allowPrivilegeEscalation":false}}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"Default","nodeSelector":{"kubernetes.io/os":"linux"},"serviceAccountName":"coredns","serviceAccount":"coredns","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"CriticalAddonsOnly","operator":"Exists"},{"key":"node-role.kubernetes.io/master","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/control-plane","effect":"NoSchedule"}],"priorityClassName":"system-cluster-critical"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":1,"maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}}
I0207 08:22:09.135174   46264 request.go:1154] Request Body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"kube-dns","namespace":"kube-system","resourceVersion":"0","creationTimestamp":null,"labels":{"k8s-app":"kube-dns","kubernetes.io/cluster-service":"true","kubernetes.io/name":"CoreDNS"},"annotations":{"prometheus.io/port":"9153","prometheus.io/scrape":"true"}},"spec":{"ports":[{"name":"dns","protocol":"UDP","port":53,"targetPort":53},{"name":"dns-tcp","protocol":"TCP","port":53,"targetPort":53},{"name":"metrics","protocol":"TCP","port":9153,"targetPort":9153}],"selector":{"k8s-app":"kube-dns"},"clusterIP":"10.96.0.10"},"status":{"loadBalancer":{}}}
I0207 08:22:09.135251   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/services?timeout=10s'
I0207 08:22:09.139430   46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-system/services?timeout=10s 201 Created in 4 milliseconds
I0207 08:22:09.139452   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 3 ms Duration 4 ms
I0207 08:22:09.139464   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.139477   46264 round_trippers.go:580]     Audit-Id: b476d48f-c6d3-4d4d-8a3b-cb639e4582f5
I0207 08:22:09.139488   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.139498   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.139509   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.139520   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.139531   46264 round_trippers.go:580]     Content-Length: 1619
I0207 08:22:09.139541   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.139575   46264 request.go:1154] Response Body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"kube-dns","namespace":"kube-system","uid":"1764a9d2-bdad-4054-bd63-96159f5b6df6","resourceVersion":"232","creationTimestamp":"2023-02-07T00:22:09Z","labels":{"k8s-app":"kube-dns","kubernetes.io/cluster-service":"true","kubernetes.io/name":"CoreDNS"},"annotations":{"prometheus.io/port":"9153","prometheus.io/scrape":"true"},"managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:prometheus.io/port":{},"f:prometheus.io/scrape":{}},"f:labels":{".":{},"f:k8s-app":{},"f:kubernetes.io/cluster-service":{},"f:kubernetes.io/name":{}}},"f:spec":{"f:clusterIP":{},"f:internalTrafficPolicy":{},"f:ports":{".":{},"k:{\"port\":53,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":53,\"protocol\":\"UDP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":9153,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"dns","protocol":"UDP","port":53,"targetPort":53},{"name":"dns-tcp","protocol":"TCP","port":53,"targetPort":53},{"name":"metrics","protocol":"TCP","port":9153,"targetPort":9153}],"selector":{"k8s-app":"kube-dns"},"clusterIP":"10.96.0.10","clusterIPs":["10.96.0.10"],"type":"ClusterIP","sessionAffinity":"None","ipFamilies":["IPv4"],"ipFamilyPolicy":"SingleStack","internalTrafficPolicy":"Cluster"},"status":{"loadBalancer":{}}}
[addons] Applied essential addon: CoreDNS
I0207 08:22:09.139798   46264 request.go:1154] Request Body: {"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"kube-proxy","namespace":"kube-system","creationTimestamp":null}}
I0207 08:22:09.139869   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s'
I0207 08:22:09.141873   46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:09.141894   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:09.141906   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.141918   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.141929   46264 round_trippers.go:580]     Content-Length: 215
I0207 08:22:09.141939   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.141950   46264 round_trippers.go:580]     Audit-Id: 259268d3-ddfa-47b2-827c-6171e7a13826
I0207 08:22:09.141960   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.141971   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.141981   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.142006   46264 request.go:1154] Response Body: {"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"kube-proxy","namespace":"kube-system","uid":"5c4536a8-0d71-4fed-b2a2-7e9e67381030","resourceVersion":"233","creationTimestamp":"2023-02-07T00:22:09Z"}}
I0207 08:22:09.143118   46264 request.go:1154] Request Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kube-proxy","namespace":"kube-system","creationTimestamp":null,"labels":{"app":"kube-proxy"},"annotations":{"kubeadm.kubernetes.io/component-config.hash":"sha256:f6d0a8dd44c5a84730118c89ecf9edd3cd167f79c2e04a3e9859ef39e8094371"}},"data":{"config.conf":"apiVersion: kubeproxy.config.k8s.io/v1alpha1\nbindAddress: 0.0.0.0\nbindAddressHardFail: false\nclientConnection:\n  acceptContentTypes: \"\"\n  burst: 0\n  contentType: \"\"\n  kubeconfig: /var/lib/kube-proxy/kubeconfig.conf\n  qps: 0\nclusterCIDR: 10.244.0.0/16\nconfigSyncPeriod: 0s\nconntrack:\n  maxPerCore: null\n  min: null\n  tcpCloseWaitTimeout: null\n  tcpEstablishedTimeout: null\ndetectLocal:\n  bridgeInterface: \"\"\n  interfaceNamePrefix: \"\"\ndetectLocalMode: \"\"\nenableProfiling: false\nhealthzBindAddress: \"\"\nhostnameOverride: \"\"\niptables:\n  masqueradeAll: false\n  masqueradeBit: null\n  minSyncPeriod: 0s\n  syncPeriod: 0s\nipvs:\n  excludeCIDRs: null\n  minSyncPeriod: 0s\n  scheduler: \"\"\n  strictARP: false\n  syncPeriod: 0s\n  tcpFinTimeout: 0s\n  tcpTimeout: 0s\n  udpTimeout: 0s\nkind: KubeProxyConfiguration\nmetricsBindAddress: \"\"\nmode: \"\"\nnodePortAddresses: null\noomScoreAdj: null\nportRange: \"\"\nshowHiddenMetricsForVersion: \"\"\nudpIdleTimeout: 0s\nwinkernel:\n  enableDSR: false\n  forwardHealthCheckVip: false\n  networkName: \"\"\n  rootHnsEndpointName: \"\"\n  sourceVip: \"\"","kubeconfig.conf":"apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n    certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt\n    server: https://10.189.20.64:6443\n  name: default\ncontexts:\n- context:\n    cluster: default\n    namespace: default\n    user: default\n  name: default\ncurrent-context: default\nusers:\n- name: default\n  user:\n    tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token"}}
I0207 08:22:09.143191   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s'
I0207 08:22:09.145229   46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 2 milliseconds
I0207 08:22:09.145251   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 2 ms
I0207 08:22:09.145269   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.145282   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.145294   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.145304   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.145315   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.145326   46264 round_trippers.go:580]     Content-Length: 2305
I0207 08:22:09.145337   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.145347   46264 round_trippers.go:580]     Audit-Id: a26d239f-aea0-4a00-90eb-f2565788d7cc
I0207 08:22:09.145387   46264 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kube-proxy","namespace":"kube-system","uid":"594a7c08-bbf1-43f5-9dc5-8c7940fccb04","resourceVersion":"234","creationTimestamp":"2023-02-07T00:22:09Z","labels":{"app":"kube-proxy"},"annotations":{"kubeadm.kubernetes.io/component-config.hash":"sha256:f6d0a8dd44c5a84730118c89ecf9edd3cd167f79c2e04a3e9859ef39e8094371"},"managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:config.conf":{},"f:kubeconfig.conf":{}},"f:metadata":{"f:annotations":{".":{},"f:kubeadm.kubernetes.io/component-config.hash":{}},"f:labels":{".":{},"f:app":{}}}}}]},"data":{"config.conf":"apiVersion: kubeproxy.config.k8s.io/v1alpha1\nbindAddress: 0.0.0.0\nbindAddressHardFail: false\nclientConnection:\n  acceptContentTypes: \"\"\n  burst: 0\n  contentType: \"\"\n  kubeconfig: /var/lib/kube-proxy/kubeconfig.conf\n  qps: 0\nclusterCIDR: 10.244.0.0/16\nconfigSyncPeriod: 0s\nconntrack:\n  maxPerCore: null\n  min: null\n  tcpCloseWaitTimeout: null\n  tcpEstablishedTimeout: null\ndetectLocal:\n  bridgeInterface: \"\"\n  interfaceNamePrefix: \"\"\ndetectLocalMode: \"\"\nenableProfiling: false\nhealthzBindAddress: \"\"\nhostnameOverride: \"\"\niptables:\n  masqueradeAll: false\n  masqueradeBit: null\n  minSyncPeriod: 0s\n  syncPeriod: 0s\nipvs:\n  excludeCIDRs: null\n  minSyncPeriod: 0s\n  scheduler: \"\"\n  strictARP: false\n  syncPeriod: 0s\n  tcpFinTimeout: 0s\n  tcpTimeout: 0s\n  udpTimeout: 0s\nkind: KubeProxyConfiguration\nmetricsBindAddress: \"\"\nmode: \"\"\nnodePortAddresses: null\noomScoreAdj: null\nportRange: \"\"\nshowHiddenMetricsForVersion: \"\"\nudpIdleTimeout: 0s\nwinkernel:\n  enableDSR: false\n  forwardHealthCheckVip: false\n  networkName: \"\"\n  rootHnsEndpointName: \"\"\n  sourceVip: \"\"","kubeconfig.conf":"apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n    certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt\n    server: https://10.189.20.64:6443\n  name: default\ncontexts:\n- context:\n    cluster: default\n    namespace: default\n    user: default\n  name: default\ncurrent-context: default\nusers:\n- name: default\n  user:\n    tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token"}}
I0207 08:22:09.146474   46264 request.go:1154] Request Body: {"kind":"DaemonSet","apiVersion":"apps/v1","metadata":{"name":"kube-proxy","namespace":"kube-system","creationTimestamp":null,"labels":{"k8s-app":"kube-proxy"}},"spec":{"selector":{"matchLabels":{"k8s-app":"kube-proxy"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"k8s-app":"kube-proxy"}},"spec":{"volumes":[{"name":"kube-proxy","configMap":{"name":"kube-proxy"}},{"name":"xtables-lock","hostPath":{"path":"/run/xtables.lock","type":"FileOrCreate"}},{"name":"lib-modules","hostPath":{"path":"/lib/modules"}}],"containers":[{"name":"kube-proxy","image":"k8s.gcr.io/kube-proxy:v1.24.8","command":["/usr/local/bin/kube-proxy","--config=/var/lib/kube-proxy/config.conf","--hostname-override=$(NODE_NAME)"],"env":[{"name":"NODE_NAME","valueFrom":{"fieldRef":{"fieldPath":"spec.nodeName"}}}],"resources":{},"volumeMounts":[{"name":"kube-proxy","mountPath":"/var/lib/kube-proxy"},{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"lib-modules","readOnly":true,"mountPath":"/lib/modules"}],"imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}}],"nodeSelector":{"kubernetes.io/os":"linux"},"serviceAccountName":"kube-proxy","hostNetwork":true,"tolerations":[{"operator":"Exists"}],"priorityClassName":"system-node-critical"}},"updateStrategy":{"type":"RollingUpdate"}},"status":{"currentNumberScheduled":0,"numberMisscheduled":0,"desiredNumberScheduled":0,"numberReady":0}}
I0207 08:22:09.146557   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/apps/v1/namespaces/kube-system/daemonsets?timeout=10s'
I0207 08:22:09.150140   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/apps/v1/namespaces/kube-system/daemonsets?timeout=10s 201 Created in 3 milliseconds
I0207 08:22:09.150161   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 3 ms Duration 3 ms
I0207 08:22:09.150172   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.150184   46264 round_trippers.go:580]     Content-Length: 3662
I0207 08:22:09.150195   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.150206   46264 round_trippers.go:580]     Audit-Id: 626f6453-7fce-4b03-8a60-df3da4680d8b
I0207 08:22:09.150216   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.150227   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.150239   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.150250   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.150307   46264 request.go:1154] Response Body: {"kind":"DaemonSet","apiVersion":"apps/v1","metadata":{"name":"kube-proxy","namespace":"kube-system","uid":"c7bf8c66-187a-447c-ba9b-c5e0b157916f","resourceVersion":"235","generation":1,"creationTimestamp":"2023-02-07T00:22:09Z","labels":{"k8s-app":"kube-proxy"},"annotations":{"deprecated.daemonset.template.generation":"1"},"managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"apps/v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:deprecated.daemonset.template.generation":{}},"f:labels":{".":{},"f:k8s-app":{}}},"f:spec":{"f:revisionHistoryLimit":{},"f:selector":{},"f:template":{"f:metadata":{"f:labels":{".":{},"f:k8s-app":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"kube-proxy\"}":{".":{},"f:command":{},"f:env":{".":{},"k:{\"name\":\"NODE_NAME\"}":{".":{},"f:name":{},"f:valueFrom":{".":{},"f:fieldRef":{}}}},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{".":{},"f:privileged":{}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/lib/modules\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/run/xtables.lock\"}":{".":{},"f:mountPath":{},"f:name":{}},"k:{\"mountPath\":\"/var/lib/kube-proxy\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:hostNetwork":{},"f:nodeSelector":{},"f:priorityClassName":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:serviceAccount":{},"f:serviceAccountName":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{},"f:volumes":{".":{},"k:{\"name\":\"kube-proxy\"}":{".":{},"f:configMap":{".":{},"f:defaultMode":{},"f:name":{}},"f:name":{}},"k:{\"name\":\"lib-modules\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"xtables-lock\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}}}}},"f:updateStrategy":{"f:rollingUpdate":{".":{},"f:maxSurge":{},"f:maxUnavailable":{}},"f:type":{}}}}}]},"spec":{"selector":{"matchLabels":{"k8s-app":"kube-proxy"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"k8s-app":"kube-proxy"}},"spec":{"volumes":[{"name":"kube-proxy","configMap":{"name":"kube-proxy","defaultMode":420}},{"name":"xtables-lock","hostPath":{"path":"/run/xtables.lock","type":"FileOrCreate"}},{"name":"lib-modules","hostPath":{"path":"/lib/modules","type":""}}],"containers":[{"name":"kube-proxy","image":"k8s.gcr.io/kube-proxy:v1.24.8","command":["/usr/local/bin/kube-proxy","--config=/var/lib/kube-proxy/config.conf","--hostname-override=$(NODE_NAME)"],"env":[{"name":"NODE_NAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}],"resources":{},"volumeMounts":[{"name":"kube-proxy","mountPath":"/var/lib/kube-proxy"},{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"lib-modules","readOnly":true,"mountPath":"/lib/modules"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","nodeSelector":{"kubernetes.io/os":"linux"},"serviceAccountName":"kube-proxy","serviceAccount":"kube-proxy","hostNetwork":true,"securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists"}],"priorityClassName":"system-node-critical"}},"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":1,"maxSurge":0}},"revisionHistoryLimit":10},"status":{"currentNumberScheduled":0,"numberMisscheduled":0,"desiredNumberScheduled":0,"numberReady":0}}
I0207 08:22:09.150630   46264 request.go:1154] Request Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:node-proxier","creationTimestamp":null},"subjects":[{"kind":"ServiceAccount","name":"kube-proxy","namespace":"kube-system"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"system:node-proxier"}}
I0207 08:22:09.150705   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s'
I0207 08:22:09.152818   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 2 milliseconds
I0207 08:22:09.152845   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 2 ms
I0207 08:22:09.152861   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.152875   46264 round_trippers.go:580]     Content-Length: 618
I0207 08:22:09.152890   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.152904   46264 round_trippers.go:580]     Audit-Id: e574e8a9-9de9-4bbe-91b9-a8c597c36561
I0207 08:22:09.152918   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.152932   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.152947   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.152961   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.152987   46264 request.go:1154] Response Body: {"kind":"ClusterRoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:node-proxier","uid":"0fe85b9d-1087-42d4-9d76-77224bc39fd6","resourceVersion":"236","creationTimestamp":"2023-02-07T00:22:09Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"ServiceAccount","name":"kube-proxy","namespace":"kube-system"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"system:node-proxier"}}
I0207 08:22:09.153136   46264 request.go:1154] Request Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kube-proxy","namespace":"kube-system","creationTimestamp":null},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["kube-proxy"]}]}
I0207 08:22:09.153219   46264 round_trippers.go:466] curl -v -XPOST  -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s'
I0207 08:22:09.155103   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:09.155124   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:09.155136   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.155148   46264 round_trippers.go:580]     Audit-Id: 32c7f790-130d-435d-b0d5-ddc24bb32df4
I0207 08:22:09.155158   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.155169   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.155179   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.155190   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.155210   46264 round_trippers.go:580]     Content-Length: 519
I0207 08:22:09.155221   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.155244   46264 request.go:1154] Response Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kube-proxy","namespace":"kube-system","uid":"56c71267-2abb-45eb-bd54-ff44b85e2620","resourceVersion":"237","creationTimestamp":"2023-02-07T00:22:09Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:rules":{}}}]},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["kube-proxy"]}]}
I0207 08:22:09.155374   46264 request.go:1154] Request Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kube-proxy","namespace":"kube-system","creationTimestamp":null},"subjects":[{"kind":"Group","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kube-proxy"}}
I0207 08:22:09.309778   46264 request.go:614] Waited for 154.307543ms due to client-side throttling, not priority and fairness, request: POST:https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s
I0207 08:22:09.309954   46264 round_trippers.go:466] curl -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s'
I0207 08:22:09.313168   46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 3 milliseconds
I0207 08:22:09.313192   46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 3 ms
I0207 08:22:09.313206   46264 round_trippers.go:577] Response Headers:
I0207 08:22:09.313223   46264 round_trippers.go:580]     Date: Tue, 07 Feb 2023 00:22:09 GMT
I0207 08:22:09.313236   46264 round_trippers.go:580]     Audit-Id: 7a390ef3-f193-4826-8a5d-f2d0d87596b1
I0207 08:22:09.313247   46264 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 08:22:09.313265   46264 round_trippers.go:580]     Content-Type: application/json
I0207 08:22:09.313283   46264 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:09.313301   46264 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:09.313337   46264 round_trippers.go:580]     Content-Length: 652
I0207 08:22:09.313390   46264 request.go:1154] Response Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kube-proxy","namespace":"kube-system","uid":"73f12586-c082-4c90-9beb-5088fade4fe4","resourceVersion":"241","creationTimestamp":"2023-02-07T00:22:09Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kube-proxy"}}
[addons] Applied essential addon: kube-proxy
I0207 08:22:09.314280   46264 loader.go:372] Config loaded from file:  /etc/kubernetes/admin.conf
I0207 08:22:09.314997   46264 loader.go:372] Config loaded from file:  /etc/kubernetes/admin.conf

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.189.20.64:6443 --token mytoken \
        --discovery-token-ca-cert-hash sha256:mysha256

fannel 网络

由于网络还没有配置，因此会有 pod 处于 pending 状态

[root@ns-yun-020064 kubevirt]# kubectl  get pod -n kube-system
NAME                                                READY   STATUS    RESTARTS   AGE
coredns-6d4b75cb6d-6x8ln                            0/1     Pending   0          48m
coredns-6d4b75cb6d-f7h6n                            0/1     Pending   0          48m
etcd-ns-yun-020064.vclound.com                      1/1     Running   0          48m
kube-apiserver-ns-yun-020064.vclound.com            1/1     Running   0          48m
kube-controller-manager-ns-yun-020064.vclound.com   1/1     Running   0          48m
kube-proxy-2pgjb                                    1/1     Running   0          48m
kube-scheduler-ns-yun-020064.vclound.com            1/1     Running   0          48m

所需镜像

加入 master 前准备

查询 master token

# kubeadm token list
TOKEN                     TTL         EXPIRES                USAGES                   DESCRIPTION                                                EXTRA GROUPS
xxxxxsssssssssssxxxxxxx   22h         2023-02-08T00:22:08Z   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token

假如 token 过期， 需要重建 token

# kubeadm token create
xxxxxxx
# kubeadm token list
TOKEN                     TTL         EXPIRES                USAGES                   DESCRIPTION                                                EXTRA GROUPS
xxxsssssssssssssssssxxx   22h         2023-02-08T00:22:08Z   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token
xxxxxxxssssssssssssssxxx  23h         2023-02-08T01:39:42Z   authentication,signing   <none>                                                     system:bootstrappers:kubeadm:default-node-token

获取 sha256 key

# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
5962e11ae392853ebb91f3092c9528528664b8e1e63fc346af5ae08c9c18e083

加入 master

# kubeadm join 10.189.20.64:6443 --token u4wj48.g0x1x8gc5ogtnd0g  --discovery-token-ca-cert-hash sha256:5962e11ae392853ebb91f3092c9528528664b8e1e63fc346af5ae08c9c18e083  --v=9

I0207 09:46:53.423490   41694 join.go:413] [preflight] found NodeName empty; using OS hostname as NodeName
I0207 09:46:53.423656   41694 initconfiguration.go:117] detected and using CRI socket: unix:///var/run/containerd/containerd.sock
[preflight] Running pre-flight checks
I0207 09:46:53.423728   41694 preflight.go:92] [preflight] Running general checks
I0207 09:46:53.423768   41694 checks.go:282] validating the existence of file /etc/kubernetes/kubelet.conf
I0207 09:46:53.423780   41694 checks.go:282] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I0207 09:46:53.423800   41694 checks.go:106] validating the container runtime
I0207 09:46:53.453132   41694 checks.go:331] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I0207 09:46:53.453217   41694 checks.go:331] validating the contents of file /proc/sys/net/ipv4/ip_forward
I0207 09:46:53.453250   41694 checks.go:646] validating whether swap is enabled or not
I0207 09:46:53.453317   41694 checks.go:372] validating the presence of executable crictl
I0207 09:46:53.453360   41694 checks.go:372] validating the presence of executable conntrack
I0207 09:46:53.453386   41694 checks.go:372] validating the presence of executable ip
I0207 09:46:53.453403   41694 checks.go:372] validating the presence of executable iptables
I0207 09:46:53.453423   41694 checks.go:372] validating the presence of executable mount
I0207 09:46:53.453444   41694 checks.go:372] validating the presence of executable nsenter
I0207 09:46:53.453476   41694 checks.go:372] validating the presence of executable ebtables
I0207 09:46:53.453495   41694 checks.go:372] validating the presence of executable ethtool
I0207 09:46:53.453517   41694 checks.go:372] validating the presence of executable socat
I0207 09:46:53.453534   41694 checks.go:372] validating the presence of executable tc
I0207 09:46:53.453550   41694 checks.go:372] validating the presence of executable touch
I0207 09:46:53.453580   41694 checks.go:518] running all checks
I0207 09:46:53.462911   41694 checks.go:403] checking whether the given node name is valid and reachable using net.LookupHost
I0207 09:46:53.463090   41694 checks.go:612] validating kubelet version
I0207 09:46:53.520011   41694 checks.go:132] validating if the "kubelet" service is enabled and active
I0207 09:46:53.536919   41694 checks.go:205] validating availability of port 10250
I0207 09:46:53.537134   41694 checks.go:282] validating the existence of file /etc/kubernetes/pki/ca.crt
I0207 09:46:53.537152   41694 checks.go:432] validating if the connectivity type is via proxy or direct
I0207 09:46:53.537182   41694 join.go:530] [preflight] Discovering cluster-info
I0207 09:46:53.537218   41694 token.go:80] [discovery] Created cluster-info discovery client, requesting info from "10.189.20.64:6443"
I0207 09:46:53.537804   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s'
I0207 09:46:53.538155   41694 round_trippers.go:510] HTTP Trace: Dial to tcp:10.189.20.64:6443 succeed
I0207 09:46:53.555023   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 17 milliseconds
I0207 09:46:53.555051   41694 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 4 ms ServerProcessing 11 ms Duration 17 ms
I0207 09:46:53.555062   41694 round_trippers.go:577] Response Headers:
I0207 09:46:53.555077   41694 round_trippers.go:580]     Audit-Id: 98d909e9-c3fc-46a6-bee8-df65f845b780
I0207 09:46:53.555089   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:46:53.555100   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:46:53.555111   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 85695c0f-bcde-4eb1-9a66-d43a127635b1
I0207 09:46:53.555122   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 2f005fc8-60c8-4580-978e-e09b7606411b
I0207 09:46:53.555134   41694 round_trippers.go:580]     Content-Length: 2534
I0207 09:46:53.555144   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:46:53 GMT
I0207 09:46:53.555199   41694 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"cluster-info","namespace":"kube-public","uid":"c8d08668-6dbf-4cba-85a0-fee86d04140c","resourceVersion":"5963","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:kubeconfig":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2023-02-07T01:39:42Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{"f:jws-kubeconfig-8foyh9":{},"f:jws-kubeconfig-u4wj48":{}}}}]},"data":{"jws-kubeconfig-8foyh9":"eyJhbGciOiJIUzI1NiIsImtpZCI6Ijhmb3loOSJ9.._gTdzkghhYnowgfCAyRsT-s_fxjPg0ONgDV6lRGxxAU","jws-kubeconfig-u4wj48":"eyJhbGciOiJIUzI1NiIsImtpZCI6InU0d2o0OCJ9..uFnCIPSFDU57rMV_vdLptB9kJpJCAOfvsm3HYiKoYCc","kubeconfig":"apiVersion: v1\nclusters:\n- cluster:\n    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJek1ESXdOekF3TWpFMU4xb1hEVE16TURJd05EQXdNakUxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTHExCk9iRUdDR0JWbldldEpTNFNJU0J6R0s2bGthamVSczV5MmhCVnZVZVpEb3BWZUVrNjNycURNc3UwZDN5MXJqUzEKeUMxeG9hcjJIVVM4NWpvVmpmMzFsY2xCS3NGTWJkWkE2T2VBdk1oMGFjcjkrRVBYMzUzRGlNRndnT0poSjYrQwpWRm53MEpZeFlOeXB3MDFWUURuZ1ZGK0lKY29GRllZSEVmbGRWTVhFRldhN1RrYjVPRXhTVGZhMjhvaFdJQzdiClNPWjhvMmdXdGhWQ3k0SzBXb3pVYkdrQ3plYTAyU1lJTlJzNjhQTERZMmlkUUx6dG5QUWltWHJ1UzM2RFpOUFYKWXBLQzRpQlEvNC9UV0VOdnhqWFZUUldyckdDQ2NGVzdpS0lwMFIveTZBSTkwUUVJTW5uMVlVODNmb2s1ZmJ2YwppMWh0a3FsUXNhWXJ4bzJHbzBNQ0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOZXNmT1M1MmtvcFZ2aG9UbmdYeGRtai8zOStNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBQzQ3ZFRYV2YwRGJDS0JheG1qZwpyZk0zdU5POUJxR004TklrSjNjMkR6c2JoUHFVL25wTXlleVZEOEdZSzR6RGVEckl3TzFGSWlYRXBDYzdhM2RwCjFwSmhkekI0N0lPWVNIY2pGS05iY0IreUxXdDMzQ3JRd2NoRHNwdkVmOUdjSjk2VStRRFh5S0o5YmFGZmhxVTgKSkR4Y05UcTU5RjNUeUk1SDcxTHhhdkRDb1B2WlN6S1NXTGtOUGRjd3FJREoxdXVDVkpNNC94dWpHN2tSMVZUSwptWUdGb0pWVTA3TTRpQm8xNXFGd1U1RmkyM0NUU3dHY1h6eHVrOGViY1FWVDl3aTZMQjQyaGk1bHhXbVcrR3hEClVOaHhoK0laQTA3QzdtOUlnSDRXaUgzQjNTTXZZNGtEYlJMTzlmdVhvbGVBUDQreWlXY1BmM0NLT3NYdkRHbC8KZDBVPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==\n    server: https://10.189.20.64:6443\n  name: \"\"\ncontexts: null\ncurrent-context: \"\"\nkind: Config\npreferences: {}\nusers: null\n"}}
I0207 09:46:53.556220   41694 token.go:118] [discovery] Requesting info from "10.189.20.64:6443" again to validate TLS against the pinned public key
I0207 09:46:53.556613   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s'
I0207 09:46:53.556829   41694 round_trippers.go:510] HTTP Trace: Dial to tcp:10.189.20.64:6443 succeed
I0207 09:46:53.562955   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 6 milliseconds
I0207 09:46:53.562981   41694 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 4 ms ServerProcessing 1 ms Duration 6 ms
I0207 09:46:53.562994   41694 round_trippers.go:577] Response Headers:
I0207 09:46:53.563006   41694 round_trippers.go:580]     Content-Length: 2534
I0207 09:46:53.563018   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:46:53 GMT
I0207 09:46:53.563029   41694 round_trippers.go:580]     Audit-Id: a4f8e513-e3f7-456a-8233-1961fc2ce15c
I0207 09:46:53.563045   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:46:53.563061   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:46:53.563073   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 85695c0f-bcde-4eb1-9a66-d43a127635b1
I0207 09:46:53.563084   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 2f005fc8-60c8-4580-978e-e09b7606411b
I0207 09:46:53.563186   41694 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"cluster-info","namespace":"kube-public","uid":"c8d08668-6dbf-4cba-85a0-fee86d04140c","resourceVersion":"5963","creationTimestamp":"2023-02-07T00:22:08Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:08Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:kubeconfig":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2023-02-07T01:39:42Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{"f:jws-kubeconfig-8foyh9":{},"f:jws-kubeconfig-u4wj48":{}}}}]},"data":{"jws-kubeconfig-8foyh9":"eyJhbGciOiJIUzI1NiIsImtpZCI6Ijhmb3loOSJ9.._gTdzkghhYnowgfCAyRsT-s_fxjPg0ONgDV6lRGxxAU","jws-kubeconfig-u4wj48":"eyJhbGciOiJIUzI1NiIsImtpZCI6InU0d2o0OCJ9..uFnCIPSFDU57rMV_vdLptB9kJpJCAOfvsm3HYiKoYCc","kubeconfig":"apiVersion: v1\nclusters:\n- cluster:\n    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJek1ESXdOekF3TWpFMU4xb1hEVE16TURJd05EQXdNakUxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTHExCk9iRUdDR0JWbldldEpTNFNJU0J6R0s2bGthamVSczV5MmhCVnZVZVpEb3BWZUVrNjNycURNc3UwZDN5MXJqUzEKeUMxeG9hcjJIVVM4NWpvVmpmMzFsY2xCS3NGTWJkWkE2T2VBdk1oMGFjcjkrRVBYMzUzRGlNRndnT0poSjYrQwpWRm53MEpZeFlOeXB3MDFWUURuZ1ZGK0lKY29GRllZSEVmbGRWTVhFRldhN1RrYjVPRXhTVGZhMjhvaFdJQzdiClNPWjhvMmdXdGhWQ3k0SzBXb3pVYkdrQ3plYTAyU1lJTlJzNjhQTERZMmlkUUx6dG5QUWltWHJ1UzM2RFpOUFYKWXBLQzRpQlEvNC9UV0VOdnhqWFZUUldyckdDQ2NGVzdpS0lwMFIveTZBSTkwUUVJTW5uMVlVODNmb2s1ZmJ2YwppMWh0a3FsUXNhWXJ4bzJHbzBNQ0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOZXNmT1M1MmtvcFZ2aG9UbmdYeGRtai8zOStNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBQzQ3ZFRYV2YwRGJDS0JheG1qZwpyZk0zdU5POUJxR004TklrSjNjMkR6c2JoUHFVL25wTXlleVZEOEdZSzR6RGVEckl3TzFGSWlYRXBDYzdhM2RwCjFwSmhkekI0N0lPWVNIY2pGS05iY0IreUxXdDMzQ3JRd2NoRHNwdkVmOUdjSjk2VStRRFh5S0o5YmFGZmhxVTgKSkR4Y05UcTU5RjNUeUk1SDcxTHhhdkRDb1B2WlN6S1NXTGtOUGRjd3FJREoxdXVDVkpNNC94dWpHN2tSMVZUSwptWUdGb0pWVTA3TTRpQm8xNXFGd1U1RmkyM0NUU3dHY1h6eHVrOGViY1FWVDl3aTZMQjQyaGk1bHhXbVcrR3hEClVOaHhoK0laQTA3QzdtOUlnSDRXaUgzQjNTTXZZNGtEYlJMTzlmdVhvbGVBUDQreWlXY1BmM0NLT3NYdkRHbC8KZDBVPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==\n    server: https://10.189.20.64:6443\n  name: \"\"\ncontexts: null\ncurrent-context: \"\"\nkind: Config\npreferences: {}\nusers: null\n"}}
I0207 09:46:53.563762   41694 token.go:135] [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "10.189.20.64:6443"
I0207 09:46:53.563780   41694 discovery.go:52] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
I0207 09:46:53.563806   41694 join.go:544] [preflight] Fetching init configuration
I0207 09:46:53.563819   41694 join.go:590] [preflight] Retrieving KubeConfig objects
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
I0207 09:46:53.564358   41694 round_trippers.go:466] curl -v -XGET  -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" -H "Authorization: Bearer <masked>" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s'
I0207 09:46:53.566186   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s 200 OK in 1 milliseconds
I0207 09:46:53.566221   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 09:46:53.566243   41694 round_trippers.go:577] Response Headers:
I0207 09:46:53.566259   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 2f005fc8-60c8-4580-978e-e09b7606411b
I0207 09:46:53.566273   41694 round_trippers.go:580]     Content-Length: 910
I0207 09:46:53.566288   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:46:53 GMT
I0207 09:46:53.566302   41694 round_trippers.go:580]     Audit-Id: b26a3266-d49f-417e-b1f1-b704e42a630b
I0207 09:46:53.566315   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:46:53.566329   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:46:53.566343   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 85695c0f-bcde-4eb1-9a66-d43a127635b1
I0207 09:46:53.566383   41694 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubeadm-config","namespace":"kube-system","uid":"ea94739b-1e8c-4a2f-ad2b-0b9c1769816d","resourceVersion":"204","creationTimestamp":"2023-02-07T00:22:07Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:ClusterConfiguration":{}}}}]},"data":{"ClusterConfiguration":"apiServer:\n  extraArgs:\n    authorization-mode: Node,RBAC\n  timeoutForControlPlane: 4m0s\napiVersion: kubeadm.k8s.io/v1beta3\ncertificatesDir: /etc/kubernetes/pki\nclusterName: kubernetes\ncontrollerManager: {}\ndns: {}\netcd:\n  local:\n    dataDir: /var/lib/etcd\nimageRepository: k8s.gcr.io\nkind: ClusterConfiguration\nkubernetesVersion: v1.24.8\nnetworking:\n  dnsDomain: cluster.local\n  podSubnet: 10.244.0.0/16\n  serviceSubnet: 10.96.0.0/12\nscheduler: {}\n"}}
I0207 09:46:53.567364   41694 round_trippers.go:466] curl -v -XGET  -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" -H "Authorization: Bearer <masked>" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps/kube-proxy?timeout=10s'
I0207 09:46:53.568927   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps/kube-proxy?timeout=10s 200 OK in 1 milliseconds
I0207 09:46:53.568950   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 09:46:53.568964   41694 round_trippers.go:577] Response Headers:
I0207 09:46:53.568977   41694 round_trippers.go:580]     Content-Length: 2305
I0207 09:46:53.568988   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:46:53 GMT
I0207 09:46:53.568999   41694 round_trippers.go:580]     Audit-Id: e8784a5b-9976-4ac5-acd3-8fef58f9c71a
I0207 09:46:53.569010   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:46:53.569021   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:46:53.569032   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 85695c0f-bcde-4eb1-9a66-d43a127635b1
I0207 09:46:53.569043   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 2f005fc8-60c8-4580-978e-e09b7606411b
I0207 09:46:53.569092   41694 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kube-proxy","namespace":"kube-system","uid":"594a7c08-bbf1-43f5-9dc5-8c7940fccb04","resourceVersion":"234","creationTimestamp":"2023-02-07T00:22:09Z","labels":{"app":"kube-proxy"},"annotations":{"kubeadm.kubernetes.io/component-config.hash":"sha256:f6d0a8dd44c5a84730118c89ecf9edd3cd167f79c2e04a3e9859ef39e8094371"},"managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:09Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:config.conf":{},"f:kubeconfig.conf":{}},"f:metadata":{"f:annotations":{".":{},"f:kubeadm.kubernetes.io/component-config.hash":{}},"f:labels":{".":{},"f:app":{}}}}}]},"data":{"config.conf":"apiVersion: kubeproxy.config.k8s.io/v1alpha1\nbindAddress: 0.0.0.0\nbindAddressHardFail: false\nclientConnection:\n  acceptContentTypes: \"\"\n  burst: 0\n  contentType: \"\"\n  kubeconfig: /var/lib/kube-proxy/kubeconfig.conf\n  qps: 0\nclusterCIDR: 10.244.0.0/16\nconfigSyncPeriod: 0s\nconntrack:\n  maxPerCore: null\n  min: null\n  tcpCloseWaitTimeout: null\n  tcpEstablishedTimeout: null\ndetectLocal:\n  bridgeInterface: \"\"\n  interfaceNamePrefix: \"\"\ndetectLocalMode: \"\"\nenableProfiling: false\nhealthzBindAddress: \"\"\nhostnameOverride: \"\"\niptables:\n  masqueradeAll: false\n  masqueradeBit: null\n  minSyncPeriod: 0s\n  syncPeriod: 0s\nipvs:\n  excludeCIDRs: null\n  minSyncPeriod: 0s\n  scheduler: \"\"\n  strictARP: false\n  syncPeriod: 0s\n  tcpFinTimeout: 0s\n  tcpTimeout: 0s\n  udpTimeout: 0s\nkind: KubeProxyConfiguration\nmetricsBindAddress: \"\"\nmode: \"\"\nnodePortAddresses: null\noomScoreAdj: null\nportRange: \"\"\nshowHiddenMetricsForVersion: \"\"\nudpIdleTimeout: 0s\nwinkernel:\n  enableDSR: false\n  forwardHealthCheckVip: false\n  networkName: \"\"\n  rootHnsEndpointName: \"\"\n  sourceVip: \"\"","kubeconfig.conf":"apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n    certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt\n    server: https://10.189.20.64:6443\n  name: default\ncontexts:\n- context:\n    cluster: default\n    namespace: default\n    user: default\n  name: default\ncurrent-context: default\nusers:\n- name: default\n  user:\n    tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token"}}
I0207 09:46:53.570836   41694 kubelet.go:92] attempting to download the KubeletConfiguration from the new format location (UnversionedKubeletConfigMap=true)
I0207 09:46:53.570936   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Authorization: Bearer <masked>" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config?timeout=10s'
I0207 09:46:53.572421   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config?timeout=10s 200 OK in 1 milliseconds
I0207 09:46:53.572452   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 09:46:53.572463   41694 round_trippers.go:577] Response Headers:
I0207 09:46:53.572475   41694 round_trippers.go:580]     Audit-Id: 022b8274-a137-46dd-ba5c-a9e4070007b1
I0207 09:46:53.572486   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:46:53.572502   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:46:53.572517   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 85695c0f-bcde-4eb1-9a66-d43a127635b1
I0207 09:46:53.572534   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 2f005fc8-60c8-4580-978e-e09b7606411b
I0207 09:46:53.572546   41694 round_trippers.go:580]     Content-Length: 1657
I0207 09:46:53.572561   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:46:53 GMT
I0207 09:46:53.572614   41694 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubelet-config","namespace":"kube-system","uid":"c6277399-bbe5-4b2d-ad78-94a120465f7e","resourceVersion":"207","creationTimestamp":"2023-02-07T00:22:07Z","annotations":{"kubeadm.kubernetes.io/component-config.hash":"sha256:53821686a8ddaca8403df63803a726d2ab31b145c7da69ed7b78c45d2e223a18"},"managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:kubelet":{}},"f:metadata":{"f:annotations":{".":{},"f:kubeadm.kubernetes.io/component-config.hash":{}}}}}]},"data":{"kubelet":"apiVersion: kubelet.config.k8s.io/v1beta1\nauthentication:\n  anonymous:\n    enabled: false\n  webhook:\n    cacheTTL: 0s\n    enabled: true\n  x509:\n    clientCAFile: /etc/kubernetes/pki/ca.crt\nauthorization:\n  mode: Webhook\n  webhook:\n    cacheAuthorizedTTL: 0s\n    cacheUnauthorizedTTL: 0s\ncgroupDriver: systemd\nclusterDNS:\n- 10.96.0.10\nclusterDomain: cluster.local\ncpuManagerReconcilePeriod: 0s\nevictionPressureTransitionPeriod: 0s\nfileCheckFrequency: 0s\nhealthzBindAddress: 127.0.0.1\nhealthzPort: 10248\nhttpCheckFrequency: 0s\nimageMinimumGCAge: 0s\nkind: KubeletConfiguration\nlogging:\n  flushFrequency: 0\n  options:\n    json:\n      infoBufferSize: \"0\"\n  verbosity: 0\nmemorySwap: {}\nnodeStatusReportFrequency: 0s\nnodeStatusUpdateFrequency: 0s\nrotateCertificates: true\nruntimeRequestTimeout: 0s\nshutdownGracePeriod: 0s\nshutdownGracePeriodCriticalPods: 0s\nstaticPodPath: /etc/kubernetes/manifests\nstreamingConnectionIdleTimeout: 0s\nsyncFrequency: 0s\nvolumeStatsAggPeriod: 0s\n"}}
I0207 09:46:53.575186   41694 interface.go:432] Looking for default routes with IPv4 addresses
I0207 09:46:53.575204   41694 interface.go:437] Default route transits interface "bond0.20"
I0207 09:46:53.575677   41694 interface.go:209] Interface bond0.20 is up
I0207 09:46:53.575736   41694 interface.go:257] Interface "bond0.20" has 2 addresses :[10.189.20.65/24 fe80::ee38:8fff:fe79:2726/64].
I0207 09:46:53.575769   41694 interface.go:224] Checking addr  10.189.20.65/24.
I0207 09:46:53.575783   41694 interface.go:231] IP found 10.189.20.65
I0207 09:46:53.575809   41694 interface.go:263] Found valid IPv4 address 10.189.20.65 for interface "bond0.20".
I0207 09:46:53.575822   41694 interface.go:443] Found active IP 10.189.20.65
I0207 09:46:53.595058   41694 preflight.go:103] [preflight] Running configuration dependant checks
I0207 09:46:53.595083   41694 controlplaneprepare.go:220] [download-certs] Skipping certs download
I0207 09:46:53.595096   41694 kubelet.go:119] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
I0207 09:46:53.595938   41694 kubelet.go:134] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
I0207 09:46:53.596387   41694 loader.go:372] Config loaded from file:  /etc/kubernetes/bootstrap-kubelet.conf
I0207 09:46:53.597309   41694 kubelet.go:155] [kubelet-start] Checking for an existing Node in the cluster with name "ns-yun-020065.vclound.com" and status "Ready"
I0207 09:46:53.597569   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Authorization: Bearer <masked>" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:46:53.599312   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 1 milliseconds
I0207 09:46:53.599342   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 09:46:53.599352   41694 round_trippers.go:577] Response Headers:
I0207 09:46:53.599363   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:46:53 GMT
I0207 09:46:53.599374   41694 round_trippers.go:580]     Audit-Id: eb935141-22eb-421d-a6d8-12ea4b9ddeb9
I0207 09:46:53.599386   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:46:53.599397   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:46:53.599408   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 85695c0f-bcde-4eb1-9a66-d43a127635b1
I0207 09:46:53.599418   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 2f005fc8-60c8-4580-978e-e09b7606411b
I0207 09:46:53.599429   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:46:53.599452   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:46:53.599657   41694 kubelet.go:170] [kubelet-start] Stopping the kubelet
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
I0207 09:46:58.815014   41694 loader.go:372] Config loaded from file:  /etc/kubernetes/kubelet.conf
I0207 09:46:58.815934   41694 cert_rotation.go:137] Starting client certificate rotation controller
I0207 09:46:58.816335   41694 loader.go:372] Config loaded from file:  /etc/kubernetes/kubelet.conf
I0207 09:46:58.816720   41694 kubelet.go:218] [kubelet-start] preserving the crisocket information for the node
I0207 09:46:58.816737   41694 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///var/run/containerd/containerd.sock" to the Node API object "ns-yun-020065.vclound.com" as an annotation
I0207 09:46:59.317578   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:46:59.317939   41694 round_trippers.go:510] HTTP Trace: Dial to tcp:10.189.20.64:6443 succeed
I0207 09:46:59.325295   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 7 milliseconds
I0207 09:46:59.325320   41694 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 4 ms ServerProcessing 2 ms Duration 7 ms
I0207 09:46:59.325340   41694 round_trippers.go:577] Response Headers:
I0207 09:46:59.325360   41694 round_trippers.go:580]     Audit-Id: 2b6076e0-66ee-4ddd-b1b7-db85f10064b7
I0207 09:46:59.325382   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:46:59.325402   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:46:59.325420   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:46:59.325441   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:46:59.325462   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:46:59.325482   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:46:59 GMT
I0207 09:46:59.325537   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:46:59.816926   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:46:59.820124   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 3 milliseconds
I0207 09:46:59.820145   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 3 ms Duration 3 ms
I0207 09:46:59.820156   41694 round_trippers.go:577] Response Headers:
I0207 09:46:59.820168   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:46:59.820179   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:46:59.820190   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:46:59.820200   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:46:59.820211   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:46:59.820229   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:46:59 GMT
I0207 09:46:59.820240   41694 round_trippers.go:580]     Audit-Id: 29154d24-0045-4bf2-89a4-df257ded6dab
I0207 09:46:59.820275   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:00.317892   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:00.320944   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 3 milliseconds
I0207 09:47:00.320965   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 3 ms
I0207 09:47:00.320976   41694 round_trippers.go:577] Response Headers:
I0207 09:47:00.320988   41694 round_trippers.go:580]     Audit-Id: ca1030a1-0e9b-46af-bfcc-d1d137b8e678
I0207 09:47:00.320999   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:00.321010   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:00.321020   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:00.321031   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:00.321042   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:00.321052   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:00 GMT
I0207 09:47:00.321072   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:00.817692   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:00.820549   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 2 milliseconds
I0207 09:47:00.820571   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 09:47:00.820590   41694 round_trippers.go:577] Response Headers:
I0207 09:47:00.820610   41694 round_trippers.go:580]     Audit-Id: 508a106d-f789-4a87-8249-8a547bb60a42
I0207 09:47:00.820630   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:00.820652   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:00.820671   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:00.820694   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:00.820714   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:00.820735   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:00 GMT
I0207 09:47:00.820769   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:01.317184   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:01.319995   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 2 milliseconds
I0207 09:47:01.320029   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 09:47:01.320044   41694 round_trippers.go:577] Response Headers:
I0207 09:47:01.320056   41694 round_trippers.go:580]     Audit-Id: 6f0ffc6a-180e-4b4e-9e55-a2c40fc4a94b
I0207 09:47:01.320068   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:01.320079   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:01.320090   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:01.320101   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:01.320113   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:01.320129   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:01 GMT
I0207 09:47:01.320151   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:01.817736   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:01.820359   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 2 milliseconds
I0207 09:47:01.820380   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 09:47:01.820398   41694 round_trippers.go:577] Response Headers:
I0207 09:47:01.820418   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:01.820440   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:01 GMT
I0207 09:47:01.820461   41694 round_trippers.go:580]     Audit-Id: 07becaa2-2839-4ccd-8aa4-18118ad70309
I0207 09:47:01.820479   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:01.820499   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:01.820520   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:01.820540   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:01.820573   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:02.316981   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:02.319958   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 2 milliseconds
I0207 09:47:02.319978   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 09:47:02.319990   41694 round_trippers.go:577] Response Headers:
I0207 09:47:02.320002   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:02.320012   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:02 GMT
I0207 09:47:02.320023   41694 round_trippers.go:580]     Audit-Id: 5c78c5b1-9622-43ae-868d-a1507d2cb269
I0207 09:47:02.320033   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:02.320044   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:02.320055   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:02.320067   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:02.320088   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:02.817693   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:02.820777   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 3 milliseconds
I0207 09:47:02.820808   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 3 ms
I0207 09:47:02.820820   41694 round_trippers.go:577] Response Headers:
I0207 09:47:02.820832   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:02.820843   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:02.820854   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:02.820864   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:02 GMT
I0207 09:47:02.820875   41694 round_trippers.go:580]     Audit-Id: 4ead0070-d607-4072-8087-4573e9ab6650
I0207 09:47:02.820885   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:02.820896   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:02.820920   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:03.317541   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:03.320285   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 2 milliseconds
I0207 09:47:03.320306   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 09:47:03.320317   41694 round_trippers.go:577] Response Headers:
I0207 09:47:03.320328   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:03.320346   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:03 GMT
I0207 09:47:03.320366   41694 round_trippers.go:580]     Audit-Id: 83295dc5-ddfd-4e35-86a9-63d5d22bddd8
I0207 09:47:03.320386   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:03.320406   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:03.320427   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:03.320447   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:03.320485   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:03.816944   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:03.820061   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 3 milliseconds
I0207 09:47:03.820083   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 3 ms Duration 3 ms
I0207 09:47:03.820093   41694 round_trippers.go:577] Response Headers:
I0207 09:47:03.820104   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:03 GMT
I0207 09:47:03.820115   41694 round_trippers.go:580]     Audit-Id: 9dd9ab4d-c462-40d0-8dd9-753d5abd1dee
I0207 09:47:03.820126   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:03.820137   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:03.820148   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:03.820158   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:03.820169   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:03.820190   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:04.317804   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:04.320889   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 3 milliseconds
I0207 09:47:04.320909   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 3 ms
I0207 09:47:04.320920   41694 round_trippers.go:577] Response Headers:
I0207 09:47:04.320930   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:04.320942   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:04.320952   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:04.320963   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:04.320980   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:04.320993   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:04 GMT
I0207 09:47:04.321004   41694 round_trippers.go:580]     Audit-Id: 932b42bd-b804-4318-8d4c-f829c3390413
I0207 09:47:04.321040   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:04.817624   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:04.819682   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 2 milliseconds
I0207 09:47:04.819710   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 2 ms
I0207 09:47:04.819733   41694 round_trippers.go:577] Response Headers:
I0207 09:47:04.819748   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:04.819764   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:04.819778   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:04 GMT
I0207 09:47:04.819804   41694 round_trippers.go:580]     Audit-Id: 843f340a-5553-4b32-b7a2-30e4c33f9a1b
I0207 09:47:04.819820   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:04.819834   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:04.819848   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:04.819872   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:05.316939   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:05.319218   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 2 milliseconds
I0207 09:47:05.319238   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 09:47:05.319249   41694 round_trippers.go:577] Response Headers:
I0207 09:47:05.319260   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:05.319271   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:05.319282   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:05.319296   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:05 GMT
I0207 09:47:05.319307   41694 round_trippers.go:580]     Audit-Id: 56e90d0c-fe95-42fb-9957-50c62adf40f7
I0207 09:47:05.319318   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:05.319328   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:05.319348   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:05.816872   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:05.819211   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 2 milliseconds
I0207 09:47:05.819233   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 09:47:05.819244   41694 round_trippers.go:577] Response Headers:
I0207 09:47:05.819255   41694 round_trippers.go:580]     Audit-Id: 3130a91a-213b-4949-9dcb-caead4b70631
I0207 09:47:05.819266   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:05.819277   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:05.819288   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:05.819299   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:05.819310   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:05.819320   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:05 GMT
I0207 09:47:05.819341   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:06.317917   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:06.319733   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 1 milliseconds
I0207 09:47:06.319755   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 09:47:06.319765   41694 round_trippers.go:577] Response Headers:
I0207 09:47:06.319776   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:06.319800   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:06.319819   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:06 GMT
I0207 09:47:06.319836   41694 round_trippers.go:580]     Audit-Id: f7df1b08-d300-4f1e-82eb-fed624ff2db7
I0207 09:47:06.319851   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:06.319863   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:06.319874   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:06.319895   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:06.817346   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:06.819488   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 404 Not Found in 2 milliseconds
I0207 09:47:06.819510   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 2 ms Duration 2 ms
I0207 09:47:06.819520   41694 round_trippers.go:577] Response Headers:
I0207 09:47:06.819531   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:06.819542   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:06.819553   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:06.819564   41694 round_trippers.go:580]     Content-Length: 220
I0207 09:47:06.819575   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:06 GMT
I0207 09:47:06.819585   41694 round_trippers.go:580]     Audit-Id: cdf83284-b302-4fee-87d0-44e9d9f9a142
I0207 09:47:06.819596   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:06.819617   41694 request.go:1154] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"ns-yun-020065.vclound.com\" not found","reason":"NotFound","details":{"name":"ns-yun-020065.vclound.com","kind":"nodes"},"code":404}
I0207 09:47:07.317109   41694 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:07.319009   41694 round_trippers.go:553] GET https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 200 OK in 1 milliseconds
I0207 09:47:07.319029   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 09:47:07.319041   41694 round_trippers.go:577] Response Headers:
I0207 09:47:07.319053   41694 round_trippers.go:580]     Audit-Id: 2b621425-5bf8-443b-966d-213399c51db5
I0207 09:47:07.319064   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:07.319075   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:07.319085   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:07.319096   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:07.319107   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:07 GMT
I0207 09:47:07.319182   41694 request.go:1154] Response Body: {"kind":"Node","apiVersion":"v1","metadata":{"name":"ns-yun-020065.vclound.com","uid":"4657eb62-4174-4b22-bbd4-3bf915711404","resourceVersion":"6508","creationTimestamp":"2023-02-07T01:47:07Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"ns-yun-020065.vclound.com","kubernetes.io/os":"linux"},"annotations":{"node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"},"managedFields":[{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2023-02-07T01:47:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2023-02-07T01:47:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}}}]},"spec":{"podCIDR":"10.244.1.0/24","podCIDRs":["10.244.1.0/24"],"taints":[{"key":"node.kubernetes.io/not-ready","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"24","ephemeral-storage":"898781408Ki","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131850144Ki","pods":"110"},"allocatable":{"cpu":"24","ephemeral-storage":"828316944242","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131747744Ki","pods":"110"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2023-02-07T01:47:07Z","lastTransitionTime":"2023-02-07T01:47:07Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2023-02-07T01:47:07Z","lastTransitionTime":"2023-02-07T01:47:07Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2023-02-07T01:47:07Z","lastTransitionTime":"2023-02-07T01:47:07Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"False","lastHeartbeatTime":"2023-02-07T01:47:07Z","lastTransitionTime":"2023-02-07T01:47:07Z","reason":"KubeletNotReady","message":"[container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized, failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes \"ns-yun-020065.vclound.com\" not found]"}],"addresses":[{"type":"InternalIP","address":"10.189.20.65"},{"type":"Hostname","address":"ns-yun-020065.vclound.com"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"b0357f8003b24b43b0213851413aeb36","systemUUID":"a0bd1c39-eaa2-e511-8a7d-2827798f38ec","bootID":"ca6b74ee-fa0c-4faf-a53f-7144e32ea28f","kernelVersion":"4.18.0-305.3.1.el8.x86_64","osImage":"CentOS Linux 8","containerRuntimeVersion":"containerd://1.6.15","kubeletVersion":"v1.24.8","kubeProxyVersion":"v1.24.8","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["k8s.gcr.io/etcd:3.5.5-0"],"sizeBytes":301522620},{"names":["k8s.gcr.io/kube-apiserver:v1.24.10"],"sizeBytes":131733971},{"names":["k8s.gcr.io/kube-apiserver:v1.24.8"],"sizeBytes":131113427},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.10"],"sizeBytes":121342265},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.8"],"sizeBytes":120746809},{"names":["k8s.gcr.io/kube-proxy:v1.24.10"],"sizeBytes":112211511},{"names":["k8s.gcr.io/kube-proxy:v1.24.8"],"sizeBytes":111867447},{"names":["docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2"],"sizeBytes":60635609},{"names":["k8s.gcr.io/kube-scheduler:v1.24.10"],"sizeBytes":52751160},{"names":["k8s.gcr.io/kube-scheduler:v1.24.8"],"sizeBytes":52339512},{"names":["k8s.gcr.io/coredns/coredns:v1.8.6","k8s.gcr.io/coredns:v1.8.6"],"sizeBytes":46957023},{"names":["docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.2"],"sizeBytes":8250502},{"names":["k8s.gcr.io/pause:3.7"],"sizeBytes":714605},{"names":["registry.k8s.io/pause:3.6"],"sizeBytes":685866}]}}
I0207 09:47:07.320843   41694 request.go:1154] Request Body: {"metadata":{"annotations":{"kubeadm.alpha.kubernetes.io/cri-socket":"unix:///var/run/containerd/containerd.sock"}}}
I0207 09:47:07.320912   41694 round_trippers.go:466] curl -v -XPATCH  -H "Accept: application/json, */*" -H "Content-Type: application/strategic-merge-patch+json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s'
I0207 09:47:07.324686   41694 round_trippers.go:553] PATCH https://10.189.20.64:6443/api/v1/nodes/ns-yun-020065.vclound.com?timeout=10s 200 OK in 3 milliseconds
I0207 09:47:07.324708   41694 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 3 ms Duration 3 ms
I0207 09:47:07.324725   41694 round_trippers.go:577] Response Headers:
I0207 09:47:07.324740   41694 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: b4e51592-3979-4a1e-846d-8b8620a0f93a
I0207 09:47:07.324755   41694 round_trippers.go:580]     Date: Tue, 07 Feb 2023 01:47:07 GMT
I0207 09:47:07.324769   41694 round_trippers.go:580]     Audit-Id: 50b75ba2-c4c3-48ce-9c58-b110f894e2ce
I0207 09:47:07.324783   41694 round_trippers.go:580]     Cache-Control: no-cache, private
I0207 09:47:07.324810   41694 round_trippers.go:580]     Content-Type: application/json
I0207 09:47:07.324827   41694 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 08824c56-a6f2-4de2-9178-939150bf4075
I0207 09:47:07.324903   41694 request.go:1154] Response Body: {"kind":"Node","apiVersion":"v1","metadata":{"name":"ns-yun-020065.vclound.com","uid":"4657eb62-4174-4b22-bbd4-3bf915711404","resourceVersion":"6514","creationTimestamp":"2023-02-07T01:47:07Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"ns-yun-020065.vclound.com","kubernetes.io/os":"linux"},"annotations":{"kubeadm.alpha.kubernetes.io/cri-socket":"unix:///var/run/containerd/containerd.sock","node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"},"managedFields":[{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2023-02-07T01:47:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}}}}},{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T01:47:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2023-02-07T01:47:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}}}}]},"spec":{"podCIDR":"10.244.1.0/24","podCIDRs":["10.244.1.0/24"],"taints":[{"key":"node.kubernetes.io/not-ready","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"24","ephemeral-storage":"898781408Ki","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131850144Ki","pods":"110"},"allocatable":{"cpu":"24","ephemeral-storage":"828316944242","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"131747744Ki","pods":"110"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2023-02-07T01:47:07Z","lastTransitionTime":"2023-02-07T01:47:07Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2023-02-07T01:47:07Z","lastTransitionTime":"2023-02-07T01:47:07Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2023-02-07T01:47:07Z","lastTransitionTime":"2023-02-07T01:47:07Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"False","lastHeartbeatTime":"2023-02-07T01:47:07Z","lastTransitionTime":"2023-02-07T01:47:07Z","reason":"KubeletNotReady","message":"[container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized, failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes \"ns-yun-020065.vclound.com\" not found]"}],"addresses":[{"type":"InternalIP","address":"10.189.20.65"},{"type":"Hostname","address":"ns-yun-020065.vclound.com"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"b0357f8003b24b43b0213851413aeb36","systemUUID":"a0bd1c39-eaa2-e511-8a7d-2827798f38ec","bootID":"ca6b74ee-fa0c-4faf-a53f-7144e32ea28f","kernelVersion":"4.18.0-305.3.1.el8.x86_64","osImage":"CentOS Linux 8","containerRuntimeVersion":"containerd://1.6.15","kubeletVersion":"v1.24.8","kubeProxyVersion":"v1.24.8","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["k8s.gcr.io/etcd:3.5.5-0"],"sizeBytes":301522620},{"names":["k8s.gcr.io/kube-apiserver:v1.24.10"],"sizeBytes":131733971},{"names":["k8s.gcr.io/kube-apiserver:v1.24.8"],"sizeBytes":131113427},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.10"],"sizeBytes":121342265},{"names":["k8s.gcr.io/kube-controller-manager:v1.24.8"],"sizeBytes":120746809},{"names":["k8s.gcr.io/kube-proxy:v1.24.10"],"sizeBytes":112211511},{"names":["k8s.gcr.io/kube-proxy:v1.24.8"],"sizeBytes":111867447},{"names":["docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2"],"sizeBytes":60635609},{"names":["k8s.gcr.io/kube-scheduler:v1.24.10"],"sizeBytes":52751160},{"names":["k8s.gcr.io/kube-scheduler:v1.24.8"],"sizeBytes":52339512},{"names":["k8s.gcr.io/coredns/coredns:v1.8.6","k8s.gcr.io/coredns:v1.8.6"],"sizeBytes":46957023},{"names":["docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.2"],"sizeBytes":8250502},{"names":["k8s.gcr.io/pause:3.7"],"sizeBytes":714605},{"names":["registry.k8s.io/pause:3.6"],"sizeBytes":685866}]}}

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

检查 node 状态

# kubectl get nodes
NAME                        STATUS     ROLES           AGE     VERSION
ns-yun-020064.vclound.com   NotReady   control-plane   88m     v1.24.8
ns-yun-020065.vclound.com   NotReady   <none>          2m59s   v1.24.8
ns-yun-020066.vclound.com   NotReady   <none>          31s     v1.24.8
ns-yun-020067.vclound.com   NotReady   <none>          32s     v1.24.8
ns-yun-020068.vclound.com   NotReady   <none>          31s     v1.24.8

打通 fannel 网络

# kubectl  apply -f kube-flannel.yml
namespace/kube-flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created

检查 pod 是否可用

# kubectl  -n kube-system get pods
NAME                                                READY   STATUS    RESTARTS   AGE
coredns-6d4b75cb6d-6x8ln                            1/1     Running   0          89m
coredns-6d4b75cb6d-f7h6n                            1/1     Running   0          89m
etcd-ns-yun-020064.vclound.com                      1/1     Running   0          90m
kube-apiserver-ns-yun-020064.vclound.com            1/1     Running   0          90m
kube-controller-manager-ns-yun-020064.vclound.com   1/1     Running   0          90m
kube-proxy-2pgjb                                    1/1     Running   0          89m
kube-proxy-gnfg4                                    1/1     Running   0          2m46s
kube-proxy-nshkb                                    1/1     Running   0          2m46s
kube-proxy-p5s4p                                    1/1     Running   0          5m14s
kube-proxy-x72fz                                    1/1     Running   0          2m46s
kube-scheduler-ns-yun-020064.vclound.com            1/1     Running   0          90m

容器没有问题，基本集群已经搭建成功