K8s:离线部署Kubernetes1.26.12及采用外部Harbor

本文详细介绍了在离线环境下使用KubeKey部署KubeSphere集群的完整流程。主要内容包括：1）在联网服务器下载KubeKey工具、获取镜像列表并构建离线安装包；2）在离线服务器创建配置文件，配置私有镜像仓库信息；3）通过KubeKey命令执行离线安装。

一、离线环境

CPU:Hygon C86 7285 32-core Processor
操作系统：麒麟操作系统
containerd：1.7.27
Harbor:2.13.1

二、一台能上互联网的服务器

2.1、下载KubeKey3.1.10

https://kubernetes.pek3b.qingstor.com/kubekey/releases/download/v3.1.10/kubekey-v3.1.10-linux-amd64.tar.gz

解压后，会有 KubeKey 二进制文件 kk。
#tar -zxvf kubekey-v3.1.10-linux-amd64.tar.gz

2.2、获取kubesphere版本信息及镜像列表

https://get-images.kubesphere.io/ 

镜像列表文件如下：

2.3、创建 manifest 文件

# chmod +x kk
./kk create manifest --with-kubernetes v1.26.12  --with-registry
使用 kk 离线部署镜像仓库，添加 --with-registry 打包镜像仓库的安装文件
该命令将创建一个 manifest-sample.yaml 文件。

 2.4、编辑 manifest 文件

#vi manifest-sample.yaml
复制kk-manifest-mirror.yaml（若访问 DockerHub 受限） 中的镜像列表，添加到新创建的 manifest-sample.yaml 文件中。

 

apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Manifest
metadata:
  name: sample
spec:
  arches:
  - amd64
  operatingSystems: []
  kubernetesDistributions:
  - type: kubernetes
    version: v1.26.12
  components:
    helm: 
      version: v3.14.3
    cni: 
      version: v1.2.0
    etcd: 
      version: v3.5.13
    containerRuntimes:
    - type: docker
      version: 24.0.9
    - type: containerd
      version: 1.7.13
    calicoctl:
      version: v3.27.4
    crictl: 
      version: v1.29.0
    docker-registry:
      version: "2"
    harbor:
      version: v2.10.1
    docker-compose:
      version: v2.26.1
  images:
  - registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.9
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.26.12
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.26.12
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.26.12
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.26.12
  - registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.9.3
  - registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.22.20
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.27.4
  - registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.27.4
  - registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.27.4
  - registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.27.4
  - registry.cn-beijing.aliyuncs.com/kubesphereio/typha:v3.27.4
  - registry.cn-beijing.aliyuncs.com/kubesphereio/flannel:v0.21.3
  - registry.cn-beijing.aliyuncs.com/kubesphereio/flannel-cni-plugin:v1.1.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/cilium:v1.15.3
  - registry.cn-beijing.aliyuncs.com/kubesphereio/operator-generic:v1.15.3
  - registry.cn-beijing.aliyuncs.com/kubesphereio/hybridnet:v0.8.6
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-ovn:v1.10.10
  - registry.cn-beijing.aliyuncs.com/kubesphereio/multus-cni:v3.8
  - registry.cn-beijing.aliyuncs.com/kubesphereio/provisioner-localpv:3.3.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/linux-utils:3.3.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.9.6-alpine
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-vip:v0.7.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kata-deploy:stable
  - registry.cn-beijing.aliyuncs.com/kubesphereio/node-feature-discovery:v0.10.0
## ks-corenew
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/ks-apiserver:v4.1.2
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/ks-console:v4.1.2
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/ks-controller-manager:v4.1.2
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/kubectl:v1.27.16
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/redis:7.2.4-alpine
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/haproxy:2.9.6-alpine
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/ks-extensions-museum:v1.1.2
  ## devops
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/devops-apiserver:v4.1.2
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/devops-controller:v4.1.2
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/devops-tools:v4.1.2
  - swr.cn-southwest-2.myhuaweicloud.com/ks/kubesphere/devops-jenkins:v4.1.2-2.346.3
  - swr.cn-southwest-2.my