使用RKE构建企业生产级Kubernetes集群
一、RKE工具介绍
-
RKE是一款经过CNCF认证的开源Kubernetes发行版,可以在Docker容器内运行。
-
它通过删除大部分主机依赖项,并为部署、升级和回滚提供一个稳定的路径,从而解决了Kubernetes最常见的安装复杂性问题。
-
借助RKE,Kubernetes可以完全独立于正在运行的操作系统和平台,轻松实现Kubernetes的自动化运维。
-
只要运行受支持的Docker版本,就可以通过RKE部署和运行Kubernetes。仅需几分钟,RKE便可通过单条命令构建一个集群,其声明式配置使Kubernetes升级操作具备原子性且安全。
二、集群主机准备
2.1 集群主机配置要求
2.1.1 部署集群环境说明
部署Kubernetes集群机器需要满足以下几个条件:
1)一台或多台机器,操作系统 CentOS7
2)硬件配置:2GB或更多RAM,2个CPU或更多CPU,硬盘100GB或更多
3) 集群中所有机器之间网络互通
4)可以访问外网,需要拉取镜像,如果服务器不能上网,需要提前下载镜像并导入节点
5)禁止swap分区
2.1.2 软件环境
| 软件 | 版本 |
|---|---|
| 操作系统 | CentOS7 |
| docker-ce | 20.10.12 |
| kubernetes | 1.22.5 |
2.1.3 集群主机名称、IP地址及角色规划
| 主机名称 | IP地址 | 角色 |
|---|---|---|
| master01 | 192.168.10.10 | controlplane、rancher、rke |
| master02 | 192.168.10.11 | controlpane |
| worker01 | 192.168.10.12 | worker |
| worker02 | 192.168.10.13 | worker |
| etcd01 | 192.168.10.14 | etcd |
2.2 集群主机名称配置
所有集群主机均要配置对应的主机名称即可。
# hostnamectl set-hostname xxx
把xxx替换为对应的主机名
192.168.10.10 master01
192.168.10.11 master02
192.168.10.12 worker01
192.168.10.13 worker02
192.168.10.14 etcd01
2.3 集群主机IP地址配置
所有集群主机均要配置对应的主机IP地址即可。
# vim /etc/sysconfig/network-scripts/ifcfg-ens33
# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="none" 修改为静态
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
IPV6_ADDR_GEN_MODE="stable-privacy"
NAME="eth0"
DEVICE="eth0"
ONBOOT="yes"
添加如下内容:
IPADDR="192.168.10.XXX"
PREFIX="24"
GATEWAY="192.168.10.2"
DNS1="119.29.29.29"
2.4 主机名与IP地址解析
所有主机均要配置。
# vim /etc/hosts
# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.10.10 master01
192.168.10.11 master02
192.168.10.12 worker01
192.168.10.13 worker02
192.168.10.14 etcd01
2.5 配置ip_forward及过滤机制
所有主机均要配置
将桥接的IPv4流量传递到iptables的链
# vim /etc/sysctl.conf
# cat /etc/sysctl.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
加载br_netfilter模块,要不然加载配置文件会错误
# modprobe br_netfilter
# sysctl -p /etc/sysctl.conf
2.6 主机安全设置
所有主机均要设置
2.6.1 防火墙
# systemctl stop firewalld
# systemctl disable firewalld
# firewall-cmd --state
2.6.2 selinux
修改完成后一定要重启操作系统
永久关闭,一定要重启操作系统后生效。
sed -ri 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
临时关闭,不重启操作系统,即刻生效。
# setenforce 0
2.7 主机swap分区设置
所有主机均要配置
永久关闭,需要重启操作系统生效。
# sed -ri 's/.*swap.*/#&/' /etc/fstab
在该行加上注释
# cat /etc/fstab
......
#/dev/mapper/centos_192-swap swap swap defaults 0 0
临时关闭,不需要重启操作系统,即刻生效。
# swapoff -a
2.8 时间同步
所有主机均要配置
# yum -y insall ntpdate
# crontab -e
0 */1 * * * ntpdate time1.aliyun.com
三、Docker部署
所有主机均要配置
3.1 配置Docker YUM源
在这里插入图片描述

# wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
查看是否有docker-ce.repo
# ls /etc/yum.repos.d/
3.2 安装Docker CE
# yum -y install docker-ce
3.3 启动Docker服务
# systemctl enable docker
# systemctl start docker
# docker version
3.4 配置Docker容器镜像加速器
# vim /etc/docker/daemon.json
# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://s27w6kze.mirror.aliyuncs.com"]
}
四、docker compose安装
# curl -L "https://github.com/docker/compose/releases/download/1.28.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
# chmod +x /usr/local/bin/docker-compose
# ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
# docker-compose --version
五、添加rancher用户
使用CentOS时,不能使用 root 账号,因此要添加专用的账号进行 docker相关 操作。
所有集群主机均需要操作
# useradd rancher
把用户添加到docker组里面
# usermod -aG docker rancher
设置密码
# echo 123 | passwd --stdin rancher
查看是否添加到docker组
# cat /etc/group | grep docker
六、生成ssh证书用于部署集群
rke二进制文件安装主机上创建密钥,即为control主机,用于部署集群。
6.1 生成ssh证书
# ssh-keygen
# ls .ssh
6.2 复制证书到集群中所有主机
# ssh-copy-id rancher@master01
# ssh-copy-id rancher@master02
# ssh-copy-id rancher@worker01
# ssh-copy-id rancher@worker02
# ssh-copy-id rancher@etcd01
6.3 验证ssh证书是否可用
本次在master01上部署rke二进制文件。
在rke二进制文件安装主机机测试连接其它集群主机,验证是否可使用docker ps命令即可。
# ssh rancher@主机名
远程主机# docker ps
七、rke工具下载
本次在master01上部署rke二进制文件。

# wget https://github.com/rancher/rke/releases/download/v1.3.7/rke_linux-amd64
# mv rke_linux-amd64 /usr/local/bin/rke
# chmod +x /usr/local/bin/rke
# rke --version
rke version v1.3.7
八、初始化rke配置文件
# mkdir -p /app/rancher
# cd /app/rancher
# rke config --name cluster.yml
[+] Cluster Level SSH Private Key Path [~/.ssh/id_rsa]: ~/.ssh/id_rsa # 集群私钥路径
[+] Number of Hosts [1]: 3 集群中有3个节点
[+] SSH Address of host (1) [none]: 192.168.10.10 第一个节点IP地址
[+] SSH Port of host (1) [22]: 22 第一个节点SSH访问端口
[+] SSH Private Key Path of host (192.168.10.10) [none]: ~/.ssh/id_rsa 第一个节点私钥路径
[+] SSH User of host (192.168.10.10) [ubuntu]: rancher 远程用户名
[+] Is host (192.168.10.10) a Control Plane host (y/n)? [y]: y 是否为k8s集群控制节点
[+] Is host (192.168.10.10) a Worker host (y/n)? [n]: n 不是worker节点
[+] Is host (192.168.10.10) an etcd host (y/n)? [n]: n 不是etcd节点
[+] Override Hostname of host (192.168.10.10) [none]: # 不覆盖现有主机名
[+] Internal IP of host (192.168.10.10) [none]: # 主机局域网IP地址
[+] Docker socket path on host (192.168.10.10) [/var/run/docker.sock]: 主机上docker.sock路径
[+] SSH Address of host (2) [none]: 192.168.10.12 第二个节点
[+] SSH Port of host (2) [22]: 22 远程端口
[+] SSH Private Key Path of host (192.168.10.12) [none]: ~/.ssh/id_rsa 私钥路径
[+] SSH User of host (192.168.10.12) [ubuntu]: rancher 远程访问用户
[+] Is host (192.168.10.12) a Control Plane host (y/n)? [y]: n 不是控制节点
[+] Is host (192.168.10.12) a Worker host (y/n)? [n]: y 是worker节点
[+] Is host (192.168.10.12) an etcd host (y/n)? [n]: n 不是etcd节点
[+] Override Hostname of host (192.168.10.12) [none]: # 不覆盖现有主机名
[+] Internal IP of host (192.168.10.12) [none]: # 主机局域网IP地址
[+] Docker socket path on host (192.168.10.12) [/var/run/docker.sock]: 主机上docker.sock路径
[+] SSH Address of host (3) [none]: 192.168.10.14 第三个节点
[+] SSH Port of host (3) [22]: 22 远程端口
[+] SSH Private Key Path of host (192.168.10.14) [none]: ~/.ssh/id_rsa 私钥路径
[+] SSH User of host (192.168.10.14) [ubuntu]: rancher 远程访问用户
[+] Is host (192.168.10.14) a Control Plane host (y/n)? [y]: n 不是控制节点
[+] Is host (192.168.10.14) a Worker host (y/n)? [n]: n 不是worker节点
[+] Is host (192.168.10.14) an etcd host (y/n)? [n]: y 是etcd节点
[+] Override Hostname of host (192.168.10.14) [none]: # 不覆盖现有主机名
[+] Internal IP of host (192.168.10.14) [none]: # 主机局域网IP地址
[+] Docker socket path on host (192.168.10.14) [/var/run/docker.sock]: 主机上docker.sock路径
[+] Network Plugin Type (flannel, calico, weave, canal, aci) [canal]: 使用的网络插件
[+] Authentication Strategy [x509]: # 认证策略
[+] Authorization Mode (rbac, none) [rbac]: # 认证模式
[+] Kubernetes Docker image [rancher/hyperkube:v1.21.9-rancher1]: rancher/hyperkube:v1.21.9-rancher1 # 集群容器镜像
[+] Cluster domain [cluster.local]: # 集群域名
[+] Service Cluster IP Range [10.43.0.0/16]: 集群中Servic IP地址范围
[+] Enable PodSecurityPolicy [n]: n #是否开启Pod安装策略
[+] Cluster Network CIDR [10.42.0.0/16]: # 集群Pod网络
[+] Cluster DNS Service IP [10.43.0.10]: # 集群DNS Service IP地址
[+] Add addon manifest URLs or YAML files [no]: # 是否增加插件manifest URL或配置文件
[root@master01 rancher]# ls
cluster.yml
在cluster.yaml文件中
kube-controller: image: "" extra_args: # 如果后面需要部署kubeflow或istio则一定要配置以下参数 cluster-signing-cert-file: "/etc/kubernetes/ssl/kube-ca.pem" cluster-signing-key-file: "/etc/kubernetes/ssl/kube-ca-key.pem"
九、集群部署
# pwd
/app/rancher
# rke up
输出:
INFO[0000] Running RKE version: v1.3.7
INFO[0000] Initiating Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [192.168.10.14]
INFO[0000] [dialer] Setup tunnel for host [192.168.10.10]
INFO[0000] [dialer] Setup tunnel for host [192.168.10.12]
INFO[0000] Checking if container [cluster-state-deployer] is running on host [192.168.10.14], try #1
INFO[0000] Checking if container [cluster-state-deployer] is running on host [192.168.10.10], try #1
INFO[0000] Checking if container [cluster-state-deployer] is running on host [192.168.10.12], try #1
INFO[0000] [certificates] Generating CA kubernetes certificates
INFO[0000] [certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates
INFO[0000] [certificates] GenerateServingCertificate is disabled, checking if there are unused kubelet certificates
INFO[0000] [certificates] Generating Kubernetes API server certificates
INFO[0000] [certificates] Generating Service account token key
INFO[0000] [certificates] Generating Kube Controller certificates
INFO[0000] [certificates] Generating Kube Scheduler certificates
INFO[0000] [certificates] Generating Kube Proxy certificates
INFO[0001] [certificates] Generating Node certificate
INFO[0001] [certificates] Generating admin certificates and kubeconfig
INFO[0001] [certificates] Generating Kubernetes API server proxy client certificates
INFO[0001] [certificates] Generating kube-etcd-192-168-10-14 certificate and key
INFO[0001] Successfully Deployed state file at [./cluster.rkestate]
INFO[0001] Building Kubernetes cluster
INFO[0001] [dialer] Setup tunnel for host [192.168.10.12]
INFO[0001] [dialer] Setup tunnel for host [192.168.10.14]
INFO[0001] [dialer] Setup tunnel for host [192.168.10.10]
INFO[0001] [network] Deploying port listener containers
INFO[0001] Image [rancher/rke-tools:v0.1.78] exists on host [192.168.10.14]
INFO[0001] Starting container [rke-etcd-port-listener] on host [192.168.10.14], try #1
INFO[0001] [network] Successfully started [rke-etcd-port-listener] container on host [192.168.10.14]
INFO[0001] Image [rancher/rke-tools:v0.1.78] exists on host [192.168.10.10]
INFO[0001] Starting container [rke-cp-port-listener] on host [192.168.10.10], try #1
INFO[0002] [network] Successfully started [rke-cp-port-listener] container on host [192.168.10.10]
INFO[0002] Image [rancher/rke-tools:v0.1.78] exists on host [192.168.10.12]
INFO[0002] Starting container [rke-worker-port-listener] on host [192.168.10.12], try #1
INFO[0002] [network] Successfully started [rke-worker-port-listener] container on host [192.168.10.12]
INFO[0002] [network] Port listener containers deployed successfully
INFO[0002] [network] Running control plane -> etcd port checks
INFO[0002] [network] Checking if host [192.168.10.10] can connect to host(s) [192.168.10.14] on port(s) [2379], try #1
INFO[0002] Image [rancher/rke-tools:v0.1.78] exists on host [192.168.10.10]
INFO[0002] Starting container [rke-port-checker] on host [192.168.10.10], try #1
INFO[0002] [network] Successfully started [rke-port-checker] container on host [192.168.10.10]
INFO[0002] Removing container [rke-port-checker] on host [192.168.10.10], try #1
INFO[0002] [network] Running control plane -> worker port checks
INFO[0002] [network] Checking if host [192.168.10.10] can connect to host(s) [192.168.10.12] on port(s) [10250], try #1
INFO[0002] Image [rancher/rke-tools:v0.1.78] exists on host [192.168.10.10]
INFO[0003] Starting container [rke-port-checker] on host [192.168.10.10], try #1
INFO[0003] [network] Successfully started [rke-port-checker] container on host [192.168.10.10]
INFO[0003] Removing container [rke-port-checker] on host [192.168.10.10], try #1
INFO[0003] [network] Running workers -> control plane port checks
INFO[0003] [network] Checking if host [192.168.10.12] can connect to host(s) [192.168.10.10] on port(s) [6443], try #1
INFO[0003] Image [rancher/rke-tools:v0.1.78] exists on host [192.168.10.12]
INFO[0003] Starting container [rke-port-checker] on host [192.168.10.12], try #1
INFO[0003] [network] Successfully started [rke-port-checker] container on host [192.168.10.12]
INFO[0003] Removing container [rke-port-checker] on host [192.168.10.12], try #1
INFO[0003] [network] Checking KubeAPI port Control Plane hosts
INFO[0003] [network] Removing port listener containers
INFO[0003] Removing container [rke-etcd-port-listener] on host [192.168.10.14], try #1
INFO[0003] [remove/rke-etcd-port-listener] Successfully removed container on host [192.168.10.14]
INFO[0003] Removing container [rke-cp-port-listener] on host [192.168.10.10], try #1
INFO[0003] [remove/rke-cp-port-listener] Successfully removed container on host [192.168.10.10]
INFO[0003] Removing container [rke-worker-port-listener] on host [192.168.10.12], try #1
INFO[0003


834

被折叠的 条评论
为什么被折叠?



