will技客

专注于网站数据采集,网络爬虫研究的个人博客。

Rancher2.3 + k3s部署记录(实验性)

Linux 0 评

前言

服务器配置及网络规划

  • Debian9 + Docker19.03.6 + Rancher2.3.5 + k3s1.17.2
  • main(juhu-ai): 4核/16GB/40GB 172.16.32.6 单机部署rancher2 k3s server
  • node1(juhu-webs): 4核/8GB/40GB 172.16.32.5
  • node2(juhu-spider): 4核/8GB/40GB 172.16.32.4

常用端口占用

  • rancher: 9443, 9080, 2376
  • k3s: 6443
  • redis: 26379,6379
  • zookeeper: 2181, 2888, 3888
  • memcached: 11211
  • web: 80, 443
  • pyspider: 6555
  • newSpider: 10190, 10150, 10180, 10100, 10102

基础操作

修改主机名hostname $hostname并修改文件/etc/hostname
配置每台主机的hosts(/etc/hosts),添加host_ip $hostname到/etc/hosts文件中

安装所需软件包

sudo apt-get install apt-transport-https ca-certificates curl software-properties-common -y

安装docker

sudo curl -fsSL "https://download.docker.com/linux/debian/gpg" | apt-key add -
sudo echo "deb [arch=amd64] https://download.docker.com/linux/debian stretch stable" > /etc/apt/sources.list.d/docker.list
sudo apt-get update && apt-get install -y docker-ce

#centos系统或者以上方法无法安装,请用官方脚本
sudo curl -fsSL get.docker.com | sh

master安装k3s

sudo wget https://github.com/rancher/k3s/releases/download/v1.17.2%2Bk3s1/k3s-arm64

sudo cp k3s-arm64 /usr/local/bin/k3s

sudo chmod +x /usr/local/bin/k3s

sudo docker pull registry.cn-shenzhen.aliyuncs.com/ilemonrain/pause-amd64:3.1
sudo docker tag registry.cn-shenzhen.aliyuncs.com/ilemonrain/pause-amd64:3.1 k8s.gcr.io/pause-amd64:3.1

#sudo curl -sfL https://docs.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh \
sudo curl -sfL https://docs.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh \
-s - server --datastore-endpoint='mysql://k3sdbadmin:k3sdbadmin20@tcp(mysql.rdsmmwb9xnfn103.rds.gz.baidubce.com:3306)/k3sdb'

修改k3s启动配置

vim /etc/systemd/system/multi-user.target.wants/k3s.service

[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
After=network-online.target

[Service]
Type=notify
EnvironmentFile=/etc/systemd/system/k3s.service.env
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
KillMode=process
Delegate=yes
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
#ExecStart=/usr/local/bin/k3s server
#修改ExecStart的值,使用外部mysql数据储存
ExecStart=/usr/local/bin/k3s server \
--docker --no-deploy traefik \
--datastore-endpoint='mysql://k3sdbadmin:k3sdbadmin20@tcp(mysql.rdsmmwb9xnfn103.rds.gz.baidubce.com:3306)/k3sdb'

[Install]
WantedBy=multi-user.targe

重启

sudo systemctl daemon-reload
sudo service k3s restart

#稍等数十秒,查看k3s是否正常运行
k3s kubectl get node

#正常显示
NAME                STATUS   ROLES    AGE     VERSION
instance-ioi6l138   Ready    master   6m28s   v1.17.2+k3s1

k3s启动时出现错误:

Error 1071: Specified key was too long; max key length is 767 bytes

原因:我使用的是mysql5.6,索引列大于767 bytes时会报错,可以调整mysql配置参数并将表kine的row_format设置为dynamic格式

#修改innodb_large_prefix,innodb_file_format参数

mysql> set global innodb_large_prefix=1;

mysql> set global innodb_file_format=BARRACUDA;

具体参考链接

savle k3s安装

#K3S_TOKEN在k3s主服务器的/var/lib/rancher/k3s/server/node-token
cat /var/lib/rancher/k3s/server/node-token

sudo curl -sfL https://docs.rancher.cn/k3s/k3s-install.sh | K3S_URL=https://juhu-ai:6443 \
K3S_TOKEN={k3s_token} INSTALL_K3S_MIRROR=cn sh -

安装rancher

第1次这里搞错了,请无视

# 这里搞错了,安装了1.6的
# Launch on each node in your HA cluster
sudo docker run -d --restart=unless-stopped --name rancher-server -p 9080:8080 -p 9345:9345 \
     -v /data/docker/rancher-server/var/lib/rancher/:/var/lib/rancher/ \
     -v /data/docker/rancher-server/var/log/auditlog:/var/log/auditlog \
     -e CATTLE_SYSTEM_CATALOG=bundled \
     -e AUDIT_LEVEL=3 \
     rancher/server \
     --db-host mysql.rdsmmwb9xnfn103.rds.gz.baidubce.com --db-port 3306 \
     --db-user rancher --db-pass rancher2020 --db-name rancher \
     --advertise-address 172.16.32.6 \
     --advertise-http-port 9080

juhu-ai上单机部署rancher2

docker run -d --restart=unless-stopped \
-p 9080:80 -p 9443:443 \
-v /data/docker/rancher-server/var/lib/rancher/:/var/lib/rancher/ \
-v /data/docker/rancher-server/var/log/auditlog:/var/log/auditlog \
-e CATTLE_SYSTEM_CATALOG=bundled \
-e AUDIT_LEVEL=3 \
rancher/rancher:stable

访问rancher ip:9443,新导入集群,然后在K3S的主控节点执行命令

curl --insecure -sfL https://172.16.32.6/v3/import/***.yaml | kubectl apply -f -

等待数十秒连接成功

部署参考

rancher2文档

k3s文档

部署k3s高可用集群(实验性)