使用Kubeadm部署Kubernetes 1.6.1

kubernetes

部署方案与环境准备

k8s集群 kubernetes 1.6.1 + Docker 1.12

容器网络 calico

监控 prometheus

服务暴露 traefik + ingress

日志收集 graylog

1.准备机器

1
2
3
192.168.61.41 node1
192.168.61.42 node2
192.168.61.43 node3

系统版本要求Centos 7.2或以上, 内核版本 3.18以上(docker的overlay stroage 要求)

2.准备docker的yum源,创建/etc/yum.repos.d/docker.repo

1
2
3
4
5
[docker-repo]
name=Docker Repository
baseurl=http://mirrors.aliyun.com/docker-engine/yum/repo/main/centos/$releasever/
enabled=1
gpgcheck=0

3.Kuberntes的yum源,创建/etc/yum.repos.d/k8s.repo

1
2
3
4
5
[k8s-repo]
name=k8s Repository
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0

安装docker

查看docker版本

1
yum list docker-engine --showduplicates

Kubernetes 1.6还没有针对docker 1.13和最新的docker 17.03上做测试和验证,所以这里安装Kubernetes官方推荐的Docker 1.12版本。

1
yum install -y docker-engine-1.12.6-1.el7.centos.x86_64

系统配置

根据官方文档Installing Kubernetes on Linux with kubeadm中的Limitations小节中的内容,对各节点系统做如下设置:

创建/etc/sysctl.d/k8s.conf文件,添加如下内容:

1
2
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

执行sysctl -p /etc/sysctl.d/k8s.conf使修改生效

安装kubeadm和kubelet

查看kubeadm, kubelet, kubectl, kubernets-cni的最新版本:

1
2
3
yum list kubeadm --showduplicates
kubeadm.x86_64 1.6.1-0 @k8s-repo
kubeadm.x86_64 1.6.0-0 k8s-repo

1
2
3
yum list kubelet --showduplicates
kubelet.x86_64 1.6.1-0 @k8s-repo
kubelet.x86_64 1.6.0-0 k8s-repo

kubeadm和kubelet已经是1.6.1版本,直接安装

1
yum install -y kubelet kubeadm kubectl kubernetes-cni

启用kubelet服务

1
systemctl enable kubelet.service

初始化集群

准备MasterConfiguration文件
cat k8s_master_config.yaml

1
2
3
4
5
6
7
apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
api:
advertiseAddress: 192.168.61.41
networking:
dnsDomain: cluster.local
kubernetesVersion: v1.6.1

准备镜像

kubeadm部署时默认会去gcr.io拉取镜像, 这个域名国内被墙了, 所以要曲线救国

  1. 先购买一台海外的vps, 部署squid, 参考使用Squid3搭建Docker镜像下载代理
  2. 给docker增加HTTP_PROXY设置
    编辑/usr/lib/systemd/system/docker.service
    增加如下内容:
    1
    2
    3
    Environment="HTTP_PROXY=YOUR_VPS_IP:3128"
    Environment="HTTPS_PROXY=YOUR_VPS_IP:3128"
    Environment="NO_PROXY=YOUR_REGISTRY_DOMAIN"

ip地址改为自己的vps的地址, 保存重启docker

1
systemctl daemon-reload & systemctl restart docker

  1. 拉取kubernetes部署所需镜像,参考Master Images
  2. 拉取镜像完成后tag为自己的私服的域名然后push
  3. 编辑 /etc/profile,加入docker私服地址到环境变量
    1
    export KUBE_REPO_PREFIX=registry.youregistry.com/google_containers

这样配置的话,kubeadm在init时会向registry.youregistry.com 拉取镜像

部署集群

使用kubeadm初始化集群,选择node1作为Master Node,在node1上执行下面的命令:

1
kubeadm init --config k8s_master_config.yaml

k8s_master_config.yaml为kubeadm的config文件

1
2
3
4
5
6
7
apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
api:
advertiseAddress: 192.168.61.41
networking:
dnsDomain: cluster.local
kubernetesVersion: v1.6.1

kubeadm init执行成功后输出下面的信息:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
kubeadm init --config k8s_master_config.yaml
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.6.1
[init] Using Authorization mode: RBAC
[preflight] Running pre-flight checks
[preflight] Starting the kubelet service
[certificates] Generated CA certificate and key.
[certificates] Generated API server certificate and key.
[certificates] API Server serving cert is signed for DNS names [node0 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.61.41]
[certificates] Generated API server kubelet client certificate and key.
[certificates] Generated service account token signing key and public key.
[certificates] Generated front-proxy CA certificate and key.
[certificates] Generated front-proxy client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
[apiclient] Created API client, waiting for the control plane to become ready
[apiclient] All control plane components are healthy after 14.583864 seconds
[apiclient] Waiting for at least one node to register
[apiclient] First node has registered after 6.008990 seconds
[token] Using token: e7986d.e440de5882342711
[apiconfig] Created RBAC rules
[addons] Created essential addon: kube-proxy
[addons] Created essential addon: kube-dns
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run (as a regular user):
sudo cp /etc/kubernetes/admin.conf $HOME/
sudo chown $(id -u):$(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
http://kubernetes.io/docs/admin/addons/
You can now join any number of machines by running the following on each node
as root:
kubeadm join --token e7986d.e440de5882342711 192.168.61.41:6443

出现token说明Master初始化完成,使用kubeadm初始化的Kubernetes集群在Master节点上的核心组件:kube-apiserver,kube-scheduler, kube-controller-manager是以静态Pod的形式运行的。

1
2
3
4
5
ls -l /etc/kubernetes/manifests/
etcd.yaml
kube-apiserver.yaml
kube-controller-manager.yaml
kube-scheduler.yaml

如果想修改kubernetes组件的参数, 可以修改对应组件的yaml配置文件, 然后重启kubelet服务使之生效

1
systemctl restart kubelet

部署Pod Network

1
kubectl create -f http://yangl.me/file/calico.yaml

查看pod状态

1
kubectl get pod --all-namespaces -o wide

使master node参与工作负载

1
kubectl taint nodes --all node-role.kubernetes.io/master-

测试DNS是否正常

1
kubectl run curl --image=radial/busyboxplus:curl -i --tty

进入容器之后测试下nslookup kubernetes.default

1
2
3
4
5
[ root@curl-2421989462-vldmp:/ ]$ nslookup kubernetes.default
Server: 10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
Name: kubernetes.default
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local

测试没问题后删除pod

1
kubectl delete deploy curl

节点加入到集群

1
kubeadm join --token c9b51c.416daf3a9bf8ebcd 192.168.61.41:6443

如果token忘记了,可以在master上执行

1
kubeadm token list

部署traefik以暴露服务

traefik(https://traefik.io)是一款开源的反向代理与负载均衡工具。它最大的优点是能够与常见的微服务系统直接整合,可以实现自动化动态配置。
关于traefik的详细信息请参考traefik简介

下面来部署traefik

1
kubectl create -f http://yangl.me/file/traefik-with-ui-rbac.yaml

配置文件中traefik-ui暴露的域名为traefik-ui.local, 所以只需配置一个A记录指向此域名, 也可以改成自己想要的域名.

安装Dashboard

1
kubectl create -f http://yangl.me/file/kubernetes-dashboard.yaml

配置文件中dashboard暴露的域名为dashboard.local, 同样需要设置一个A记录或者host

prometheus监控系统部署

prometheus的部署比较简单, 可参考prometheus-kubernetes

未完待续…

坚持原创技术分享,您的支持将鼓励我继续创作!