相对于静态存储, 动态存储的优势:
● 管理员无需预先创建大量的PV作为存储资源;
● 静态存储需要用户申请PVC时保证容量和读写类型与预置PV的容量及读写类型完全匹配, 而动态存储则无需如此.
本文使用NFS存储类型完成动态存储, 需要如下步骤.
1. 创建NFS服务
参见上篇文章: 安装NFS服务
2. 部署存储供应卷
根据PVC的请求, 动态创建PV存储.
[root@bogon statefulset]# cat deployment-nfs.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: nfs-client-provisioner
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccount: nfs-provisioner
containers:
- name: nfs-client-provisioner
image: registry.cn-hangzhou.aliyuncs.com/open-ali/nfs-client-provisioner
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs
- name: NFS_SERVER
value: 192.168.64.133
- name: NFS_PATH
value: /home/nfs
volumes:
- name: nfs-client-root
nfs:
server: 192.168.64.133
path: /home/nfs创建:
[root@bogon statefulset]# kubectl create -f deployment-nfs.yaml
[root@bogon statefulset]# kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
nfs-client-provisioner 1 1 1 1 36m3 部署storageclass
[root@bogon statefulset]# cat storageclass-nfs.yaml
apiVersion: storage.k8s.io/v1beta1
kind: StorageClass
metadata:
name: managed-nfs-storage
provisioner: fuseim.pri/ifs创建:
[root@bogon statefulset]# kubectl create -f storageclass-nfs.yaml
[root@bogon statefulset]# kubectl get storageclass
NAME PROVISIONER AGE
managed-nfs-storage fuseim.pri/ifs 36m4. 部署StatefulSet验证动态分配
[root@bogon statefulset]# cat statefulset-nfs.yaml
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: web
spec:
serviceName: "nginx1"
replicas: 1
volumeClaimTemplates:
- metadata:
name: test
annotations:
volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 2Gi
template:
metadata:
labels:
app: nginx1
spec:
serviceAccount: nfs-provisioner
containers:
- name: nginx1
image: nginx:1.7.9
volumeMounts:
- mountPath: "/mnt"
name: test创建:
[root@bogon statefulset]# kubectl create -f statefulset-nfs.yaml
statefulset "web" created
查看pod一直在创建中:
[root@bogon statefulset]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-56dd854578-p4z8x 1/1 Running 0 3m
web-0 0/1 ContainerCreating 0 2m开始排错, 查看pod详细信息:
[root@bogon statefulset]# kubectl describe pod web-0
Name: web-0
Namespace: default
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m default-scheduler Successfully assigned web-0 to 192.168.64.134
Normal SuccessfulMountVolume 2m kubelet, 192.168.64.134 MountVolume.SetUp succeeded for volume "default-token-6bgzc"
Warning FailedMount 34s kubelet, 192.168.64.134 Unable to mount volumes for pod "web-0_default(f7f0dd6a-88d1-11e8-87be-000c2964ecfc)": timeout expired waiting for volumes to attach/mount for pod "default"/"web-0". list of unattached/unmounted volumes=[test]查看storageclass:
[root@bogon statefulset]# kubectl describe storageclass managed-nfs-storage
Name: managed-nfs-storage
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ExternalProvisioning 3s (x18 over 4m) persistentvolume-controller waiting for a volume to be created, either by external provisioner "fuseim.pri/ifs" or manually created by system administrator可以判断是存储供应出的问题.
查看供应日志:
[root@bogon statefulset]# kubectl logs -f nfs-client-provisioner-56dd854578-p4z8x
E0716 08:33:59.141420 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:default:default" cannot list persistentvolumeclaims at the cluster scope
E0716 08:33:59.141995 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:384: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:default:default" cannot list storageclasses.storage.k8s.io at the cluster scope
E0716 08:34:00.121657 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:412: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:serviceaccount:default:default" cannot list persistentvolumes at the cluster scope
E0716 08:34:00.147990 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:default:default" cannot list persistentvolumeclaims at the cluster scope
E0716 08:34:00.152728 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:384: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:default:default" cannot list storageclasses.storage.k8s.io at the cluster scope
E0716 08:34:01.127057 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:412: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:serviceaccount:default:default" cannot list persistentvolumes at the cluster scope
E0716 08:34:01.153919 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:default:default" cannot list persistentvolumeclaims at the cluster scope
E0716 08:34:01.159801 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:384: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:default:default" cannot list storageclasses.storage.k8s.io at the cluster scope
E0716 08:34:02.134286 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:412: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:serviceaccount:default:default" cannot list persistentvolumes at the cluster scope
E0716 08:34:02.156738 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:default:default" cannot list persistentvolumeclaims at the cluster scope
E0716 08:34:02.163755 1 reflector.go:201] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:384: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:default:default" cannot list storageclasses.storage.k8s.io at the cluster scope
报错没有权限. 需要创建权限体系, 步骤如下.
5 构建权限体系
ServiceAccount也是一种账号, 供运行在pod中的进程使用, 为pod中的进程提供必要的身份证明.
创建serviceaccount
[root@bogon statefulset]# cat serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-provisioner创建role
[root@bogon statefulset]# cat serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-provisioner
[root@bogon statefulset]# cat clusterrole.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-provisioner-runner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["services", "endpoints"]
verbs: ["get"]
- apiGroups: ["extensions"]
resources: ["podsecuritypolicies"]
resourceNames: ["nfs-provisioner"]
verbs: ["use"]账户和角色绑定
[root@bogon statefulset]# cat clusterrolebinding.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-provisioner
subjects:
- kind: ServiceAccount
name: nfs-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: nfs-provisioner-runner
apiGroup: rbac.authorization.k8s.io创建:
kubectl create -f serviceaccount.yaml -f clusterrole.yaml -f clusterrolebinding.yaml在存储供应卷和StatefulSet文件中分别放入如下配置:
serviceAccount: nfs-provisioner例如StatefulSet文件:
[root@bogon statefulset]# cat statefulset-nfs.yaml
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: web
spec:
serviceName: "nginx1"
replicas: 1
volumeClaimTemplates:
- metadata:
name: test
annotations:
volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 2Gi
template:
metadata:
labels:
app: nginx1
spec:
serviceAccount: nfs-provisioner
containers:
- name: nginx1
image: nginx:1.7.9
volumeMounts:
- mountPath: "/mnt"
name: testkubernetes1.6以后默认开启rbac, 做完如上操作然后再次创建即可.
