brunch

매거진 Google 클라우드 전문가되기

라이킷 댓글

You can make anything
by writing

C.S.Lewis

계정을 잊어버리셨나요?

by Master Seo Aug 18. 2021

161. GCP 쿠버네티스 배포

<1> cloudshell 로 쿠버네티스 클러스터 만들기

<2> 배포 객체 알아보기

<3> 배포 확장

<4> 순차적 업데이트 하기

<5> 배포 계속 하기

<6> 롤백

<7> 카나리 배포

<8> Blue/Green 배포

<9> 개인 정리

<10> 같이 보면 좋을 사이트

<1> cloudshell 로 쿠버네티스 클러스터 만들기

cloudshell 실행

인증

인증 설정 참고하기

https://brunch.co.kr/@topasvga/1132

gcloud auth list

. Credentialed Accounts

ACTIVE ACCOUNT

To set the active account, run:

$ gcloud config set account `ACCOUNT`

프로젝트 기준이다

프로젝트 확인하자

gcloud config list project

[core]

project = xxxxxx-gcp-01-xxxxxxxxxxxxx

Your active configuration is: [cloudshell-11143]

지역 설정하기

gcloud config set compute/zone us-central1-a

Updated property [compute/zone].

컨테이너 클러스터 만들기

gcloud container clusters create bootcamp --num-nodes 5 --scopes "https://www.googleapis.com/auth/projecthosting,storage-rw"

WARNING: Starting in January 2021, clusters will use the Regular release channel by default when `--cluster-version`, `--release-channel`, `--no-enable-autoupgrade`, and `--no-enable-autorepair` flags are not specified.

WARNING: Starting with version 1.18, clusters will have shielded GKE nodes by default.

NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS

bootcamp us-central1-a 1.20.8-gke.900 104.198.218.137 e2-medium 1.20.8-gke.900 5 RUNNING

단축 명령어와 정보 확인하기

alias k=kubectl

complete -F __start_kubectl k

k get ns,no,po,svc,deploy,rs,ing,ep

$ k get ns,no,po,svc,deploy,rs,ing,ep

NAME STATUS AGE

namespace/default Active 5m20s

namespace/kube-node-lease Active 5m22s

namespace/kube-public Active 5m22s

namespace/kube-system Active 5m22s

NAME STATUS ROLES AGE VERSION

node/gke-bootcamp-default-pool-f351b6bf-0dvf Ready <none> 4m31s v1.20.8-gke.900

node/gke-bootcamp-default-pool-f351b6bf-1n4p Ready <none> 4m28s v1.20.8-gke.900

node/gke-bootcamp-default-pool-f351b6bf-61sh Ready <none> 4m29s v1.20.8-gke.900

node/gke-bootcamp-default-pool-f351b6bf-kmg5 Ready <none> 4m30s v1.20.8-gke.900

node/gke-bootcamp-default-pool-f351b6bf-l5kg Ready <none> 4m30s v1.20.8-gke.900

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 5m20s

NAME ENDPOINTS AGE

endpoints/kubernetes 104.198.218.137:443 5m20s

다른 터미널로 모니터링 하기

watch -d 'kubectl get ns,no,po,svc,deploy,rs,ing,ep -o wide'

4개 있음.

네임 스페이스

노드

서비스

엔드포인트

NAME STATUS AGE

namespace/default Active 8m53s

namespace/kube-node-lease Active 8m55s

namespace/kube-public Active 8m55s

namespace/kube-system Active 8m55s

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME

node/gke-bootcamp-default-pool-f351b6bf-0dvf Ready <none> 8m4s v1.20.8-gke.900 10.128.0.5 35.225.42.121 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-1n4p Ready <none> 8m1s v1.20.8-gke.900 10.128.0.4 35.226.102.103 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-61sh Ready <none> 8m2s v1.20.8-gke.900 10.128.0.2 34.123.244.168 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-kmg5 Ready <none> 8m3s v1.20.8-gke.900 10.128.0.3 35.222.108.131 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-l5kg Ready <none> 8m3s v1.20.8-gke.900 10.128.0.6 34.71.140.207 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 8m53s <none>

NAME ENDPOINTS AGE

endpoints/kubernetes 104.198.218.137:443 8m53s

8. pod 상세 정보 보기

k get pod -o wide

<2> 배포 객체 보기

deployment 설명

kubectl explain deployment

kubectl explain deployment --recursive

kubectl explain deployment.metadata.name

<3> 배포 하기

vi deployments/auth.yaml

확인

cat deployments/auth.yaml

배포 만들기

kubectl create -f deployments/auth.yaml

deployment.apps/auth created

배포 확인

kubectl get deployments

NAME READY UP-TO-DATE AVAILABLE AGE

auth 0/1 1 0 7s

Pod 확인

kubectl get pods

NAME READY STATUS RESTARTS AGE

auth-777f7b6c94-shp24 1/1 Running 0 5m52s

전체 확인

kubectl get ns,no,po,svc,deploy,rs,ing,ep

7개 있음.

네임 스페이스

노드

pod

서비스

디플로이먼트

리플리카셋

엔드포인트

$ kubectl get ns,no,po,svc,deploy,rs,ing,ep

NAME STATUS AGE

namespace/default Active 17m

namespace/kube-node-lease Active 17m

namespace/kube-public Active 17m

namespace/kube-system Active 17m

NAME STATUS ROLES AGE VERSION

node/gke-bootcamp-default-pool-f351b6bf-0dvf Ready <none> 16m v1.20.8-gke.900

node/gke-bootcamp-default-pool-f351b6bf-1n4p Ready <none> 16m v1.20.8-gke.900

node/gke-bootcamp-default-pool-f351b6bf-61sh Ready <none> 16m v1.20.8-gke.900

node/gke-bootcamp-default-pool-f351b6bf-kmg5 Ready <none> 16m v1.20.8-gke.900

node/gke-bootcamp-default-pool-f351b6bf-l5kg Ready <none> 16m v1.20.8-gke.900

NAME READY STATUS RESTARTS AGE

pod/auth-777f7b6c94-shp24 1/1 Running 0 2m52s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 17m

NAME READY UP-TO-DATE AVAILABLE AGE

deployment.apps/auth 1/1 1 1 2m52s

NAME DESIRED CURRENT READY AGE

replicaset.apps/auth-777f7b6c94 1 1 1 2m52s

NAME ENDPOINTS AGE

endpoints/kubernetes 104.198.218.137:443 17m

배포

kubectl create -f services/auth.yaml

변경 모니터링

service/auth 생김

endpoints/kubernetes 생김

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

pod/auth-777f7b6c94-shp24 1/1 Running 0 8m41s 10.112.4.3 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR

service/auth ClusterIP 10.116.8.187 <none> 80/TCP 92s app=auth

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 23m <none>

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR

deployment.apps/auth 1/1 1 1 8m41s auth kelseyhightower/auth:1.0.0 app=auth

NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR

replicaset.apps/auth-777f7b6c94 1 1 1 8m41s auth kelseyhightower/auth:1.0.0 app=auth,pod-template-hash=777f7b6c94

NAME ENDPOINTS AGE

endpoints/auth 10.112.4.3:80 92s

endpoints/kubernetes 104.198.218.137:443 23m

kubectl create -f deployments/hello.yaml

kubectl create -f services/hello.yaml

hello 배포 후 확인 ?

pod/hello-b6 생김

service/hello 생김

deployment.apps/hello 생김

endpoints/hello 생김

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

pod/auth-777f7b6c94-shp24 1/1 Running 0 9m38s 10.112.4.3 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-b698b846f-kwq2l 1/1 Running 0 29s 10.112.1.4 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-kztrr 1/1 Running 0 29s 10.112.4.4 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-b698b846f-qbjhw 1/1 Running 0 29s 10.112.2.7 gke-bootcamp-default-pool-f351b6bf-l5kg <none> <none>

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR

service/auth ClusterIP 10.116.8.187 <none> 80/TCP 2m29s app=auth

service/hello ClusterIP 10.116.12.194 <none> 80/TCP 28s app=hello

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 23m <none>

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR

deployment.apps/auth 1/1 1 1 9m38s auth kelseyhightower/auth:1.0.0 app=auth

deployment.apps/hello 3/3 3 3 29s hello kelseyhightower/hello:1.0.0 app=hello

NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR

replicaset.apps/auth-777f7b6c94 1 1 1 9m38s auth kelseyhightower/auth:1.0.0 app=auth,pod-template-hash=777f7b6c94

replicaset.apps/hello-b698b846f 3 3 3 29s hello kelseyhightower/hello:1.0.0 app=hello,pod-template-hash=b698b846f

NAME ENDPOINTS AGE

endpoints/auth 10.112.4.3:80 2m29s

endpoints/hello 10.112.1.4:80,10.112.2.7:80,10.112.4.4:80 28s

endpoints/kubernetes 104.198.218.137:443 23m

front 배포 ?

프런트엔드용 ConfigMap 만듬

외부 IP를 가져와서 프런트엔드와 ip 매칭됨

kubectl create secret generic tls-certs --from-file tls/

kubectl create configmap nginx-frontend-conf --from-file=nginx/frontend.conf

kubectl create -f deployments/frontend.yaml

kubectl create -f services/frontend.yaml

kubectl get services frontend

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

pod/auth-777f7b6c94-shp24 1/1 Running 0 11m 10.112.4.3 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/frontend-6869899d96-cqd9g 1/1 Running 0 76s 10.112.1.5 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-kwq2l 1/1 Running 0 2m6s 10.112.1.4 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-kztrr 1/1 Running 0 2m6s 10.112.4.4 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-b698b846f-qbjhw 1/1 Running 0 2m6s 10.112.2.7 gke-bootcamp-default-pool-f351b6bf-l5kg <none> <none>

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR

service/auth ClusterIP 10.116.8.187 <none> 80/TCP 4m6s app=auth

service/frontend LoadBalancer 10.116.11.18 104.154.139.226 443:30670/TCP 75s app=frontend

service/hello ClusterIP 10.116.12.194 <none> 80/TCP 2m5s app=hello

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 25m <none>

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR

deployment.apps/auth 1/1 1 1 11m auth kelseyhightower/auth:1.0.0 app=auth

deployment.apps/frontend 1/1 1 1 76s nginx nginx:1.9.14 app=frontend

deployment.apps/hello 3/3 3 3 2m6s hello kelseyhightower/hello:1.0.0 app=hello

NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR

replicaset.apps/auth-777f7b6c94 1 1 1 11m auth kelseyhightower/auth:1.0.0 app=auth,pod-template-hash=777f7b6c94

replicaset.apps/frontend-6869899d96 1 1 1 76s nginx nginx:1.9.14 app=frontend,pod-template-hash=6869899d96

replicaset.apps/hello-b698b846f 3 3 3 2m6s hello kelseyhightower/hello:1.0.0 app=hello,pod-template-hash=b698b846f

NAME ENDPOINTS AGE

endpoints/auth 10.112.4.3:80 4m6s

endpoints/frontend 10.112.1.5:443 75s

curl -ks

https://<EXTERNAL-IP>

스크립트

curl -ks https://`kubectl get svc frontend -o=jsonpath="{.status.loadBalancer.ingress[0].ip}"

<3> 배포 확장

설명

kubectl explain deployment.spec.replicas

replicas 업데이트 ?

kubectl scale deployment hello --replicas=5

kubectl get ns,no,po,svc,deploy,rs,ing,ep -o wide cs-

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

pod/auth-777f7b6c94-shp24 1/1 Running 0 15m 10.112.4.3 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/frontend-6869899d96-cqd9g 1/1 Running 0 5m49s 10.112.1.5 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-kwq2l 1/1 Running 0 6m39s 10.112.1.4 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-kztrr 1/1 Running 0 6m39s 10.112.4.4 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-b698b846f-qbjhw 1/1 Running 0 6m39s 10.112.2.7 gke-bootcamp-default-pool-f351b6bf-l5kg <none> <none>

pod/hello-b698b846f-vshmf 1/1 Running 0 84s 10.112.0.3 gke-bootcamp-default-pool-f351b6bf-61sh <none> <none>

pod/hello-b698b846f-z5kb8 1/1 Running 0 84s 10.112.3.3 gke-bootcamp-default-pool-f351b6bf-0dvf <none> <none>

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR

service/auth ClusterIP 10.116.8.187 <none> 80/TCP 8m39s app=auth

service/frontend LoadBalancer 10.116.11.18 104.154.139.226 443:30670/TCP 5m48s app=frontend

service/hello ClusterIP 10.116.12.194 <none> 80/TCP 6m38s app=hello

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 30m <none>

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR

deployment.apps/auth 1/1 1 1 15m auth kelseyhightower/auth:1.0.0 app=auth

deployment.apps/frontend 1/1 1 1 5m49s nginx nginx:1.9.14 app=frontend

deployment.apps/hello 5/5 5 5 6m39s hello kelseyhightower/hello:1.0.0 app=hello

NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR

replicaset.apps/auth-777f7b6c94 1 1 1 15m auth kelseyhightower/auth:1.0.0 app=auth,pod-template-hash=777f7b6c94

replicaset.apps/frontend-6869899d96 1 1 1 5m49s nginx nginx:1.9.14 app=frontend,pod-template-hash=6869899d96

replicaset.apps/hello-b698b846f 5 5 5 6m39s hello kelseyhightower/hello:1.0.0 app=hello,pod-template-hash=b698b846f

pod 증가 확인

kubectl get pods | grep hello- | wc -l

pod 축소 확인

kubectl get pods | grep hello- | wc -l

<4> 순차적 업데이트 하기

이미지 변경

kubectl edit deployment hello

hello 2.0으로

저장 하는 순간 업데이트 배포가 시작 됨

replicaset 확인 ?

kubectl get replicaset

NAME DESIRED CURRENT READY AGE

auth-777f7b6c94 1 1 1 20m

frontend-6869899d96 1 1 1 10m

hello-7fd855c77d 2 2 1 23s

hello-b698b846f 2 2 2 11m

변경 히스토리 확인

kubectl rollout history deployment/hello

deployment.apps/hello

REVISION CHANGE-CAUSE

1 <none>

2 <none>

업데이트 일시 중지

$ kubectl rollout pause deployment/hello

deployment.apps/hello paused

현재 상태 확인

$ kubectl rollout status deployment/hello

deployment "hello" successfully rolled out

pod에서 확인

$ kubectl get pods -o jsonpath --template='{range .items[*]}{.metadata.name}{"\t"}{"\t"}{.spec.containers[0].image}{"\n"}{end}'

auth-777f7b6c94-shp24 kelseyhightower/auth:1.0.0

frontend-6869899d96-cqd9g nginx:1.9.14

hello-7fd855c77d-5vmrb kelseyhightower/hello:2.0.0

hello-7fd855c77d-fcnx7 kelseyhightower/hello:2.0.0

hello-7fd855c77d-j22km kelseyhightower/hello:2.0.0

확인2

kubectl get ns,no,po,svc,deploy,rs,ing,ep -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

pod/auth-777f7b6c94-shp24 1/1 Running 0 22m 10.112.4.3 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/frontend-6869899d96-cqd9g 1/1 Running 0 12m 10.112.1.5 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-7fd855c77d-5vmrb 1/1 Running 0 2m53s 10.112.4.5 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-7fd855c77d-fcnx7 1/1 Running 0 2m18s 10.112.0.4 gke-bootcamp-default-pool-f351b6bf-61sh <none> <none>

pod/hello-7fd855c77d-j22km 1/1 Running 0 2m35s 10.112.3.4 gke-bootcamp-default-pool-f351b6bf-0dvf <none> <none>

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR

service/auth ClusterIP 10.116.8.187 <none> 80/TCP 15m app=auth

service/frontend LoadBalancer 10.116.11.18 104.154.139.226 443:30670/TCP 12m app=frontend

service/hello ClusterIP 10.116.12.194 <none> 80/TCP 13m app=hello

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 37m <none>

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR

deployment.apps/auth 1/1 1 1 22m auth kelseyhightower/auth:1.0.0 app=auth

deployment.apps/frontend 1/1 1 1 12m nginx nginx:1.9.14 app=frontend

deployment.apps/hello 3/3 3 3 13m hello kelseyhightower/hello:2.0.0 app=hello

NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR

replicaset.apps/auth-777f7b6c94 1 1 1 22m auth kelseyhightower/auth:1.0.0 app=auth,pod-template-hash=777f7b6c94

replicaset.apps/frontend-6869899d96 1 1 1 12m nginx nginx:1.9.14 app=frontend,pod-template-hash=6869899d96

replicaset.apps/hello-7fd855c77d 3 3 3 2m53s hello kelseyhightower/hello:2.0.0 app=hello,pod-template-hash=7fd855c77d

replicaset.apps/hello-b698b846f 0 0 0 13m hello kelseyhightower/hello:1.0.0 app=hello,pod-template-hash=b698b846f

NAME ENDPOINTS AGE

<5> 배포 계속 하기

resume으로 배포 계속 하기

$ kubectl rollout resume deployment/hello

deployment.apps/hello resumed

출시 완료 상태

$ kubectl rollout status deployment/hello

deployment "hello" successfully rolled out

<6> 롤백

undo로 롤백

$ kubectl rollout undo deployment/hello

deployment.apps/hello rolled back

롤백 확인1

$ kubectl rollout history deployment/hello

deployment.apps/hello

REVISION CHANGE-CAUSE

2 <none>

3 <none>

롤백 확인2

kubectl get pods -o jsonpath --template='{range .items[*]}{.metadata.name}{"\t"}{"\t"}{.spec.containers[0].image}{"\n"}{end}'

$ kubectl get pods -o jsonpath --template='{range .items[*]}{.metadata.name}{"\t"}{"\t"}{.spec.containers[0].image}{"\n"}{end}'

auth-777f7b6c94-shp24 kelseyhightower/auth:1.0.0

frontend-6869899d96-cqd9g nginx:1.9.14

hello-7fd855c77d-5vmrb kelseyhightower/hello:2.0.0

hello-b698b846f-8v45g kelseyhightower/hello:1.0.0

hello-b698b846f-kdpkg kelseyhightower/hello:1.0.0

hello-b698b846f-ztr5g kelseyhightower/hello:1.0.0

<7> 카나리 배포

조금 배포해서 상태 보기

cat deployments/hello-canary.yaml

kubectl create -f deployments/hello-canary.yaml

카나리 배포 확인

디플로이 먼트 확인

$ kubectl get deployments

NAME READY UP-TO-DATE AVAILABLE AGE

auth 1/1 1 1 26m

frontend 1/1 1 1 17m

hello 3/3 3 3 17m

hello-canary 1/1 1 1 47s

전체 확인

kubectl get ns,no,po,svc,deploy,rs,ing,ep -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

pod/auth-777f7b6c94-shp24 1/1 Running 0 27m 10.112.4.3 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/frontend-6869899d96-cqd9g 1/1 Running 0 17m 10.112.1.5 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-8v45g 1/1 Running 0 3m3s 10.112.1.6 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-kdpkg 1/1 Running 0 2m48s 10.112.2.8 gke-bootcamp-default-pool-f351b6bf-l5kg <none> <none>

pod/hello-b698b846f-ztr5g 1/1 Running 0 2m35s 10.112.4.6 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-canary-67bf66fdc6-wwlt4 1/1 Running 0 82s 10.112.4.7 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR

service/auth ClusterIP 10.116.8.187 <none> 80/TCP 20m app=auth

service/frontend LoadBalancer 10.116.11.18 104.154.139.226 443:30670/TCP 17m app=frontend

service/hello ClusterIP 10.116.12.194 <none> 80/TCP 18m app=hello

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 41m <none>

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR

deployment.apps/auth 1/1 1 1 27m auth kelseyhightower/auth:1.0.0 app=auth

deployment.apps/frontend 1/1 1 1 17m nginx nginx:1.9.14 app=frontend

deployment.apps/hello 3/3 3 3 18m hello kelseyhightower/hello:1.0.0 app=hello

deployment.apps/hello-canary 1/1 1 1 82s hello kelseyhightower/hello:2.0.0 app=hello

NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR

replicaset.apps/auth-777f7b6c94 1 1 1 27m auth kelseyhightower/auth:1.0.0 app=auth,pod-template-hash=777f7b6c94

replicaset.apps/frontend-6869899d96 1 1 1 17m nginx nginx:1.9.14 app=frontend,pod-template-hash=6869899d96

replicaset.apps/hello-7fd855c77d 0 0 0 7m33s hello kelseyhightower/hello:2.0.0 app=hello,pod-template-hash=7fd855c77d

replicaset.apps/hello-b698b846f 3 3 3 18m hello kelseyhightower/hello:1.0.0 app=hello,pod-template-hash=b698b846f

replicaset.apps/hello-canary-67bf66fdc6 1 1 1 82s hello kelseyhightower/hello:2.0.0 app=hello,pod-template-hash=67bf66fdc6

<8> Blue/Green 배포

2배의 리소스가 필요

blue 배포

kubectl apply -f services/hello-blue.yaml

새로운 'green' 배포 만들기

Green 에서 버전 라벨과 이미지 경로를 업데이트

watch -d 'kubectl get ns,no,po,svc,deploy,rs,ing,ep -o wide'

Every 2.0s: kubectl get ns,no,po,svc,deploy,rs,ing,ep -o wide cs-646454401406-default-default-s8qwv: Tue Aug 17 00:25:19 2021

NAME STATUS AGE

namespace/default Active 48m

namespace/kube-node-lease Active 48m

namespace/kube-public Active 48m

namespace/kube-system Active 48m

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIMEnode/gke-bootcamp-default-pool-f351b6bf-0dvf Ready <none> 47m v1.20.8-gke.900 10.128.0.5 35.225.42.121 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-1n4p Ready <none> 47m v1.20.8-gke.900 10.128.0.4 35.226.102.103 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-61sh Ready <none> 47m v1.20.8-gke.900 10.128.0.2 34.123.244.168 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-kmg5 Ready <none> 47m v1.20.8-gke.900 10.128.0.3 35.222.108.131 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-l5kg Ready <none> 47m v1.20.8-gke.900 10.128.0.6 34.71.140.207 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

pod/auth-777f7b6c94-shp24 1/1 Running 0 34m 10.112.4.3 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/frontend-6869899d96-cqd9g 1/1 Running 0 24m 10.112.1.5 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-8v45g 1/1 Running 0 9m32s 10.112.1.6 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-kdpkg 1/1 Running 0 9m17s 10.112.2.8 gke-bootcamp-default-pool-f351b6bf-l5kg <none> <none>

pod/hello-b698b846f-ztr5g 1/1 Running 0 9m4s 10.112.4.6 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-canary-67bf66fdc6-wwlt4 1/1 Running 0 7m51s 10.112.4.7 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR

service/auth ClusterIP 10.116.8.187 <none> 80/TCP 26m app=auth

service/frontend LoadBalancer 10.116.11.18 104.154.139.226 443:30670/TCP 24m app=frontend

service/hello ClusterIP 10.116.12.194 <none> 80/TCP 24m app=hello,version=1.0.0

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 48m <none>

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR

deployment.apps/auth 1/1 1 1 34m auth kelseyhightower/auth:1.0.0 app=auth

deployment.apps/frontend 1/1 1 1 24m nginx nginx:1.9.14 app=frontend

deployment.apps/hello 3/3 3 3 24m hello kelseyhightower/hello:1.0.0 app=hello

deployment.apps/hello-canary 1/1 1 1 7m51s hello kelseyhightower/hello:2.0.0 app=hello

NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR

replicaset.apps/auth-777f7b6c94 1 1 1 34m auth kelseyhightower/auth:1.0.0 app=auth,pod-template-hash=777f7b6c94

replicaset.apps/frontend-6869899d96 1 1 1 24m nginx nginx:1.9.14 app=frontend,pod-template-hash=6869899d96

replicaset.apps/hello-7fd855c77d 0 0 0 14m hello kelseyhightower/hello:2.0.0 app=hello,pod-template-hash=7fd855c77d

replicaset.apps/hello-b698b846f 3 3 3 24m hello kelseyhightower/hello:1.0.0 app=hello,pod-template-hash=b698b846f

replicaset.apps/hello-canary-67bf66fdc6 1 1 1 7m51s hello kelseyhightower/hello:2.0.0 app=hello,pod-template-hash=67bf66fdc6

그린 배포

kubectl create -f deployments/hello-green.yaml

$ kubectl create -f deployments/hello-green.yaml

deployment.apps/hello-green created

버전 1.0 확인

$ curl -ks https://`kubectl get svc frontend -o=jsonpath="{.status.loadBalancer.ingress[0].ip}"`/version

{"version":"1.0.0"}

새버전으로 업데이트

$ kubectl apply -f services/hello-green.yaml

service/hello configured

버전확인 ?

$ curl -ks https://`kubectl get svc frontend -o=jsonpath="{.status.loadBalancer.ingress[0].ip}"`/version

{"version":"2.0.0"}

상태 확인

Every 2.0s: kubectl get ns,no,po,svc,deploy,rs,ing,ep -o wide

NAME STATUS AGE

namespace/default Active 52m

namespace/kube-node-lease Active 52m

namespace/kube-public Active 52m

namespace/kube-system Active 52m

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME

node/gke-bootcamp-default-pool-f351b6bf-0dvf Ready <none> 51m v1.20.8-gke.900 10.128.0.5 35.225.42.121 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-1n4p Ready <none> 51m v1.20.8-gke.900 10.128.0.4 35.226.102.103 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-61sh Ready <none> 51m v1.20.8-gke.900 10.128.0.2 34.123.244.168 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-kmg5 Ready <none> 51m v1.20.8-gke.900 10.128.0.3 35.222.108.131 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-l5kg Ready <none> 51m v1.20.8-gke.900 10.128.0.6 34.71.140.207 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

pod/auth-777f7b6c94-shp24 1/1 Running 0 37m 10.112.4.3 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/frontend-6869899d96-cqd9g 1/1 Running 0 27m 10.112.1.5 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-8v45g 1/1 Running 0 13m 10.112.1.6 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-kdpkg 1/1 Running 0 13m 10.112.2.8 gke-bootcamp-default-pool-f351b6bf-l5kg <none> <none>

pod/hello-b698b846f-ztr5g 1/1 Running 0 12m 10.112.4.6 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-canary-67bf66fdc6-wwlt4 1/1 Running 0 11m 10.112.4.7 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-green-5f9cb59bcf-gv758 1/1 Running 0 2m20s 10.112.3.5 gke-bootcamp-default-pool-f351b6bf-0dvf <none> <none>

pod/hello-green-5f9cb59bcf-m2p8n 1/1 Running 0 2m20s 10.112.0.5 gke-bootcamp-default-pool-f351b6bf-61sh <none> <none>

pod/hello-green-5f9cb59bcf-vrcgb 1/1 Running 0 2m20s 10.112.1.7 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR

service/auth ClusterIP 10.116.8.187 <none> 80/TCP 30m app=auth

service/frontend LoadBalancer 10.116.11.18 104.154.139.226 443:30670/TCP 27m app=frontend

service/hello ClusterIP 10.116.12.194 <none> 80/TCP 28m app=hello,version=2.0.0

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 52m <none>

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR

deployment.apps/auth 1/1 1 1 37m auth kelseyhightower/auth:1.0.0 app=auth

deployment.apps/frontend 1/1 1 1 27m nginx nginx:1.9.14 app=frontend

deployment.apps/hello 3/3 3 3 28m hello kelseyhightower/hello:1.0.0 app=hello

deployment.apps/hello-canary 1/1 1 1 11m hello kelseyhightower/hello:2.0.0 app=hello

블루로 롤백

$ kubectl apply -f services/hello-blue.yaml

service/hello configured

$ curl -ks https://`kubectl get svc frontend -o=jsonpath="{.status.loadBalancer.ingress[0].ip}"`/version

{"version":"1.0.0"}

kubectl get ns,no,po,svc,deploy,rs,ing,ep -o wide

NAME STATUS AGE

namespace/default Active 53m

namespace/kube-node-lease Active 53m

namespace/kube-public Active 53m

namespace/kube-system Active 53m

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME

node/gke-bootcamp-default-pool-f351b6bf-0dvf Ready <none> 53m v1.20.8-gke.900 10.128.0.5 35.225.42.121 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-1n4p Ready <none> 52m v1.20.8-gke.900 10.128.0.4 35.226.102.103 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-61sh Ready <none> 52m v1.20.8-gke.900 10.128.0.2 34.123.244.168 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-kmg5 Ready <none> 52m v1.20.8-gke.900 10.128.0.3 35.222.108.131 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

node/gke-bootcamp-default-pool-f351b6bf-l5kg Ready <none> 52m v1.20.8-gke.900 10.128.0.6 34.71.140.207 Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

pod/auth-777f7b6c94-shp24 1/1 Running 0 39m 10.112.4.3 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/frontend-6869899d96-cqd9g 1/1 Running 0 29m 10.112.1.5 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-8v45g 1/1 Running 0 14m 10.112.1.6 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

pod/hello-b698b846f-kdpkg 1/1 Running 0 14m 10.112.2.8 gke-bootcamp-default-pool-f351b6bf-l5kg <none> <none>

pod/hello-b698b846f-ztr5g 1/1 Running 0 14m 10.112.4.6 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-canary-67bf66fdc6-wwlt4 1/1 Running 0 13m 10.112.4.7 gke-bootcamp-default-pool-f351b6bf-1n4p <none> <none>

pod/hello-green-5f9cb59bcf-gv758 1/1 Running 0 3m55s 10.112.3.5 gke-bootcamp-default-pool-f351b6bf-0dvf <none> <none>

pod/hello-green-5f9cb59bcf-m2p8n 1/1 Running 0 3m55s 10.112.0.5 gke-bootcamp-default-pool-f351b6bf-61sh <none> <none>

pod/hello-green-5f9cb59bcf-vrcgb 1/1 Running 0 3m55s 10.112.1.7 gke-bootcamp-default-pool-f351b6bf-kmg5 <none> <none>

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR

service/auth ClusterIP 10.116.8.187 <none> 80/TCP 32m app=auth

service/frontend LoadBalancer 10.116.11.18 104.154.139.226 443:30670/TCP 29m app=frontend

service/hello ClusterIP 10.116.12.194 <none> 80/TCP 30m app=hello,version=1.0.0

service/kubernetes ClusterIP 10.116.0.1 <none> 443/TCP 53m <none>

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR

deployment.apps/auth 1/1 1 1 39m auth kelseyhightower/auth:1.0.0 app=auth

deployment.apps/frontend 1/1 1 1 29m nginx nginx:1.9.14 app=frontend

deployment.apps/hello 3/3 3 3 30m hello kelseyhightower/hello:1.0.0 app=hello

deployment.apps/hello-canary 1/1 1 1 13m hello kelseyhightower/hello:2.0.0 app=hello

<10> 같이 보면 좋을 사이트

https://brunch.co.kr/@topasvga/1800

157.(몰아보기) 구글 클라우드 쿠버네티스

<1> 구글 클라우드 쿠버네티스 - 2021 <2> 구글 클라우드 쿠버네티스 - 2020 <3> (몰아보기) 쿠버네티스 , AWS EKS , GCP GKE 사용 <1> 구글 클라우드 쿠버네티스 - 2021 https://brunch.co.kr/@topasvga/1802 https://brunch.

brunch.co.kr/@topasvga/1800

감사합니다.

keyword