11. 오라클 클라우드 - 쿠버네티스 -CLB, NLB
구조
외부------- Load Balancer --------- Node1, Node2 등 각 포트에 접속 ------- 클러스터 -- Pod 접속
<1> 디플로이 먼트 생성
<2> nlb, clb 생성
<3> 동작 확인
<4> 안되는것 ?
<5> 삭제
<1> 디플로이 먼트 생성
1
node ip
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
10.0.10.141 Ready node 6d23h v1.25.4 10.0.10.141 <none> Oracle Linux Server 8.6 5.4.17-2136.314.6.2.el8uek.x86_64 cri-o://1.25.1-111.el8
10.0.10.80 Ready node 6d23h v1.25.4 10.0.10.80 <none> Oracle Linux Server 8.6 5.4.17-2136.314.6.2.el8uek.x86_64 cri-o://1.25.1-111.el8
2
topasvga@cloudshell:~ (ap-seoul-1)$ cat << EOF > echo-pod.yaml
> apiVersion: apps/v1
> kind: Deployment
> metadata:
> name: deploy-echo
> spec:
> replicas: 3
> selector:
> matchLabels:
> app: deploy-websrv
> template:
> metadata:
> labels:
> app: deploy-websrv
> spec:
> containers:
> - name: cndk-websrv
> image: k8s.gcr.io/echoserver:1.5
> ports:
> - containerPort: 8080
> EOF
2
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl apply -f echo-pod.yaml
deployment.apps/deploy-echo created
3
pod ip 확인
ip가 3개 모두 나오는데 1분정도 걸린다.
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl get pod -o wide -l app=deploy-websrv |awk 'NR>1 {print $6}'
<none>
<none>
10.244.0.167
4
파드 세부 정보 확인
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
deploy-echo-84888f6548-974j8 1/1 Running 0 20s 10.244.0.66 10.0.10.80 <none> <none>
deploy-echo-84888f6548-stlkr 1/1 Running 0 20s 10.244.0.67 10.0.10.80 <none> <none>
deploy-echo-84888f6548-xp8lj 1/1 Running 0 20s 10.244.0.167 10.0.10.141 <none> <none>
5
pod ip 확인
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl get pod -o wide -l app=deploy-websrv |awk 'NR>1 {print $6}'
10.244.0.66
10.244.0.67
10.244.0.167
6
pod 이름 확인
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl get pod -l app=deploy-websrv -o=name
pod/deploy-echo-84888f6548-974j8
pod/deploy-echo-84888f6548-stlkr
pod/deploy-echo-84888f6548-xp8lj
7
topasvga@cloudshell:~ (ap-seoul-1)$ for pod in $(kubectl get pod -o wide -l app=deploy-websrv |awk 'NR>1 {print $6}'); do curl -s $pod:8080 | egrep '(Hostname|nginx|client_address)' ; done
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl logs -l app=deploy-websrv -f
8
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl get deploy,svc,ep,pods
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/deploy-echo 3/3 3 3 6m33s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 9m27s
NAME ENDPOINTS AGE
endpoints/oracle.com-oci <none> 38m
NAME READY STATUS RESTARTS AGE
pod/deploy-echo-84888f6548-974j8 1/1 Running 0 6m33s
pod/deploy-echo-84888f6548-stlkr 1/1 Running 0 6m33s
pod/deploy-echo-84888f6548-xp8lj 1/1 Running 0 6m33s
<2> nlb, clb 생성
1
topasvga@cloudshell:~ (ap-seoul-1)$ cat << EOF > aws-svc-clb.yaml
> apiVersion: v1
> kind: Service
> metadata:
> name: aws-svc-clb
> spec:
> ports:
> - name: awsclb-webport
> port: 80
> targetPort: 8080
> nodePort: 30000
> selector:
> app: deploy-websrv
> type: LoadBalancer
> EOF
2
nlb 생성
topasvga@cloudshell:~ (ap-seoul-1)$ cat << EOF > aws-svc-nlb.yaml
> apiVersion: v1
> kind: Service
> metadata:
> name: aws-svc-nlb
> annotations:
> service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
> spec:
> ports:
> - name: awsnlb-webport
> port: 80
> targetPort: 8080
> nodePort: 31000
> selector:
> app: deploy-websrv
> type: LoadBalancer
> EOF
3
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl apply -f aws-svc-clb.yaml
service/aws-svc-clb created
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl apply -f aws-svc-nlb.yaml
service/aws-svc-nlb created
4
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl get svc -o wide
(1분 걸림)
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
aws-svc-clb LoadBalancer 10.96.149.52 <pending> 80:30000/TCP 26s app=deploy-websrv
aws-svc-nlb LoadBalancer 10.96.235.20 <pending> 80:31000/TCP 24s app=deploy-websrv
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 11m <none>
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
aws-svc-clb LoadBalancer 10.96.149.52 152.70.239.14 80:30000/TCP 98s app=deploy-websrv
aws-svc-nlb LoadBalancer 10.96.235.20 131.186.23.211 80:31000/TCP 96s app=deploy-websrv
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 12m <none>
// 오라클 클라우드는 EXTERNAL-IP가 공인 ip로 나온다.
// AWS 는 EXTERNAL-IP가 도메인으로 나온다.
5
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl get endpoints
NAME ENDPOINTS AGE
aws-svc-clb 10.244.0.167:8080,10.244.0.66:8080,10.244.0.67:8080 106s
aws-svc-nlb 10.244.0.167:8080,10.244.0.66:8080,10.244.0.67:8080 104s
oracle.com-oci <none> 41m
6
topasvga@cloudshell:~ (ap-seoul-1)$ kubectl logs -l app=deploy-websrv -f
<3> 동작 확인
1
외부에 서버 1대에 접속하여 확인
putty
login as: opc
2
[opc@ora-pub1 ~]$ EXIP1=152.70.239.14
[opc@ora-pub1 ~]$ EXIP2=131.186.23.211
3
[opc@ora-pub1 ~]$ curl -s --connect-timeout 1 $EXIP1 | egrep '(Hostname|nginx|client_address)'
Hostname: deploy-echo-84888f6548-stlkr
server_version=nginx: 1.13.0 - lua: 10008
client_address=10.244.0.1
[opc@ora-pub1 ~]$ curl -s --connect-timeout 1 $EXIP2 | egrep '(Hostname|nginx|client_address)'
Hostname: deploy-echo-84888f6548-974j8
server_version=nginx: 1.13.0 - lua: 10008
client_address=10.244.0.1
4
lb 분산 잘 된다.
[opc@ora-pub1 ~]$ for i in {1..100}; do curl -s $EXIP1 | grep Hostname ; done | sort | uniq -c | sort -nr
36 Hostname: deploy-echo-84888f6548-stlkr
35 Hostname: deploy-echo-84888f6548-xp8lj
29 Hostname: deploy-echo-84888f6548-974j8
[opc@ora-pub1 ~]$ for i in {1..100}; do curl -s $EXIP2 | grep Hostname ; done | sort | uniq -c | sort -nr
36 Hostname: deploy-echo-84888f6548-xp8lj
33 Hostname: deploy-echo-84888f6548-stlkr
31 Hostname: deploy-echo-84888f6548-974j8
[opc@ora-pub1 ~]$ for i in {1..100}; do curl -s $EXIP1 | grep client_address ; done | sort | uniq -c | sort -nr
33 client_address=10.244.0.1
29 client_address=10.244.0.128
21 client_address=10.244.0.129
17 client_address=10.244.0.0
[opc@ora-pub1 ~]$ for i in {1..100}; do curl -s $EXIP2 | grep client_address ; done | sort | uniq -c | sort -nr
40 client_address=10.244.0.128
30 client_address=10.244.0.1
20 client_address=10.244.0.0
10 client_address=10.244.0.129
[opc@ora-pub1 ~]$ while true; do curl -s --connect-timeout 1 $EXIP1 | egrep '(Hostname|nginx|client_address)'; echo "--------------" ; date "+%Y-%m-%d %H:%M:%S" ; sleep 1; done
Hostname: deploy-echo-84888f6548-974j8
server_version=nginx: 1.13.0 - lua: 10008
client_address=10.244.0.1
--------------
2023-02-21 10:05:40
Hostname: deploy-echo-84888f6548-974j8
server_version=nginx: 1.13.0 - lua: 10008
client_address=10.244.0.128
--------------
2023-02-21 10:05:41
Hostname: deploy-echo-84888f6548-stlkr
server_version=nginx: 1.13.0 - lua: 10008
client_address=10.244.0.128
--------------
2023-02-21 10:05:42
Hostname: deploy-echo-84888f6548-xp8lj
server_version=nginx: 1.13.0 - lua: 10008
client_address=10.244.0.0
--------------
[opc@ora-pub1 ~]$ while true; do curl -s --connect-timeout 1 $EXIP2 | egrep '(Hostname|nginx|client_address)'; echo "--------------" ; date "+%Y-%m-%d %H:%M:%S" ; sleep 1; done
Hostname: deploy-echo-84888f6548-stlkr
server_version=nginx: 1.13.0 - lua: 10008
client_address=10.244.0.128
--------------
2023-02-21 10:05:54
Hostname: deploy-echo-84888f6548-stlkr
server_version=nginx: 1.13.0 - lua: 10008
client_address=10.244.0.1
--------------
2023-02-21 10:05:55
Hostname: deploy-echo-84888f6548-stlkr
server_version=nginx: 1.13.0 - lua: 10008
client_address=10.244.0.128
--------------
<4> 안되는것 ?
1
안되는것 ?
HTTP/HTTS 처리
TLS 종료
도메인 기반 라우팅 처리
2
대안?
인그레스
HTTP/HTTPS 처리 가능.
하지만, TCP/UDP 처리 불가여서 TCP/UDP 처리 위해서는 LoadBalancer 사용해야 한다.
<5> 삭제
kubectl delete deploy,svc --all
감사합니다.
