39탄-3. EKS 기초-네트워크 - 로드 밸런서
다음은 EKS 생성된 상태에서 진행합니다.
AWS 네트워크 워크셥을 참고해서 작성합니다.
https://www.eksworkshop.com/docs/fundamentals/exposing/loadbalancer/
<1> 명령서버 Cloud9 ,실습 서버 EKS를 만들기
<2> cloud9에서 미리 환경 설정
<3> 로드밸런서 생성
<4> IP모드로 변경해보자
<5> pod를 3개로 늘려보자.
<6> 인그레스 만들어보자.
<7> 커스템 네트워킹
<8> 삭제
<1> 명령서버 Cloud9 ,실습 서버 EKS를 만들기
1
Cloud9 만들자.
2
EKS 만들자.
아래 메뉴얼 참고
https://brunch.co.kr/@topasvga/3518
<2> cloud9에서 미리 환경 설정
1
prepare-environment exposing/load-balancer
2
설정된 환경 확인
kubectl get svc -l app.kubernetes.io/created-by=eks-workshop -A
kubectl -n ui describe service ui
kubectl -n ui describe service ui
<3> 로드밸런서 생성
~/environment/eks-workshop/modules/exposing/load-balancer/nlb/nlb.yaml
apiVersion: v1
kind: Service
metadata:
name: ui-nlb
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: external
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
namespace: ui
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 8080
name: http
selector:
app.kubernetes.io/name: ui
app.kubernetes.io/instance: ui
app.kubernetes.io/component: service
kubectl apply -k ~/environment/eks-workshop/modules/exposing/load-balancer/nlb
3
ec2-user:~/environment $ kubectl get service -n ui
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ui ClusterIP 172.20.171.116 <none> 80/TCP 7m31s
ui-nlb LoadBalancer 172.20.169.45 k8s-ui-uinlb-2f82a3af0a-665145a02b51657e.elb.ap-northeast-2.amazonaws.com 80:31942/TCP 25s
aws elbv2 describe-load-balancers --query 'LoadBalancers[?contains(LoadBalancerName, `k8s-ui-uinlb`) == `true`]'
ALB_ARN=$(aws elbv2 describe-load-balancers --query 'LoadBalancers[?contains(LoadBalancerName, `k8s-ui-uinlb`) == `true`].LoadBalancerArn' | jq -r '.[0]')
TARGET_GROUP_ARN=$(aws elbv2 describe-target-groups --load-balancer-arn $ALB_ARN | jq -r '.TargetGroups[0].TargetGroupArn')
aws elbv2 describe-target-health --target-group-arn $TARGET_GROUP_ARN
kubectl get service -n ui ui-nlb -o jsonpath="{.status.loadBalancer.ingress[*].hostname}{'\n'}"
wait-for-lb $(kubectl get service -n ui ui-nlb -o jsonpath="{.status.loadBalancer.ingress[*].hostname}{'\n'}")
콘솔에서 로드밸런서 확인하자!!!
로드밸런서를 웹브라우저로 접속 해보자~
<4> IP모드로 변경해보자
~/environment/eks-workshop/modules/exposing/load-balancer/ip-mode/nlb.yaml
apiVersion: v1
kind: Service
metadata:
name: ui-nlb
annotations:
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
namespace: ui
kubectl apply -k ~/environment/eks-workshop/modules/exposing/load-balancer/ip-mode
service/ui-nlb configured
콘솔에서 보자.
kubectl describe service/ui-nlb -n ui
ALB_ARN=$(aws elbv2 describe-load-balancers --query 'LoadBalancers[?contains(LoadBalancerName, `k8s-ui-uinlb`) == `true`].LoadBalancerArn' | jq -r '.[0]')
TARGET_GROUP_ARN=$(aws elbv2 describe-target-groups --load-balancer-arn $ALB_ARN | jq -r '.TargetGroups[0].TargetGroupArn')
aws elbv2 describe-target-health --target-group-arn $TARGET_GROUP_ARN
콘솔에서 보자.
<5> pod를 3개로 늘려보자.
kubectl scale -n ui deployment/ui --replicas=3
kubectl wait --for=condition=Ready pod -n ui -l app.kubernetes.io/name=ui --timeout=60s
ALB_ARN=$(aws elbv2 describe-load-balancers --query 'LoadBalancers[?contains(LoadBalancerName, `k8s-ui-uinlb`) == `true`].LoadBalancerArn' | jq -r '.[0]')
TARGET_GROUP_ARN=$(aws elbv2 describe-target-groups --load-balancer-arn $ALB_ARN | jq -r '.TargetGroups[0].TargetGroupArn')
aws elbv2 describe-target-health --target-group-arn $TARGET_GROUP_ARN
wait-for-lb $(kubectl get service -n ui ui-nlb -o jsonpath="{.status.loadBalancer.ingress[*].hostname}{'\n'}")
<6> 인그레스 만들어보자.
1
prepare-environment exposing/ingress
2
ec2-user:~/environment $ kubectl get ingress -n ui
No resources found in ui namespace.
ec2-user:~/environment $ kubectl get svc -n ui
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ui ClusterIP 172.20.171.116 <none> 80/TCP 27m
ec2-user:~/environment $ kubectl apply -k ~/environment/eks-workshop/modules/exposing/ingress/creating-ingress
ingress.networking.k8s.io/ui created
ec2-user:~/environment $ kubectl get ingress ui -n ui
NAME CLASS HOSTS ADDRESS PORTS AGE
ui alb * k8s-ui-ui-5ddc3ba496-1337242164.ap-northeast-2.elb.amazonaws.com 80 9s
시간이 좀 걸린다.
<7> 커스템 네트워킹
네트워킹탭
https://www.eksworkshop.com/docs/networking/custom-networking/
prepare-environment networking/custom-networking
환경을 enable 해줘야함!!!
kubectl set env daemonset aws-node -n kube-system AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true
상세
1
ec2-user:~/environment $
prepare-environment networking/custom-networking
Refreshing copy of workshop repository from GitHub...
Resetting the environment, please wait
Waiting for application to become ready...
Cleaning up previous lab infrastructure...
Creating infrastructure for next lab...
bash: line 130: EKS_DEFAULT_MNG_MIN: unbound variable
ec2-user:~/environment $ aws ec2 describe-vpcs --vpc-ids $VPC_ID
{
"Vpcs": [
{
"CidrBlock": "10.42.0.0/16",
"DhcpOptionsId": "dopt-0e1344cc95c5b04fa",
"State": "available",
"VpcId": "vpc-0ec884e49aaae5818",
"OwnerId": "319485572629",
"InstanceTenancy": "default",
"CidrBlockAssociationSet": [
{
"AssociationId": "vpc-cidr-assoc-017bd2856e6a729a7",
"CidrBlock": "10.42.0.0/16",
"CidrBlockState": {
"State": "associated"
}
},
{
"AssociationId": "vpc-cidr-assoc-0144325ff4751e64e",
"CidrBlock": "100.64.0.0/16",
"CidrBlockState": {
"State": "associated"
}
}
],
"IsDefault": false,
"Tags": [
{
"Key": "alpha.eksctl.io/eksctl-version",
"Value": "0.144.0"
},
{
"Key": "aws:cloudformation:logical-id",
"Value": "VPC"
},
{
"Key": "alpha.eksctl.io/cluster-name",
"Value": "eks-workshop"
},
{
"Key": "alpha.eksctl.io/cluster-oidc-enabled",
"Value": "true"
},
{
"Key": "created-by",
"Value": "eks-workshop-v2"
},
{
"Key": "eksctl.cluster.k8s.io/v1alpha1/cluster-name",
"Value": "eks-workshop"
},
{
"Key": "Name",
"Value": "eksctl-eks-workshop-cluster/VPC"
},
{
"Key": "karpenter.sh/discovery",
"Value": "eks-workshop"
},
{
"Key": "env",
"Value": "eks-workshop"
},
{
"Key": "aws:cloudformation:stack-name",
"Value": "eksctl-eks-workshop-cluster"
},
{
"Key": "aws:cloudformation:stack-id",
"Value": "arn:aws:cloudformation:ap-northeast-2:319485572629:stack/eksctl-eks-workshop-cluster/72939b30-7ec4-11ee-a993-06c248c7ee94"
}
]
}
]
}
ec2-user:~/environment $ aws ec2 describe-subnets --filters "Name=tag:created-by,Values=eks-workshop-v2" --query "Subnets[*].CidrBlock"
[
"10.42.0.0/19",
"100.64.64.0/19",
"10.42.128.0/19",
"10.42.32.0/19",
"10.42.64.0/19",
"10.42.160.0/19",
"100.64.0.0/19",
"10.42.96.0/19",
"100.64.32.0/19"
]
ec2-user:~/environment $ aws ec2 describe-vpcs --vpc-ids $VPC_ID | jq '.Vpcs[0].CidrBlockAssociationSet'
[
{
"AssociationId": "vpc-cidr-assoc-017bd2856e6a729a7",
"CidrBlock": "10.42.0.0/16",
"CidrBlockState": {
"State": "associated"
}
},
{
"AssociationId": "vpc-cidr-assoc-0144325ff4751e64e",
"CidrBlock": "100.64.0.0/16",
"CidrBlockState": {
"State": "associated"
}
}
]
ec2-user:~/environment $ kubectl set env daemonset aws-node -n kube-system AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true
daemonset.apps/aws-node env updated
ec2-user:~/environment $ kubectl apply -k ~/environment/eks-workshop/modules/networking/custom-networking/provision
configmap/eni-env-hk5dh89ktf created
eniconfig.crd.k8s.amazonaws.com/ap-northeast-2a created
eniconfig.crd.k8s.amazonaws.com/ap-northeast-2b created
eniconfig.crd.k8s.amazonaws.com/ap-northeast-2c created
ec2-user:~/environment $ kubectl get ENIConfigs
NAME AGE
ap-northeast-2a 7s
ap-northeast-2b 7s
ap-northeast-2c 7s
ec2-user:~/environment $ kubectl set env daemonset aws-node -n kube-system ENI_CONFIG_LABEL_DEF=topology.kubernetes.io/zone
daemonset.apps/aws-node env updated
ec2-user:~/environment $ kubectl set env daemonset aws-node -n kube-system ENI
2
기존 서브넷에 추가 ENI 를 새로 생성
aws eks create-nodegroup --region $AWS_REGION \
--cluster-name $EKS_CLUSTER_NAME \
--nodegroup-name custom-networking \
--instance-types t3.medium --node-role $CUSTOM_NETWORKING_NODE_ROLE \
--subnets $PRIMARY_SUBNET_1 $PRIMARY_SUBNET_2 $PRIMARY_SUBNET_3 \
--labels type=customnetworking \
--scaling-config minSize=1,maxSize=1,desiredSize=1
ec2-user:~/environment $ kubectl get nodes -L eks.amazonaws.com/nodegroup
NAME STATUS ROLES AGE VERSION NODEGROUP
ip-10-42-110-74.ap-northeast-2.compute.internal Ready <none> 47s v1.25.15-eks-4f4795d custom-networking
ip-10-42-134-167.ap-northeast-2.compute.internal Ready <none> 94m v1.25.6-eks-48e63af default
ip-10-42-160-180.ap-northeast-2.compute.internal Ready <none> 94m v1.25.6-eks-48e63af default
ip-10-42-99-228.ap-northeast-2.compute.internal Ready <none> 94m v1.25.6-eks-48e63af default
ip부족 해결
<8> 삭제
export EKS_CLUSTER_NAME=eks-workshop
delete-environment
(10분 이상 걸림)
eksctl delete cluster $EKS_CLUSTER_NAME --wait
(20분 걸림)
aws cloudformation delete-stack --stack-name eks-workshop-ide
다음은 오토 스케일러
https://brunch.co.kr/@topasvga/3521
감사합니다.
