[Kubernetes] 14. Kubernetes 시험 - EKS 활용 & Route53 & 3 Tier

문제

 

1. EKS 프라이빗 클러스터를 구축하세요

2-1. nginx/톰캣/MySQL로 WAS-DB 2 tier나 WEB-WAS-DB 3 tier를 구성, 브라우저에서 www.<본인도메인>/exam 으로 접속했을때 DB연동을 확인하는 페이지가 뜨도록 하세요.

2-2. www.<본인도메인>/h 로 접속했을때 hostname을 출력하는 페이지가 뜨도록하세요.
2-3. www.<본인도메인>/ip 로 접속했을때 ip를 출력하는 페이지가 뜨도록하세요.

RDS의 포트는 33306으로 하세요.
EKS클러스터가 접근하는 컨테이너 레지스트리는 무조건 ECR을 사용하세요.

단, 베이스이미지는 61.254.18.30:5000/ipnginx, 61.254.18.30:5000/hnginx 나 oolralra/ipnginx, oolralra/hnginx를 사용하세요. 톰캣은 아무거나 쓰셔도 됩니다

 

 

풀이

1. RDS 생성

• 비밀번호 : test123!

 

2. NAT Gateway 생성

 

• 라우팅 테이블 - NAT Gateway 연결
  • DB subnet / Private Subnet

 

3. RDS의 보안그룹 인바운드 규칙 수정

 

4. 클러스터 생성

# 환경변수 등록
export PRI_SUBNET1_ID=subnet-0b0e6e20720b77039
export PRI_SUBNET2_ID=subnet-09a7c7c69e57a1345

# 클러스터 생성
eksctl create cluster --vpc-private-subnets $PRI_SUBNET1_ID,$PRI_SUBNET2_ID --name rapa-cluster --region ap-northeast-2 --version 1.32 --nodegroup-name rapacng --node-type t3.small --nodes 1 --nodes-min 1 --nodes-max 3 --node-private-networking

 

5. LB Controller 설치

# 계정 정보 확인
aws sts get-caller-identity

# OIDC 활성화
eksctl utils associate-iam-oidc-provider --cluster $CLUSTER_NAME --approve

# IAM Role - IRSA 생성
eksctl create iamserviceaccount   --cluster=$CLUSTER_NAME   --namespace=kube-system   --name=aws-load-balancer-controller   --role-name AmazonEKSLoadBalancerControllerRole   --attach-policy-arn=arn:aws:iam::$ACCOUNT_ID:policy/AWSLoadBalancerControllerIAMPolicy   --override-existing-serviceaccounts   --approve

# IRSA 생성 확인
kubectl get sa -n kube-system | grep -i load

# LB Controller 설치
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=$CLUSTER_NAME \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller \
--set image.repository=602401143452.dkr.ecr.ap-northeast-2.amazonaws.com/amazon/aws-load-balancer-controller \
--set region=ap-northeast-2 \
--set vpcId=$VPC_ID

 

➕ 사설 레지스트리 보안 허용 

root@aws-cli:~/mani/exam/web# vi /etc/docker/daemon.json

{
  "insecure-registries": ["61.254.18.30:5000"]
}

 

6. WAS

6-1. ECR 레포지토리 생성

aws ecr create-repository --repository-name was-tomcat-alb --region ap-northeast-2

 

6-2. Docker 로그인

aws ecr get-login-password --region ap-northeast-2 | docker login --username AWS --password-stdin 798172178824.dkr.ecr.ap-northeast-2.amazonaws.com

 

6-3. Dockerfile 생성

root@aws-cli:~/mani/exam/was# vi Dockerfile 

FROM tomcat:latest
RUN apt-get update && apt-get install -y wget \
    && wget -O /usr/local/tomcat/lib/mysql-connector-java-8.0.23.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.23/mysql-connector-java-8.0.23.jar \
    && apt-get clean
WORKDIR /usr/local/tomcat/webapps/ROOT
COPY index.jsp .
EXPOSE 8080
CMD ["catalina.sh", "run"]
~

 

6-4. index.jsp 생성

root@aws-cli:~/mani/exam/was# vi index.jsp 

<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="java.sql.*"%>
<h1>DB</h2>
<%
        Connection conn=null;
        try {
                String Url="jdbc:mysql://database-1.cjeksu6gkjsq.ap-northeast-2.rds.amazonaws.com:33306/mydb";
                String Id="admin";
                String Pass="test123!";

                Class.forName("com.mysql.cj.jdbc.Driver");
                conn=DriverManager.getConnection(Url,Id,Pass);
                out.println("was-db Connection Success!");
        } catch (Exception e) {
                e.printStackTrace();
}
%>

 

6-5. Docker Image 생성

docker build -t was-tomcat-alb:3 .
docker tag was-tomcat-alb:3 $ECR/was-tomcat-alb:2
docker push $ECR/was-tomcat-alb:2

 

6-6. Dockerfile  생성

root@aws-cli:~/mani/exam/was# vi Dockerfile 

FROM tomcat:latest
RUN apt-get update && apt-get install -y wget \
    && wget -O /usr/local/tomcat/lib/mysql-connector-java-8.0.23.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.23/mysql-connector-java-8.0.23.jar \
    && apt-get clean
WORKDIR /usr/local/tomcat/webapps/ROOT
COPY index.jsp .
EXPOSE 8080
CMD ["catalina.sh", "run"]

 

6-7. WAS - Deployment & SVC 생성

kubectl create deploy was --image=798172178824.dkr.ecr.ap-northeast-2.amazonaws.com/was-tomcat-alb:2 --replicas=1
kubectl expose deploy was --target-port 8080 --port 8080

 

6-8. WAS - Ingress 정의 & 생성

vi was-ingress.yml 

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: "was-ingress"
  labels:
    app.kubernetes.io/name: "was-ingress"
  annotations:
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/target-type: ip

spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
          - pathType: Prefix
            path: /
            backend:
              service:
                name: was
                port:
                  number: 8080

kubectl apply -f was-ingress.yml

 

7. WEB

7-1. ECR 레포지토리 생성

aws ecr create-repository --repository-name web-apache-alb --region ap-northeast-2

 

6-2. Dockerfile 생성

root@aws-cli:~/mani/exam/web# vi Dockerfile 

FROM httpd:2.4
COPY httpd.conf /usr/local/apache2/conf/httpd.conf
COPY html/ /usr/local/apache2/htdocs/

 

6-3. html/index.html

root@aws-cli:~/mani/exam/web# vi html/index.html 

nginx test

 

6-4. httpd.conf - 리버스프록시

root@aws-cli:~/mani/exam/web# vi httpd.conf

ServerRoot "/usr/local/apache2"
Listen 80

# 필수 모듈 로드
LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule dir_module modules/mod_dir.so
LoadModule mime_module modules/mod_mime.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so

User daemon
Group daemon
ServerAdmin you@example.com
ServerName localhost

# 문서 루트 설정
DocumentRoot "/usr/local/apache2/htdocs"
<Directory "/usr/local/apache2/htdocs">
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>

<Directory />
    AllowOverride none
    Require all denied
</Directory>

# 로그 설정
ErrorLog /proc/self/fd/2
CustomLog /proc/self/fd/1 common

# Reverse Proxy 설정
<VirtualHost *:80>
    ProxyRequests Off
    ProxyPreserveHost On

    <Proxy *>
        Require all granted
    </Proxy>

    # /tom으로 들어오는 요청을 내부 WAS ALB로 전달
    ProxyPass /exam http://internal-k8s-default-wasingre-809bd0f211-854975371.ap-northeast-2.elb.amazonaws.com/
    ProxyPassReverse /exam http://internal-k8s-default-wasingre-809bd0f211-854975371.ap-northeast-2.elb.amazonaws.com/

# 기본 index.html 설정
DirectoryIndex index.html
</VirtualHost>

 

 

6-5. Docker Image 생성

docker build -t web-apache-alb:4 .
docker tag web-apache-alb:4 $ECR/web-apache-alb:4
docker push $ECR/web-apache-alb:4

 

6-6. WEB - Deployment & SVC 생성

kubectl create deploy web --image=798172178824.dkr.ecr.ap-northeast-2.amazonaws.com/web-apache-alb:4 --replicas=1
kubectl expose deploy web --target-port 80 --port 80

 

6-7. WEB - Ingress 정의

root@aws-cli:~/mani/exam/web# vi web-ingress.yml 

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: "web-ingress"
  labels:
    app.kubernetes.io/name: "web-ingress"
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  ingressClassName: alb
  rules:
  	# ✅DNS로 연결해주기 위해 명시
    - host: www.bboaws.shop
      http:
        paths:
          - pathType: Prefix
            path: /
            backend:
              service:
                name: web
                port:
                  number: 80
          - pathType: Prefix
            path: /h
            backend:
              service:
                name: hnginx
                port:
                  number: 80
          - pathType: Prefix
            path: /ip
            backend:
              service:
                name: ipnginx
                port:
                  number: 80

 

7. ipnginx

7-1. Dockerfile

vi Dockerfile 

FROM 61.254.18.30:5000/ipnginx

 

7-2. ECR 레포지토리 생성

aws ecr create-repository --repository-name ipnginx --region ap-northeast-2

 

7-3. Docker Image 생성

docker build -t ipnginx:1 .
docker tag ipnginx:1 $ECR/ipnginx:1
docker push $ECR/ipnginx:1

 

7-4. ipnginx - Deployment & SVC 생성

kubectl create deploy ipnginx --image=798172178824.dkr.ecr.ap-northeast-2.amazonaws.com/ipnginx:1 --replicas=1
kubectl expose deploy ipnginx --target-port 80 --port 80

 

8. hnginx

8-1. Dockerfile

vi Dockerfile

FROM 61.254.18.30:5000/hnginx

 

 

8-2. ECR 레포지토리 생성

aws ecr create-repository --repository-name hnginx --region ap-northeast-2

 

8-3. Docker Image 생성

docker build -t hnginx:1 .
docker tag hnginx:1 $ECR/hnginx:1
docker push $ECR/hnginx:1

 

8-4. hnginx - Deployment & SVC 생성

kubectl create deploy hnginx --image=798172178824.dkr.ecr.ap-northeast-2.amazonaws.com/hnginx:1 --replicas=1
kubectl expose deploy hnginx --target-port 80 --port 80

 

💡노드 용량 때문에, 리소스가 안 뜰 때, nodegroup 설정 변경

aws eks update-nodegroup-config \
  --cluster-name $CLUSTER_NAME \
  --nodegroup-name rapacng \
  --scaling-config minSize=1,maxSize=5,desiredSize=3

 

9. web-ingress 생성

kubectl apply -f web-ingress.yml

 

10. Route53 설정

• RNS 설정은 시간이 10분 이상 걸림

💡 자세한 내용은 아래 참고- https://etoile-recording.tistory.com/66

[AWS] 05-1. Router53 & S3 & TLS(HTTPS) & Cloudfront

 

11. 결과 확인