DevOps

CKA 예제 리마인더 - 34. Application Failure

Vince_rf 2025. 1. 10. 09:49

예제는 해당 아키텍처를 기반으로 풀어야함

 



alpha 네임스페이스의 issue를 fix 하세요

Environment Variables: DB_Host=mysql-service; DB_Database=Not Set; DB_User=root; DB_Password=paswrd; 2003: Can't connect to MySQL server on 'mysql-service:3306' (-2 Name does not resolve)


에러 메시지를 보니 mysql-service:3306 으로 접속이 원활하지 않다

controlplane ~ ➜  kubectl config set-context --current --namespace=alpha
Context "default" modified.

일단 귀찮으니 네임스페이스를 옮기고 deploy를 확인해보자

 

controlplane ~ ➜  kubectl describe deploy webapp-mysql

Name:                   webapp-mysql
Namespace:              alpha
CreationTimestamp:      Thu, 09 Jan 2025 23:21:14 +0000
Labels:                 name=webapp-mysql
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               name=webapp-mysql
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  name=webapp-mysql
  Containers:
   webapp-mysql:
    Image:      mmumshad/simple-webapp-mysql
    Port:       8080/TCP
    Host Port:  0/TCP
    Environment:
      DB_Host:      mysql-service
      DB_User:      root
      DB_Password:  paswrd
    Mounts:         <none>
  Volumes:          <none>
  Node-Selectors:   <none>
  Tolerations:      <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   webapp-mysql-78fd9544f6 (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  103s  deployment-controller  Scaled up replica set webapp-mysql-78fd9544f6 to 1


그리고 svc도 확인해보자


controlplane ~ ➜  kubectl get svc
NAME          TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
mysql         ClusterIP   10.43.71.82    <none>        3306/TCP         6m48s
web-service   NodePort    10.43.108.62   <none>        8080:30081/TCP   6m48s


NAME이 mysql-service가 아닌 mysql로 되어있었다. mysql을 mysql-service로 수정해주자


controlplane ~ ➜  kubectl get svc mysql -o yaml > ./mysql.yaml

controlplane ~ ➜  vim mysql.yaml

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2025-01-09T22:50:03Z"
  name: mysql-service
  namespace: alpha
  resourceVersion: "796"
  uid: 5c8c874d-4995-462d-b42f-577cc7d6d782
spec:
  clusterIP: 10.43.71.82
  clusterIPs:
  - 10.43.71.82
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - port: 3306
    protocol: TCP
    targetPort: 3306
  selector:
    name: mysql
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}



controlplane ~ ➜  kubectl delete svc mysql
service "mysql" deleted

controlplane ~ ➜  kubectl apply -f mysql.yaml 
service/mysql-service created

controlplane ~ ➜  kubectl get svc
NAME            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
mysql-service   ClusterIP   10.43.71.82    <none>        3306/TCP         4s
web-service     NodePort    10.43.108.62   <none>        8080:30081/TCP   13m


이후 정상작동 확인




beta 네임스페이스의 issue를 fix 하세요

Environment Variables: DB_Host=mysql-service; DB_Database=Not Set; DB_User=root; DB_Password=paswrd; 2003: Can't connect to MySQL server on 'mysql-service:3306' (111 Connection refused)

controlplane ~ ✖ kubectl config set-context --current --namespace=beta
Context "default" modified.

역시 귀찮으니 네임스페이스를 옮겨주고

controlplane ~ ➜  kubectl get all
NAME                                READY   STATUS    RESTARTS   AGE
pod/mysql                           1/1     Running   0          8m33s
pod/webapp-mysql-78fd9544f6-7k28c   1/1     Running   0          8m33s

NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/mysql-service   ClusterIP   10.43.27.97     <none>        3306/TCP         8m33s
service/web-service     NodePort    10.43.183.178   <none>        8080:30081/TCP   8m33s

NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/webapp-mysql   1/1     1            1           8m33s

NAME                                      DESIRED   CURRENT   READY   AGE
replicaset.apps/webapp-mysql-78fd9544f6   1         1         1       8m33s


이번엔 svc 의 NAME도 문제 없음

다시 svc를 describe 해보자

controlplane ~ ➜  kubectl describe svc mysql-service 
Name:                     mysql-service
Namespace:                beta
Labels:                   <none>
Annotations:              <none>
Selector:                 name=mysql
Type:                     ClusterIP
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.43.27.97
IPs:                      10.43.27.97
Port:                     <unset>  3306/TCP
TargetPort:               8080/TCP
Endpoints:                10.42.0.11:8080
Session Affinity:         None
Internal Traffic Policy:  Cluster
Events:                   <none>

그리고 pod를 describe

controlplane ~ ➜  kubectl describe po mysql 
Name:             mysql
Namespace:        beta
Priority:         0
Service Account:  default
Node:             controlplane/192.168.243.180
Start Time:       Thu, 09 Jan 2025 23:25:44 +0000
Labels:           name=mysql
Annotations:      <none>
Status:           Running
IP:               10.42.0.11
IPs:
  IP:  10.42.0.11
Containers:
  mysql:
    Container ID:   containerd://87bab9bd662f192c6fdb025011aee3d750d40f88b48d84c152e30e05fb793665
    Image:          mysql:5.6
    Image ID:       docker.io/library/mysql@sha256:20575ecebe6216036d25dab5903808211f1e9ba63dc7825ac20cb975e34cfcae
    Port:           3306/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 09 Jan 2025 23:25:45 +0000
    Ready:          True
    Restart Count:  0
    Environment:
      MYSQL_ROOT_PASSWORD:  paswrd
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cqqmj (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  kube-api-access-cqqmj:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  11m   default-scheduler  Successfully assigned beta/mysql to controlplane
  Normal  Pulled     11m   kubelet            Container image "mysql:5.6" already present on machine
  Normal  Created    11m   kubelet            Created container mysql
  Normal  Started    11m   kubelet            Started container mysql


svc의 targetport가 잘못되었다

port는 서비스가 클러스터 외부와 통신할 때 사용하는 포트이며 targetport는 서비스가 트래픽을 전달할 때 실제로 Pod의 컨테이너가 수신 대기 중인 포트임

클라이언트 -> port (Service) -> targetPort (Pod 컨테이너)


controlplane ~ ➜  kubectl get svc mysql-service -o yaml > mysql-service-beta.yaml

controlplane ~ ➜  vim mysql-service-beta.yaml 

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2025-01-09T23:25:44Z"
  name: mysql-service
  namespace: beta
  resourceVersion: "1027"
  uid: ceb86f9b-3339-4838-817a-44e5cc00dbea
spec:
  clusterIP: 10.43.27.97
  clusterIPs:
  - 10.43.27.97
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - port: 3306
    protocol: TCP
    targetPort: 3306
  selector:
    name: mysql
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}





controlplane ~ ➜  kubectl delete svc mysql-service && kubectl apply -f mysql-service-beta.yaml 
service "mysql-service" deleted
service/mysql-service created

controlplane ~ ➜  kubectl describe svc mysql-service 
Name:                     mysql-service
Namespace:                beta
Labels:                   <none>
Annotations:              <none>
Selector:                 name=mysql
Type:                     ClusterIP
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.43.27.97
IPs:                      10.43.27.97
Port:                     <unset>  3306/TCP
TargetPort:               3306/TCP
Endpoints:                10.42.0.11:3306
Session Affinity:         None
Internal Traffic Policy:  Cluster
Events:                   <none>


이후 정상작동 확인




gamma 네임스페이스의 issue를 fix 하세요

Environment Variables: DB_Host=mysql-service; DB_Database=Not Set; DB_User=root; DB_Password=paswrd; 2003: Can't connect to MySQL server on 'mysql-service:3306' (111 Connection refused)

controlplane ~ ➜  kubectl config set-context --current --namespace=gamma
Context "default" modified.

controlplane ~ ➜  kubectl get all
NAME                                READY   STATUS    RESTARTS   AGE
pod/mysql                           1/1     Running   0          78s
pod/webapp-mysql-78fd9544f6-h6cvq   1/1     Running   0          78s

NAME                    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
service/mysql-service   ClusterIP   10.43.18.21    <none>        3306/TCP         78s
service/web-service     NodePort    10.43.170.22   <none>        8080:30081/TCP   78s

NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/webapp-mysql   1/1     1            1           78s

NAME                                      DESIRED   CURRENT   READY   AGE
replicaset.apps/webapp-mysql-78fd9544f6   1         1         1       78s


name에 이상없고 pod 상태 확인하고

controlplane ~ ➜  kubectl describe po mysql 
Name:             mysql
Namespace:        gamma
Priority:         0
Service Account:  default
Node:             controlplane/192.168.243.180
Start Time:       Thu, 09 Jan 2025 23:54:18 +0000
Labels:           name=mysql
Annotations:      <none>
Status:           Running
IP:               10.42.0.13
IPs:
  IP:  10.42.0.13
Containers:
  mysql:
    Container ID:   containerd://1069ab73c27f5dd91a9ac8d980480e20fac6a0fd7df565afce06262f259e5b4a
    Image:          mysql:5.6
    Image ID:       docker.io/library/mysql@sha256:20575ecebe6216036d25dab5903808211f1e9ba63dc7825ac20cb975e34cfcae
    Port:           3306/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 09 Jan 2025 23:54:19 +0000
    Ready:          True
    Restart Count:  0
    Environment:
      MYSQL_ROOT_PASSWORD:  paswrd
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-hhrbc (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  kube-api-access-hhrbc:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  118s  default-scheduler  Successfully assigned gamma/mysql to controlplane
  Normal  Pulled     117s  kubelet            Container image "mysql:5.6" already present on machine
  Normal  Created    117s  kubelet            Created container mysql
  Normal  Started    117s  kubelet            Started container mysql


svc도 wide로 확인해보자


controlplane ~ ➜  kubectl get svc mysql-service -o wide
NAME            TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE     SELECTOR
mysql-service   ClusterIP   10.43.18.21   <none>        3306/TCP   2m22s   name=sql00001

pod의 name은 mysql 이지만 svc의 selector가 이상하다


controlplane ~ ➜  kubectl get svc mysql-service -o yaml > ./mysql-service-gamma.yaml

controlplane ~ ➜  vim mysql-service-gamma.yaml



apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2025-01-09T23:54:18Z"
  name: mysql-service
  namespace: gamma
  resourceVersion: "1634"
  uid: 54aa51f4-4a8e-45f4-9ea5-a37487696123
spec:
  clusterIP: 10.43.18.21
  clusterIPs:
  - 10.43.18.21
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - port: 3306
    protocol: TCP
    targetPort: 3306
  selector:
    name: mysql
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}





controlplane ~ ➜  kubectl delete svc mysql-service && kubectl apply -f mysql-service-gamma.yaml 
service "mysql-service" deleted
service/mysql-service created


controlplane ~ ✖ kubectl get svc -o wide
NAME            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE     SELECTOR
mysql-service   ClusterIP   10.43.18.21    <none>        3306/TCP         41s     name=mysql
web-service     NodePort    10.43.170.22   <none>        8080:30081/TCP   5m15s   name=webapp-mysql

이후 정상작동 확인



delta 네임스페이스의 issue를 fix 하세요

Environment Variables: DB_Host=mysql-service; DB_Database=Not Set; DB_User=sql-user; DB_Password=paswrd; 1045 (28000): Access denied for user 'sql-user'@'10.42.0.16' (using password: YES)

에러메세지부터 DB_USER가 수상하다

controlplane ~ ➜  kubectl config set-context --current --namespace=delta
Context "default" modified.


controlplane ~ ➜  kubectl describe deploy webapp-mysql 
Name:                   webapp-mysql
Namespace:              delta
CreationTimestamp:      Thu, 09 Jan 2025 23:59:57 +0000
Labels:                 name=webapp-mysql
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               name=webapp-mysql
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  name=webapp-mysql
  Containers:
   webapp-mysql:
    Image:      mmumshad/simple-webapp-mysql
    Port:       8080/TCP
    Host Port:  0/TCP
    Environment:
      DB_Host:      mysql-service
      DB_User:      sql-user
      DB_Password:  paswrd
    Mounts:         <none>
  Volumes:          <none>
  Node-Selectors:   <none>
  Tolerations:      <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   webapp-mysql-b9c9f7fbd (1/1 replicas created)
Events:
  Type    Reason             Age    From                   Message
  ----    ------             ----   ----                   -------
  Normal  ScalingReplicaSet  2m45s  deployment-controller  Scaled up replica set webapp-mysql-b9c9f7fbd to 1



문제에는 DB_USER를 root로 설정하라고 되어있으니 DB_USER 부분을 deploy의 manifest에서 수정해줘야한다

controlplane ~ ➜  kubectl get deploy webapp-mysql -o yaml > ./mysql-service-delta.yaml

controlplane ~ ➜  vim mysql-service-delta.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2025-01-09T23:59:57Z"
  generation: 1
  labels:
    name: webapp-mysql
  name: webapp-mysql
  namespace: delta
  resourceVersion: "1862"
  uid: b58d6372-47e2-4fcb-9b9b-8ff8419f670a
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      name: webapp-mysql
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        name: webapp-mysql
      name: webapp-mysql
    spec:
      containers:
      - env:
        - name: DB_Host
          value: mysql-service
        - name: DB_User
          value: root
        - name: DB_Password
          value: paswrd




controlplane ~ ➜  kubectl apply -f mysql-service-delta.yaml --force
Warning: resource deployments/webapp-mysql is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/webapp-mysql configured

controlplane ~ ➜  kubectl get deploy
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
webapp-mysql   1/1     1            1           4m23s

controlplane ~ ➜  kubectl describe deploy webapp-mysql 
Name:                   webapp-mysql
Namespace:              delta
CreationTimestamp:      Thu, 09 Jan 2025 23:59:57 +0000
Labels:                 name=webapp-mysql
Annotations:            deployment.kubernetes.io/revision: 2
Selector:               name=webapp-mysql
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  name=webapp-mysql
  Containers:
   webapp-mysql:
    Image:      mmumshad/simple-webapp-mysql
    Port:       8080/TCP
    Host Port:  0/TCP
    Environment:
      DB_Host:      mysql-service
      DB_User:      root
      DB_Password:  paswrd
    Mounts:         <none>
  Volumes:          <none>
  Node-Selectors:   <none>
  Tolerations:      <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  webapp-mysql-b9c9f7fbd (0/0 replicas created)
NewReplicaSet:   webapp-mysql-78fd9544f6 (1/1 replicas created)
Events:
  Type    Reason             Age    From                   Message
  ----    ------             ----   ----                   -------
  Normal  ScalingReplicaSet  4m29s  deployment-controller  Scaled up replica set webapp-mysql-b9c9f7fbd to 1
  Normal  ScalingReplicaSet  17s    deployment-controller  Scaled up replica set webapp-mysql-78fd9544f6 to 1
  Normal  ScalingReplicaSet  16s    deployment-controller  Scaled down replica set webapp-mysql-b9c9f7fbd to 0 from 1


이후 정상작동 확인







epsilon 네임스페이스의 issue를 fix 하세요

Environment Variables: DB_Host=mysql-service; DB_Database=Not Set; DB_User=sql-user; DB_Password=paswrd; 1045 (28000): Access denied for user 'sql-user'@'10.42.0.19' (using password: YES)

DB_USER부분이 또 잘못되어 있음

controlplane ~ ➜  kubectl config set-context --current --namespace=epsilon
Context "default" modified.


controlplane ~ ➜  kubectl get deploy webapp-mysql -o yaml > ./mysql-service-epsilon.yaml

controlplane ~ ➜  vim mysql-service-epsilon.yaml 


apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2025-01-10T00:05:54Z"
  generation: 1
  labels:
    name: webapp-mysql
  name: webapp-mysql
  namespace: epsilon
  resourceVersion: "2099"
  uid: 6aacd045-4f40-41b8-bd3c-8e26800d38b7
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      name: webapp-mysql
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        name: webapp-mysql
      name: webapp-mysql
    spec:
      containers:
      - env:
        - name: DB_Host
          value: mysql-service
        - name: DB_User
          value: root
        - name: DB_Password
          value: paswrd




controlplane ~ ➜  kubectl apply -f mysql-service-epsilon.yaml
Warning: resource deployments/webapp-mysql is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/webapp-mysql configured

그래도 해결이 되지 않는데

controlplane ~ ➜  kubectl describe po mysql 
Name:             mysql
Namespace:        epsilon
Priority:         0
Service Account:  default
Node:             controlplane/192.168.129.236
Start Time:       Fri, 10 Jan 2025 00:26:28 +0000
Labels:           name=mysql
Annotations:      <none>
Status:           Running
IP:               10.42.0.18
IPs:
  IP:  10.42.0.18
Containers:
  mysql:
    Container ID:   containerd://15088f982f0dc12f0be8e0bc8da403e4f873035e122edcb498827078679ad2cf
    Image:          mysql:5.6
    Image ID:       docker.io/library/mysql@sha256:20575ecebe6216036d25dab5903808211f1e9ba63dc7825ac20cb975e34cfcae
    Port:           3306/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 10 Jan 2025 00:26:29 +0000
    Ready:          True
    Restart Count:  0
    Environment:
      MYSQL_ROOT_PASSWORD:  passwooooorrddd
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-k6tkm (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  kube-api-access-k6tkm:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  2m8s  default-scheduler  Successfully assigned epsilon/mysql to controlplane
  Normal  Pulled     2m8s  kubelet            Container image "mysql:5.6" already present on machine
  Normal  Created    2m8s  kubelet            Created container mysql
  Normal  Started    2m8s  kubelet            Started container mysql


MYSQL 패스워드도 이상함



controlplane ~ ➜  kubectl get po mysql -o yaml > mysql-epsilon.yaml

controlplane ~ ➜  vim mysql-epsilon.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2025-01-10T00:26:28Z"
  labels:
    name: mysql
  name: mysql
  namespace: epsilon
  resourceVersion: "1427"
  uid: cbcdb8c9-9d68-4eef-88b6-94e235f51663
spec:
  containers:
  - env:
    - name: MYSQL_ROOT_PASSWORD
      value: paswrd




controlplane ~ ➜  kubectl apply -f mysql-epsilon.yaml --force
Warning: resource pods/mysql is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
pod/mysql configured

controlplane ~ ➜  kubectl describe po mysql
Name:             mysql
Namespace:        epsilon
Priority:         0
Service Account:  default
Node:             controlplane/192.168.129.236
Start Time:       Fri, 10 Jan 2025 00:31:47 +0000
Labels:           name=mysql
Annotations:      <none>
Status:           Running
IP:               10.42.0.21
IPs:
  IP:  10.42.0.21
Containers:
  mysql:
    Container ID:   containerd://b495d26f0b723ec08c9459c2bd2aacdaf04e243a841b095219badfa61a975718
    Image:          mysql:5.6
    Image ID:       docker.io/library/mysql@sha256:20575ecebe6216036d25dab5903808211f1e9ba63dc7825ac20cb975e34cfcae
    Port:           3306/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 10 Jan 2025 00:31:47 +0000
    Ready:          True
    Restart Count:  0
    Environment:
      MYSQL_ROOT_PASSWORD:  paswrd
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-k6tkm (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  kube-api-access-k6tkm:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason   Age   From     Message
  ----    ------   ----  ----     -------
  Normal  Pulled   22s   kubelet  Container image "mysql:5.6" already present on machine
  Normal  Created  22s   kubelet  Created container mysql
  Normal  Started  22s   kubelet  Started container mysql



이후 정상작동 확인





zeta 네임스페이스의 issue를 fix 하세요

502 bad gate

controlplane ~ ➜  kubectl config set-context --current --namespace=zeta
Context "default" modified.

controlplane ~ ➜  kubectl get all
NAME                               READY   STATUS    RESTARTS   AGE
pod/mysql                          1/1     Running   0          54s
pod/webapp-mysql-b9c9f7fbd-dmcbw   1/1     Running   0          54s

NAME                    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
service/mysql-service   ClusterIP   10.43.194.22   <none>        3306/TCP         54s
service/web-service     NodePort    10.43.225.20   <none>        8080:30088/TCP   54s

NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/webapp-mysql   1/1     1            1           54s

NAME                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/webapp-mysql-b9c9f7fbd   1         1         1       54s


502 bad gate 라면 일단 web-app 자체에 svc가 잘못 세팅되어 있다고 의심해볼만하다

controlplane ~ ➜  kubectl get svc -o wide
NAME            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE    SELECTOR
mysql-service   ClusterIP   10.43.194.22   <none>        3306/TCP         6m3s   name=mysql
web-service     NodePort    10.43.225.20   <none>        8080:30088/TCP   6m3s   name=webapp-mysql

port가 잘못되어 있음

controlplane ~ ➜  kubectl get svc web-service -o yaml > web-zeta.yaml

controlplane ~ ➜  vim web-zeta.yaml

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2025-01-10T00:32:48Z"
  name: web-service
  namespace: zeta
  resourceVersion: "1680"
  uid: 1be0bd9f-84fe-4908-9755-b9d1320d0876
spec:
  clusterIP: 10.43.225.20
  clusterIPs:
  - 10.43.225.20
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - nodePort: 30081
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    name: webapp-mysql
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}



controlplane ~ ➜  kubectl delete svc web-service && kubectl apply -f web-zeta.yaml 
service "web-service" deleted
service/web-service created


아직도 해결되지 않기 때문에

pod 상태 확인

controlplane ~ ➜  kubectl describe po mysql
Name:             mysql
Namespace:        zeta
Priority:         0
Service Account:  default
Node:             controlplane/192.168.129.236
Start Time:       Fri, 10 Jan 2025 00:32:48 +0000
Labels:           name=mysql
Annotations:      <none>
Status:           Running
IP:               10.42.0.22
IPs:
  IP:  10.42.0.22
Containers:
  mysql:
    Container ID:   containerd://0fd95aa0c0cf4b6399005cb14f37b7e69b525706213b09b9c826a677220f8918
    Image:          mysql:5.6
    Image ID:       docker.io/library/mysql@sha256:20575ecebe6216036d25dab5903808211f1e9ba63dc7825ac20cb975e34cfcae
    Port:           3306/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 10 Jan 2025 00:32:48 +0000
    Ready:          True
    Restart Count:  0
    Environment:
      MYSQL_ROOT_PASSWORD:  passwooooorrddd
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8z4xq (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  kube-api-access-8z4xq:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  118s  default-scheduler  Successfully assigned zeta/mysql to controlplane
  Normal  Pulled     118s  kubelet            Container image "mysql:5.6" already present on machine
  Normal  Created    118s  kubelet            Created container mysql
  Normal  Started    118s  kubelet            Started container mysql

패스워드 이상함


controlplane ~ ➜  kubectl get po mysql -o yaml > mysql-zeta.yaml

controlplane ~ ➜  vim mysql-zeta.yaml 


apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2025-01-10T00:32:48Z"
  labels:
    name: mysql
  name: mysql
  namespace: zeta
  resourceVersion: "1690"
  uid: b5c2ca99-e1f4-4fe2-acb6-061ee169760f
spec:
  containers:
  - env:
    - name: MYSQL_ROOT_PASSWORD
      value: paswrd



controlplane ~ ➜  kubectl apply -f mysql-zeta.yaml --force
Warning: resource pods/mysql is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
pod/mysql configured


deploy  확인

controlplane ~ ➜  kubectl describe deploy webapp-mysql 
Name:                   webapp-mysql
Namespace:              zeta
CreationTimestamp:      Fri, 10 Jan 2025 00:32:48 +0000
Labels:                 name=webapp-mysql
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               name=webapp-mysql
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  name=webapp-mysql
  Containers:
   webapp-mysql:
    Image:      mmumshad/simple-webapp-mysql
    Port:       8080/TCP
    Host Port:  0/TCP
    Environment:
      DB_Host:      mysql-service
      DB_User:      sql-user
      DB_Password:  paswrd
    Mounts:         <none>
  Volumes:          <none>
  Node-Selectors:   <none>
  Tolerations:      <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   webapp-mysql-b9c9f7fbd (1/1 replicas created)
Events:
  Type    Reason             Age    From                   Message
  ----    ------             ----   ----                   -------
  Normal  ScalingReplicaSet  3m58s  deployment-controller  Scaled up replica set webapp-mysql-b9c9f7fbd to 1

유저가 이상함

controlplane ~ ➜  kubectl get deploy webapp-mysql -o yaml > mysql-deploy-zeta.yaml

controlplane ~ ➜  vim mysql-deploy-zeta.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2025-01-10T00:32:48Z"
  generation: 1
  labels:
    name: webapp-mysql
  name: webapp-mysql
  namespace: zeta
  resourceVersion: "1697"
  uid: c0dfd073-81f2-4b03-940c-649cf62307a6
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      name: webapp-mysql
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        name: webapp-mysql
      name: webapp-mysql
    spec:
      containers:
      - env:
        - name: DB_Host
          value: mysql-service
        - name: DB_User
          value: root
        - name: DB_Password
          value: paswrd



controlplane ~ ➜  kubectl apply -f mysql-deploy-zeta.yaml --force
Warning: resource deployments/webapp-mysql is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/webapp-mysql configured


이후 정상작동 확인