Issue
I have a Flask app that should run on Apache2 server. I dockerized the app and it is working as expected on docker host in my test VM.
Once deployed on AKS the POD is stuck in state CrashLoopBackOff. I am using a gitlab agent to deploy into AKS with gitlab-ci, the image is stored in gitlab private registry and the image was successifully pulled into AKS.
Here are the files/logs of my apps/AKS pod
Dockerfile
FROM debian:bullseye-slim
RUN apt-get update && apt-get install -y apache2 \
libapache2-mod-wsgi-py3 \
python3 \
python3-pip \
&& apt-get clean \
&& apt-get autoremove \
&& python3 -m pip install flask \
&& rm -rf /var/lib/apt/lists/*
COPY ./requirements.txt /var/www/webApp/requirements.txt
RUN pip install -r /var/www/webApp/requirements.txt
COPY ./webApp.conf /etc/apache2/sites-available/webApp.conf
COPY ./ /var/www/webApp/
RUN a2dissite 000-default.conf
RUN a2ensite webApp.conf
RUN ln -sf /proc/self/fd/1 /var/log/apache2/access.log && \
ln -sf /proc/self/fd/1 /var/log/apache2/error.log
EXPOSE 8080
WORKDIR /var/www/webApp
CMD /usr/sbin/apache2ctl -D FOREGROUND
Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: flask-app
name: flask-app
spec:
replicas: 1
selector:
matchLabels:
app: flask-app
template:
metadata:
labels:
app: flask-app
spec:
containers:
- image: registry.gitlab.com/sast6353642/sast-01-pull/flask-appache:0a654662
name: flask-appache
ports:
- containerPort: 8080
#securityContext:
#privileged: false
imagePullPolicy: Always
#resources:
#limits:
#memory: 100Mi
#cpu: 300m
imagePullSecrets:
- name: gitlab
restartPolicy: Always
kubectl get pods
NAME READY STATUS RESTARTS AGE
flask-app-6d74447889-bghtx 0/1 CrashLoopBackOff 6 (2m27s ago) 8m18s
kubectl describe pods flask-app-6d74447889-bghtx
Name: flask-app-6d74447889-bghtx
Namespace: default
Priority: 0
Service Account: default
Node: aks-agentpool-16891536-vmss000000/10.224.0.5
Start Time: Tue, 02 Jan 2024 20:20:49 +0000
Labels: app=flask-app
pod-template-hash=6d74447889
Annotations: cni.projectcalico.org/containerID: 966c590b4ce639fa829feba8e597d63395af53ea231ab005f8d547e1721f8233
cni.projectcalico.org/podIP: 10.244.1.17/32
cni.projectcalico.org/podIPs: 10.244.1.17/32
Status: Running
IP: 10.244.1.17
IPs:
IP: 10.244.1.17
Controlled By: ReplicaSet/flask-app-6d74447889
Containers:
flask-appache:
Container ID: containerd://5cdc5e1294267fce47bda95cb81bd09654a78fcf21dfe4bbaece786d195ea778
Image: registry.gitlab.com/sast6353642/sast-01-pull/flask-appache:0a654662
Image ID: registry.gitlab.com/sast6353642/sast-01-pull/flask-appache@sha256:e798e4036dc13d72ef36c866d678fe295e3eac4b0058cd076330654bc56af317
Port: 8080/TCP
Host Port: 0/TCP
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Tue, 02 Jan 2024 20:22:20 +0000
Finished: Tue, 02 Jan 2024 20:22:20 +0000
Ready: False
Restart Count: 4
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-sj7s8 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-api-access-sj7s8:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m57s default-scheduler Successfully assigned default/flask-app-6d74447889-bghtx to aks-agentpool-16891536-vmss000000
Normal Pulled 2m57s kubelet Successfully pulled image "registry.gitlab.com/sast6353642/sast-01-pull/flask-appache:0a654662" in 864.830288ms (864.838488ms including waiting)
Normal Pulled 2m55s kubelet Successfully pulled image "registry.gitlab.com/sast6353642/sast-01-pull/flask-appache:0a654662" in 827.635558ms (827.908837ms including waiting)
Normal Pulled 2m38s kubelet Successfully pulled image "registry.gitlab.com/sast6353642/sast-01-pull/flask-appache:0a654662" in 875.237209ms (875.244689ms including waiting)
Normal Created 2m10s (x4 over 2m57s) kubelet Created container flask-appache
Normal Started 2m10s (x4 over 2m56s) kubelet Started container flask-appache
Normal Pulled 2m10s kubelet Successfully pulled image "registry.gitlab.com/sast6353642/sast-01-pull/flask-appache:0a654662" in 827.338313ms (827.349353ms including waiting)
Warning BackOff 100s (x7 over 2m55s) kubelet Back-off restarting failed container flask-appache in pod flask-app-6d74447889-bghtx_default(1177ffe3-61ff-418f-a37d-3b6d857c603b)
Normal Pulling 88s (x5 over 2m57s) kubelet Pulling image "registry.gitlab.com/sast6353642/sast-01-pull/flask-appache:0a654662"
Normal Pulled 87s kubelet Successfully pulled image "registry.gitlab.com/sast6353642/sast-01-pull/flask-appache:0a654662" in 862.977469ms (862.990149ms including waiting)
kubectl logs flask-app-6d74447889-bghtx -c containerd://5cdc5e1294267fce47bda95cb81bd09654a78fcf21dfe4bbaece786d195ea778
error: container containerd://c4c557b4e6f92ac803c7e9c42624e59e39941e3d731015e12c4bea193dd5db3b is not valid for pod flask-app-6d74447889-bghtx
Logs in diagnostics settings in Azure
exec /usr/local/bin/python: exec format error
**app.py **
from website import create_app
app = create_app()
if __name__ == "__main__":
app.run(debug=True)
Can anyone help with that please ?
I tried making changes into the Dockerfile (changing the port exposed from 80 to 8080) but nothing happened. I searched from the log container not valid for pod and did not find anything related to that.
EDIT
My .gitlab-ci.yml
stages:
- build
- deploy
variables:
DOCKER_SOURCE_IMAGE: flask-appache:$CI_COMMIT_SHORT_SHA
DOCKER_TARGET_IMAGE: $CI_REGISTRY_TAG/$DOCKER_SOURCE_IMAGE
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
CI_REGISTRY_TAG: registry.gitlab.com/sast6353642/sast-01-pull
CI_REGISTRY: registry.gitlab.com
pull-push:
stage: build
image: docker:latest
services:
- docker:dind
before_script:
- docker login ${CI_REGISTRY} -u ${USERNAME_REGISTRY} -p ${TOKEN_REGISTRY}
script:
- docker build . -t ${DOCKER_SOURCE_IMAGE}
- docker tag ${DOCKER_SOURCE_IMAGE} ${DOCKER_TARGET_IMAGE}
- docker push ${DOCKER_TARGET_IMAGE}
- docker logout ${CI_REGISTRY}
deploy:
stage: deploy
image:
name: bitnami/kubectl:latest
entrypoint: ['']
script:
- kubectl config use-context k8s-flask/k8s-connection:k8s-agent-connection
- kubectl get pods
- kubectl apply -f .
Solution
I resolved the issue. The cpu architecture of node pools (ubuntu) in AKS is ARM, I had to specify FROM --platform=linux/arm in my Dockerfile so the base image would be based on ARM instead on AMD64 so it can work in an ARM based host.
Exemple: FROM --platform=linux/arm python:3.10-alpine
If the image is built in an Amd64 based host with the --platform=linux/arm prefix for base image in the Dockerfile this will not work => Qemu has to be installed so it can provide other system architecture platforms.
Before installing Qemu
debian@debian11:~/simple-flask$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS default * docker default default running v0.11.7-0.20230525183624-798ad6b0ce9f linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
Installing Qemu on Debian/Ubuntu
sudo apt-get install -y qemu qemu-user-static
You can see ARM now in platforms
debian@debian11:~/simple-flask$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
default * docker
default default running v0.11.7-0.20230525183624-798ad6b0ce9f linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386, linux/arm64, linux/riscv64, linux/ppc64, linux/ppc64le, linux/s390x, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
If you build the image using CI/CD as gitlab-ci you just need to specify the --platform=linux/arm in you Dockerfile Before the image name.
Hope this helps !
Answered By - Louey
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.