[FIXED] User Data in AWS CloudFormation for an EC2 Instance through a custom AMI not working

Issue

I am unable to figure out why the echo commands in the shell script for EC2 Instance User Data creation in AWS CloudFormation is not running. While the "sudo systemctl" commands are working.

I did try "sudo echo" as well, which did not work.

There are no errors. The python flask application which is set to run on bootup using "sudo systemctl" command is working fine. But there is no .env file created.

I am using the free-tier Amazon Linux image from the AMI catalog:

Amazon Linux 2 AMI (HVM) - Kernel 5.10, SSD Volume Type ami-0c02fb55956c7d316 (64-bit (x86))

ImageId is a reference to the custom AMI I created for the python flask application based on the AMI mentioned above.

~/webapp/release is my working directory

# Create EC2 Instance
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !Ref ImageId
      InstanceType: t2.micro
      KeyName: csye6225
      UserData:
        Fn::Base64: 
          !Sub |
              #!/bin/bash
              cd ~/webapp/release
              echo "DB_HOST=\"${DatabaseInstance.Endpoint.Address}\"" >> .env
              echo "DB_PORT=\"${DatabaseInstance.Endpoint.Port}\"" >> .env
              echo "DB_DATABASE=\"${DatabaseName}\"" >> .env
              echo "DB_USERNAME=\"${DatabaseUser}\"" >> .env
              echo "DB_PASSWORD=\"${DatabasePassword}\"" >> .env
              echo "FILESYSTEM_DRIVER=\"s3\"" >> .env
              echo "AWS_BUCKET_NAME=\"${S3Bucket}\"" >> .env
              cd /etc/systemd/system
              sudo systemctl daemon-reload
              sudo systemctl enable flaskapp.service
              sudo systemctl start flaskapp.service
              sudo systemctl status flaskapp.service
      BlockDeviceMappings:
        - DeviceName: /dev/xvda
          Ebs:
            VolumeType: "gp2"
            DeleteOnTermination: "true"
            VolumeSize: "20"
      NetworkInterfaces:
        - AssociatePublicIpAddress: "true"
          DeviceIndex: "0"
          SubnetId: !Ref PublicSubnet2
          GroupSet: [!Ref SSHSecurityGroup]
      IamInstanceProfile: !Ref DemoInstanceProfile
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-EC2"

Error Log from /var/log/cloud-init-output.log:

Cloud-init v. 19.3-45.amzn2 running 'modules:final' at Fri, 25 Mar 2022 00:46:29 +0000. Up 19.92 seconds.
Created symlink from /etc/systemd/system/multi-user.targer.wants/flaskapp.service to /etc/systemd/system/flaskapp.service.
⬤ flaskapp.service - Flask App service
    Loaded: loaded (/etc/systemd/system/flaskapp.service; enabled; vendor preset: disabled)
    Active: active (running) since Fri 2022-03-25 UTC; 45ms ago
Main PID: 3382 ((bash))
    CGroup: /system.slice/flaskapp.service
            |__3382 (bash)

Solution

As @Marcin suggested, fixed the issue with file path in the first line of the user data script. But still the user data script did not execute. Since my AMI was a custom AMI, I had to add below fixes to make it working.

Looked up more resources on this and eventually this solution worked out perfectly.

My userdata script now has below commands:

      #!/bin/bash
      cd /home/ec2-user/webapp
      sudo rm -rf /var/lib/cloud/instance
      sudo rm -rf /var/lib/cloud/instances
      echo -n "" > .env
      echo DB_HOST=${DatabaseInstance.Endpoint.Address} >> .env
      echo DB_PORT=${DatabaseInstance.Endpoint.Port} >> .env
      echo DB_DATABASENAME=${DatabaseName} >> .env
      echo DB_USERNAME=${DatabaseUser} >> .env
      echo DB_PASSWORD=${DatabasePassword} >> .env
      echo FILESYSTEM_DRIVER=s3 >> .env
      echo BUCKET=${S3Bucket} >> .env
      echo FLASK_ENV=development >> .env
      cd /etc/systemd/system
      sudo systemctl daemon-reload
      sudo systemctl enable flaskapp.service
      sudo systemctl start flaskapp.service
      sudo systemctl status flaskapp.service

Answered By - supersaiyan