[FIXED] How do I run Django Management > Commands as cron jobs in Elastic Beanstalk running nginx

Issue

I'm upgrading a Django application from Elastic Beanstalk Python 3.6 running on 64bit Amazon Linux to Python 3.8 running on 64bit Amazon Linux 2 and moving from the Apache server to Nginx.

I'm trying to set up the cron jobs.

On Linux 1 I had 03_files.config

container_commands:
  ...
  
  02_cron_job:
    command: "cp .ebextensions/crontab.txt /etc/cron.d/autumna_cron_jobs && chmod 644 /etc/cron.d/autumna_cron_jobs"
    #leader_only: true

   ...

with crontab.txt

# Set the cron to run with utf8 encoding
PYTHONIOENCODING=utf8

...

30 0 * * * root source /opt/python/current/env && nice /opt/python/current/app/src/manage.py test_cron
...
# this file needs a blank space as the last line otherwise it will fail

I've converted this to 03_cron.config

container_commands:
  01_cron_job:
    command: "cp .ebextensions/crontab.txt /etc/cron.d/autumna_cron_jobs && chmod 644 /etc/cron.d/autumna_cron_jobs"
    #leader_only: true

with crontab.txt

# Set the cron to run with utf8 encoding
PYTHONIOENCODING=utf8

...

30 0 * * * root source /var/app/venv/staging-LQM1lest/bin/activate && nice /var/app/current/manage.py test_cron
...
# this file needs a blank space as the last line otherwise it will fail

I've actually got it running every 10 minutes to try and understand what is happening. When I look in the logs, withing \var\log\cron I have:

Apr  9 09:10:01 ip-172-31-41-78 CROND[22745]: (root) CMD (source /var/app/venv/staging-LQM1lest/bin/activate && nice /var/app/current/manage.py test_cron >> /var/log/test_cron.log.log 2>&1)
Apr  9 09:20:01 ip-172-31-41-78 CROND[22842]: (root) CMD (source /var/app/venv/staging-LQM1lest/bin/activate && nice /var/app/current/manage.py test_cron >> /var/log/test_cron.log.log 2>&1)

I think this means it is trying to run it. I do not have a test_cron.log file

Within \var\log\messages I have

Apr  9 09:20:01 ip-172-31-41-78 systemd: Started Session 18 of user root.
Apr  9 09:20:01 ip-172-31-41-78 systemd: Starting Session 18 of user root.
Apr  9 09:20:01 ip-172-31-41-78 systemd: Started Session 17 of user root.
Apr  9 09:20:01 ip-172-31-41-78 systemd: Starting Session 17 of user root.

Within the other reports I'm seeing no indication anything is going on. I don't know what is wrong so how do I get these cron jobs running?

** EDIT **

I have amended my crontab line as follows:

* * * * * root source /var/app/venv/*/bin/activate && /var/app/current/manage.py test_cron 

I have ssh'd into the application and run both those commands one after the other and it works - it should send me an email and it does. I know it is initiating because within my cron log file I have

Apr 12 10:31:01 ip-172-31-39-9 CROND[19019]: (root) CMD (source /var/app/venv/*/bin/activate && /var/app/current/manage.py test_cron )
Apr 12 10:32:01 ip-172-31-39-9 CROND[20926]: (root) CMD (source /var/app/venv/*/bin/activate && /var/app/current/manage.py test_cron )
Apr 12 10:33:01 ip-172-31-39-9 CROND[21215]: (root) CMD (source /var/app/venv/*/bin/activate && /var/app/current/manage.py test_cron )
Apr 12 10:34:01 ip-172-31-39-9 CROND[21234]: (root) CMD (source /var/app/venv/*/bin/activate && /var/app/current/manage.py test_cron )
Apr 12 10:35:01 ip-172-31-39-9 CROND[21251]: (root) CMD (source /var/app/venv/*/bin/activate && /var/app/current/manage.py test_cron )
Apr 12 10:36:01 ip-172-31-39-9 CROND[21296]: (root) CMD (source /var/app/venv/*/bin/activate && /var/app/current/manage.py test_cron )

But I do not receive the email and I cannot find any errors. Please - any ideas most welcome

Solution

For anyone experiencing similar issue, I found my issue. There were some environment variables (set for the application) which, and I do not know why, did not need setting on the Linux 1 instance but did on the Linux 2 instance. Once added to the crontab file, it worked.

I did it by adding .platform/hooks/postdeploy/01_copy_env.sh

#!/bin/bash

#Create a copy of the environment variable file.
cp /opt/elasticbeanstalk/deployment/env /tmp/custom_env_var #aws does not give access to /opt/elasticbeanstalk/deployment/env post deployment

#Set permissions to the custom_env_var file so this file can be accessed by any user on the instance. You can restrict permissions as per your requirements.
chmod 644 /tmp/custom_env_var

and setting its permissions chmod +x .platform/hooks/postdeploy/01_copy_env.sh

Then within my crontab file I've added

#!/bin/bash  
source /tmp/custom_env_var

Answered By - HenryM