I have a Anaconda Python Virtual Environment set up and if I run my project while that virutal environment is activated everything runs great.
But I have a cronjob configured to run it every hour. I piped the output to a log because it wasn't running correctly.
crontab -e:
10 * * * * bash /work/sql_server_etl/src/python/run_parallel_workflow.sh >> /home/etlservice/cronlog.log 2>&1
I get this error in the cronlog.log:
Traceback (most recent call last):
File "__parallel_workflow.py", line 10, in <module>
import yaml
ImportError: No module named yaml
That is indicative of the cronjob somehow not running the file without the virtual environment activated.
To remedy this I added a line to the /home/user/.bash_profile file:
conda activate ~/anaconda3/envs/sql_server_etl/
Now when I login the environment is activated automatically.
However, the problem persists.
I tried one more thing. I changed the cronjob, (and I also tried this in the bash file the cronjob runs) to explicitly manually activate the environment each time it runs, but to no avail:
10 * * * * conda activate ~/anaconda3/envs/sql_server_etl/ && bash /work/sql_server_etl/src/python/run_parallel_workflow.sh >> /home/etlservice/cronlog.log 2>&1
Of course, nothing I've tried has fixed it. I really know nothing about linux so maybe there's something obvious I need to change.
So, is there anyway to specify that the cronjob should run under a virutal environment?
Posted a working solution (on Ubuntu 18.04) with detailed reasoning on SO.
The short form is:
~/.bashrc (at the end of the file) to a separate file ~/.bashrc_condaAs of Anaconda 2020.02 installation, the snippet reads as follows:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/USERNAME/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/home/USERNAME/anaconda3/etc/profile.d/conda.sh" ]; then
. "/home/USERNAME/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/home/USERNAME/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
Make sure that:
/home/USERNAME/anaconda3/ is correct.~/.bashrc_conda (and no other user can write to this file).crontab -e add lines to run cronjobs on bash and to source ~/.bashrc_condaRun crontab -e and insert the following before the cronjob:
SHELL=/bin/bash
BASH_ENV=~/.bashrc_conda
crontab -e include at beginning of the cronjob conda activate my_env; as in exampleExample of entry for a script that would execute at noon 12:30 each day on the Python interpreter within the conda environment:
30 12 * * * conda activate my_env; python /path/to/script.py; conda deactivate
And that's it.
You may want to check from time to time that the snippet in ~/.bashrc_conda is up to date in case conda updates its snippet in ~/.bashrc.
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments