Running IDL code with cron jobs


This page explains how I set up cron jobs in order to run IDL code at regular intervals. One example of the use of this method is for the EIS Mapper webpage which is updated on a weekly basis.

My method is to create a shell script that runs some IDL batch code. I then create a cron job to run the script.

The shell script

The shell script that I run for the weekly updates to EIS Mapper is:

#!/bin/csh

rm -f $HOME/scripts/eis_mapper_weekly.log
sidl $HOME/scripts/eis_mapper_weekly.pro >> $HOME/scripts/eis_mapper_weekly.log
        

Note that I am using the C-shell. Make sure that your script is executable by using "chmod u+x".

The IDL code

The IDL code is written in batch mode, which means that it is not an IDL procedure. Each line in the code must be a "standalone" line, just as you would use it on the command line. The code must end with "exit" (not "end").

dat=systime(/jul)
caldat,dat,curr_month,curr_day,curr_year
IF curr_month EQ 1 THEN prev_month=12 ELSE prev_month=curr_month-1
IF curr_month EQ 1 THEN prev_year=curr_year-1 ELSE prev_year=curr_year
eis_mapper_obs_str,year=prev_year,month=prev_month,/overwrite
eis_mapper_obs_str,year=curr_year,month=curr_month,/overwrite
eis_mapper_year
IF prev_year NE curr_year THEN eis_mapper_year,prev_year
eis_mapper_year_index
exit
        

Creating the cron job

To edit your cron file, do:

> crontab -e

Note that the file will open with your default text editor. You can set this with the environment variable $EDITOR. I set mine to "vi" since this opens within the terminal (useful for operation on remote machines).

To run your script at 17:00 every Sunday, you would insert the following line into the cron file:

0 17 * * 0  ~/scripts/eis_mapper_weekly.csh
        

You can find out more information about the cron options from the Wikipedia page.

Troubleshooting

Catalina and "full disk access"

I came across a problem when using MacOS Catalina for a cron job that tried to access data on an external hard drive. The script would run fine if I ran it from the command line, but when I used cron it kept crashing.

Apple introduced new security measures in Catalina that prevent apps from accessing certain folders, including folders on removable drives. Whereas Terminal had "full disk access" (allowing my script to run successfully), cron did not.

You can find out more information about how this new feature impacts IDL at the L3Harris website.

The procedure for giving cron full disk acces is explained at this blog by Ricard Bejarano. You should check with your sysadmin before doing this, though.


Page maintained by Dr Peter R Young.