Building an Embedded Crontab System
Today I was working on my latest new application for price feeding. Part of this app is a crontab-like system. I've put several into several apps I've built in recent years, but most of them have been database-driven primarily because I thought it would be easier to use, and I could get something far better than the conventional crontab system from Unix. I was wrong.
Oh, the database-driven crontab system works fine, but it requires a database connection to work, which has it's pros and cons. The pros are obviously that it can easily be global, easily modified, thread-safe, and secure. You can write any tools you want to edit it, and it can be as complex or as simple as you want.
The cons are that unless you really take the time to write the tool, using SQL to modify the jobs is a real pain in the neck. My problem has been that I change the database crontabs so infrequently that building a tool has never really been necessary.
So this time, with this price feeder, I decided that a file-based crontab was going to be easier to use, easier to understand, and just as flexible. Face it... it's a bunch of fields where the first five are the when and the sixth is the what. So today I built a nice little crontab system that easily fits into applications.
The format of the crontab file is the same as unix crontabs, so it's easy for people to know what to expect when they look at it. The thread that runs in the app that checks for differences is simple - simply using stat() to get the last modification time, and checking to see if it's after the last known modification time of the file. If it's been updated, then we drop all the loaded jobs, reload and reparse the crontab into a series of jobs, and then check what jobs need to be done this minute.
I keep the time of the last minute we ran, sleep for 2 sec between checks, so we're going to be firing off the at-the-minute-jobs pretty darn close to the top of the minute. Also, we're not going to be firing off the same job more than once in the target minute. Pretty simple.
The most interesting thing was the parsing of the time field codes. Crontab allows three basic styles that can be put together with commas, separated by any whitespace:
- n - a single entry. For all fields it can be numeric, and for the months and days of the week field it can be the names.
- n-m - a range, inclusive of the endpoints. Again, for all fields it can be numeric and for the month and day of the week it can be the names.
- */i - every i units. This requires the i be numeric on all fields and will indicate valid times every i units. Therefore, */5 in the minutes column means every 5 minutes.
It was really surprisingly fun to have each of these cases handled nicely in the code. The logic was pretty simple - parse these fields into lists of applicable values and then put them into lists. When checking a time, simply break the actual time into the necessary components and see if each component fits into the approved values for that field for that job. If all match, then the job should be executed. If one misses, then skip it and go to the next.
With the caching of the cronjobs, we're not doing more reading and parsing than we need to. This makes the system less consuming of resources, which is always good. The command portion of the job can be anything we want - complex or simple, it's just up to what I need the app to be able to do.
When I get this app all done and everything is working, I'll probably pull this out and put it into CKit as a more reusable component. I just want to make sure that it's got all the features I need before I put it in CKit.