Repository navigation
Added sidekiq cron to allow for the scheduling of jobs - #427
Conversation
|
|
||
| schedule_file = 'config/job_schedule.yml' | ||
|
|
||
| Sidekiq::Cron::Job.load_from_hash YAML.load_file(schedule_file) if File.exist?(schedule_file) && Sidekiq.server? |
There was a problem hiding this comment.
A couple things here.
Here's a neat trick! Move Sidekiq::Cron::Job.load_from_hash to be run from within the Sidekiq.configure_server block. This will have a similar effect as Sidekiq.server? to ensure that attempting to schedule jobs is only done when a Sidekiq main process is starting up and reuses Sidekiq's own pattern for making sure that only happens once when running Sidekiq on multiple hosts.
What do you think about keeping the schedule as hash directly in this initializer? Until such time as something else can write out the job_schedule.yml in production, the schedule is pretty static. Could be:
Sidekiq.configure_server do |config|
# blahblahblah other stuff
Sidekiq::Cron::Job.load_from_hash(
'meetup_member_sync_job' => {
class: 'MeetupMemberSyncJob',
cron: '0 0 5 31 2 ?' # Feb 31st, in other words, don't run it
},
'github_collect_statistics_job' => {
class: 'GithubCollectStatisticsJob',
cron: '0 0 5 31 2 ?' # Feb 31st, in other words, don't run it
}
)
endBut also, I acknowledge the ubiquity of YAML, which might be easier for others to read.
Also also, commenting these jobs out would work as well as scheduling them for an impossible date.
There was a problem hiding this comment.
I am concerned that the sidekiq initializer is going to get congested with job scheduling configurations and I was hoping to keep it clean by setting the expectation that the job schedule is in an external file. I don't really have a strong opinion one way or the other though.
There was a problem hiding this comment.
🤔 Keeping the initializer clean is a good goal and reason to keep the YAML schedule. 👍
Added the meetup sync job to run at 4am and added the cron tab to the sidekiq UI to make sure we have visibility into the scheduled jobs.
After discussing data collections, it's aware we need to wait until she nell returns so that we can make sure we won't get in trouble. Making this only run on feb 31.
bc5552f to
e94bb18
Compare
apex-omontgomery
left a comment
There was a problem hiding this comment.
Time to cook with gasoline.
Description of changes
Added the meetup sync job to run at 4am and added the cron tab to the sidekiq UI to make sure we have visibility into the scheduled jobs.
Issue Resolved
Fixes #163