Skip to content
This repository was archived by the owner on Dec 3, 2019. It is now read-only.

Added sidekiq cron to allow for the scheduling of jobs - #427

Merged
apex-omontgomery merged 4 commits into
OperationCode:masterfrom
BRIMIL01:add_sidekiq_cron
Oct 10, 2018
Merged

apex-omontgomery merged 4 commits into
OperationCode:masterfrom
BRIMIL01:add_sidekiq_cron

Conversation

@BRIMIL01

@BRIMIL01 BRIMIL01 commented Oct 6, 2018

Copy link
Copy Markdown
Contributor

Description of changes

Added the meetup sync job to run at 4am and added the cron tab to the sidekiq UI to make sure we have visibility into the scheduled jobs.

Issue Resolved

Fixes #163

Comment thread config/initializers/sidekiq.rb Outdated

schedule_file = 'config/job_schedule.yml'

Sidekiq::Cron::Job.load_from_hash YAML.load_file(schedule_file) if File.exist?(schedule_file) && Sidekiq.server?

@robbkidd robbkidd Oct 9, 2018 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple things here.

Here's a neat trick! Move Sidekiq::Cron::Job.load_from_hash to be run from within the Sidekiq.configure_server block. This will have a similar effect as Sidekiq.server? to ensure that attempting to schedule jobs is only done when a Sidekiq main process is starting up and reuses Sidekiq's own pattern for making sure that only happens once when running Sidekiq on multiple hosts.

What do you think about keeping the schedule as hash directly in this initializer? Until such time as something else can write out the job_schedule.yml in production, the schedule is pretty static. Could be:

Sidekiq.configure_server do |config|
  # blahblahblah other stuff
  Sidekiq::Cron::Job.load_from_hash(
    'meetup_member_sync_job' => {
      class: 'MeetupMemberSyncJob',
      cron: '0 0 5 31 2 ?' # Feb 31st, in other words, don't run it
    },
    'github_collect_statistics_job' => {
      class: 'GithubCollectStatisticsJob',
      cron: '0 0 5 31 2 ?' # Feb 31st, in other words, don't run it
    }
  )
end

But also, I acknowledge the ubiquity of YAML, which might be easier for others to read.

Also also, commenting these jobs out would work as well as scheduling them for an impossible date.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am concerned that the sidekiq initializer is going to get congested with job scheduling configurations and I was hoping to keep it clean by setting the expectation that the job schedule is in an external file. I don't really have a strong opinion one way or the other though.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 Keeping the initializer clean is a good goal and reason to keep the YAML schedule. 👍

BRIMIL01 and others added 4 commits October 9, 2018 17:59
Added the meetup sync job to run at 4am and added the cron tab to the
sidekiq UI to make sure we have visibility into the scheduled jobs.
After discussing data collections, it's aware we need to wait until she nell returns so that we can make sure we won't get in trouble. Making this only run on feb 31.

@apex-omontgomery apex-omontgomery left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Time to cook with gasoline.

@apex-omontgomery
apex-omontgomery merged commit a4736c0 into OperationCode:master Oct 10, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants