-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[review] delayed_cron_job を入れ替える rake タスク #33
Changes from 3 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
module SgFargateRails | ||
class DelayedCronJobUtility | ||
class << self | ||
def refresh_cron_jobs! | ||
DelayedCronJobUtility.new.refresh_cron_jobs! | ||
end | ||
|
||
def cron_jobs | ||
Delayed::Job.where.not(cron: nil) | ||
end | ||
end | ||
|
||
def initialize | ||
unless defined?(::DelayedCronJob) | ||
raise 'DelayedCronJob not defined.' | ||
end | ||
end | ||
|
||
def refresh_cron_jobs! | ||
ActiveRecord::Base.transaction do | ||
destroy_cron_jobs! | ||
create_cron_jobs! | ||
end | ||
end | ||
|
||
private | ||
|
||
def destroy_cron_jobs! | ||
DelayedCronJobUtility.cron_jobs.find_each do |delayed_job| | ||
# NOTE: 念のため cron が設定されていることを再チェック | ||
if delayed_job.cron.present? | ||
delayed_job.destroy! | ||
end | ||
end | ||
end | ||
|
||
def create_cron_jobs! | ||
cron_settings.each do |_name, options| | ||
job_class = options[:class] | ||
job_class = options[:class].constantize unless job_class.is_a?(Class) | ||
|
||
args = options[:args] | ||
if args.blank? | ||
job_class.set(cron: options[:cron]).perform_later | ||
elsif args.is_a?(Array) | ||
job_class.set(cron: options[:cron]).perform_later(*args) | ||
elsif args.is_a?(Hash) | ||
job_class.set(cron: options[:cron]).perform_later(**args) | ||
else | ||
raise 'invalid args option.' | ||
end | ||
end | ||
end | ||
|
||
def cron_settings | ||
@cron_settings ||= Rails.application.config_for('delayed_cron_jobs') | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. delayed_cron_jobs.yamlのテンプレートは作成しない? 上記のテンプレートを作成するかどうかの設定化もやった方がよさそう。 まだ検証フェーズということであればあとでやるでもよさそう。 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. テンプレートジェネレートした方が良さそう。 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
end | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,5 @@ | ||
require 'sg_fargate_rails/adjust_cloudfront_headers' | ||
require 'sg_fargate_rails/delayed_cron_job_utility' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. delayed_jobが有効な場合のみrequireするほうが良さそう。 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ですよね。
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 必要になったタイミングでrequireするほうがエコだし、refresh_delayed_cron_jobsのrakeタスク内でrequireすれば良さそう! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. そうか!ここで require するものと完全に思い込んでました! |
||
require 'sg_fargate_rails/healthcheck' | ||
require 'sg_fargate_rails/maintenance' | ||
require 'sg_fargate_rails/remote_ip' | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -29,4 +29,11 @@ namespace :sg_fargate_rails do | |
) | ||
end | ||
end | ||
|
||
desc 'Refresh Delayed Cron Jobs' | ||
task refresh_delayed_cron_jobs: :environment do | ||
Rails.logger.info('[refresh_delayed_cron_jobs] refresh begin...') | ||
SgFargateRails::DelayedCronJobUtility.refresh_cron_jobs! | ||
Rails.logger.info('[refresh_delayed_cron_jobs] refresh end.') | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. delayed_jobが有効な場合のみrakeタスクを定義するほうが良さそう。 |
||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
この条件だけだと、現在進行系で動いてるジョブのレコードを消すことにならない?と思いました。(ちょうどいい属性があるかは未確認)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
実行中のジョブもキューデータが消されます。消えてもワーカーとしては問題なく動作することを確認してます。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
懸念があるとすると
処理が長引いて、本来はスキップされるはずの次のスケジュールに新しくジョブが追加されて、別のワーカーが部分的に並列実行しちゃうかも
でも、これは opsworks でも起きてるはずなので(そもそもスケジュールスキップされないはず)問題ではなさそう
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ジョブが成功する場合はいいとして、例外などで失敗した場合も問題ないですか?
ジョブが失敗したことを記録する先がなくなって、期待しない動作にならないかなーと。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
考えてなかった!
処理用の別のジョブを産んでくれればよいのにな delayed_cron_job ...
delayed_cron_job → rake task → delayed_job という処理順にしてもらうしかないのかな
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
エラー情報が紛失する?
=> bugsnag に残るはず
リトライを期待してたら困る?
=> 現状 (OpsWorks) の cron はリトライされないからその想定はないはず
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
異常系の実機検証してみた。
cron job でエラーが発生したら jobmon, bugsnag, cloudwatch logs にエラー情報が残る
DB (delayed_jobs) からは deploy でエラー情報が消えてしまうが、↑の3つのエラー情報があるから問題はなさそう。
cron job の処理実行中にデプロイによる refresh タスクが実行されたら
コンテナが停止され
jobmon => running のままで通知が来る
bugsnag => "rake aborted!"
cloudwatch logs => SIGTERM のログあり