The README says "It is possible to disable delayed jobs for testing purposes. Set Delayed::Worker.delay_jobs = false to execute all jobs realtime." This may be really bad advice.

Consider the following User model that implements various Devise strategies which support some kind of notification.

class User
  include Mongoid::Document
  devise :database_authenticatable, :registerable, ...
  field :notified_at, type: DateTime
  after_save :notify!, :if => :notify?
  def notify!
    super
    update_attributes!({ notified_at: Time.now.utc })
  end
end

We are overriding a black box notify! method and updating an attribute with a timestamp of the last notification. All tests green. Once this code hit production, it became an infinite loop. How is that possible?

The implementation of notify? is in the Devise gem and relies on an instance variable to signal that a notification has been sent. Setting the instance variable avoids sending the notification twice for multiple calls to save!. Our after_save callback invokes update_attributes!, which causes another notify! call unless notify? returns false. In a test, the call to super will execute the notification (setting the instance variable), but it's handled asynchronously in production with Delayed::Worker.delayed_jobs set to true. The notification will be delayed, without setting the instance variable. This is effectively code that works in a test, but loops indefinitely in production, creating as many delayed notifications as possible before being killed by Heroku after 30 seconds.

We re-enabled DJ in tests and used an observer from this gist to run them. But maybe we can have a feature in DJ that executes jobs immediately with identical semantics, like retry?