Plugins that add a Setting subclass and load it through an initialiser are failing to run tests, raising the following error during startup:

21:11:54 + bundle exec rake jenkins:unit TESTOPTS=-v
21:12:00 rm -rf jenkins/reports/unit/
21:12:03 DEPRECATION WARNING: You are using a deprecated behavior, it will be removed in version 1.13, 
21:12:03 Initializing plugins using :after => :finish_hook in your engine is deprecated. It delays the load of some assets in Sprockets. If you are the plugin author, please change your plugin engine initializer to use :before => :finisher_hook or after_initialize.  Affected initializer name is foreman_bootdisk.register_plugin. (called from each at /var/lib/workspace/workspace/test_plugin_matrix/database/sqlite3/ruby/2.1/slave/fast/foreman/config/initializers/assets_paths.rb:10)
21:12:03 rake aborted!
21:12:03 NoMethodError: super: no superclass method `load_defaults' for #<Class:0x00000002d13e48>
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activerecord-4.1.5/lib/active_record/dynamic_matchers.rb:26:in `method_missing'
21:12:03 /var/lib/workspace/workspace/test_plugin_matrix/database/sqlite3/ruby/2.1/slave/fast/foreman/app/models/setting/general.rb:5:in `load_defaults'
21:12:03 /var/lib/workspace/workspace/test_plugin_matrix/database/sqlite3/ruby/2.1/slave/fast/foreman/config/initializers/foreman.rb:21:in `each'
21:12:03 /var/lib/workspace/workspace/test_plugin_matrix/database/sqlite3/ruby/2.1/slave/fast/foreman/config/initializers/foreman.rb:21:in `<top (required)>'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:241:in `load'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:241:in `block in load'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:232:in `load_dependency'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:241:in `load'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/engine.rb:648:in `block in load_config_initializer'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/notifications.rb:161:in `instrument'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/engine.rb:647:in `load_config_initializer'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/engine.rb:612:in `block (2 levels) in <class:Engine>'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/engine.rb:611:in `each'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/engine.rb:611:in `block in <class:Engine>'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/initializable.rb:30:in `instance_exec'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/initializable.rb:30:in `run'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/initializable.rb:55:in `block in run_initializers'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/initializable.rb:44:in `each'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/initializable.rb:44:in `tsort_each_child'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/initializable.rb:54:in `run_initializers'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/application.rb:300:in `initialize!'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/railtie.rb:194:in `public_send'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/railtie.rb:194:in `method_missing'
21:12:03 /var/lib/workspace/workspace/test_plugin_matrix/database/sqlite3/ruby/2.1/slave/fast/foreman/config/environment.rb:5:in `<top (required)>'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `require'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `block in require'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:232:in `load_dependency'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `require'
21:12:03 /var/lib/workspace/workspace/test_plugin_matrix/database/sqlite3/ruby/2.1/slave/fast/foreman/test/test_helper.rb:20:in `block in <top (required)>'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/spork-1.0.0rc4/lib/spork.rb:24:in `prefork'
21:12:03 /var/lib/workspace/workspace/test_plugin_matrix/database/sqlite3/ruby/2.1/slave/fast/foreman/test/test_helper.rb:9:in `<top (required)>'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `require'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `block in require'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:232:in `load_dependency'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `require'
21:12:03 /var/lib/workspace/workspace/test_plugin_matrix/database/sqlite3/ruby/2.1/slave/fast/foreman/test/unit/application_mailer_test.rb:1:in `<top (required)>'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `require'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `block in require'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:232:in `load_dependency'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `require'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/test_unit/sub_test_task.rb:114:in `block (3 levels) in define'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/test_unit/sub_test_task.rb:114:in `each'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/test_unit/sub_test_task.rb:114:in `block (2 levels) in define'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/test_unit/sub_test_task.rb:113:in `each'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/gems/railties-4.1.5/lib/rails/test_unit/sub_test_task.rb:113:in `block in define'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/bin/ruby_executable_hooks:15:in `eval'
21:12:03 /usr/local/rvm/gems/ruby-2.1.5@test_plugin_matrix-3/bin/ruby_executable_hooks:15:in `<main>'
21:12:03 Tasks: TOP => jenkins:unit => test:units
21:12:03 (See full trace by running task with --trace)

They all started failing between Jan 28, 2016 9:42 PM and Jan 31, 2016 3:54 AM. I can't narrow down any commit on Foreman develop that causes it, suggesting something external (like a gem). 1.10-stable seems unaffected.

It can be reproduced locally by deleting the test DB and running rake test.