Project

General

Profile

Actions

Bug #24648

closed

undefined method `with_indifferent_access' for #<String:0x00000008ecc658> (NoMethodError)

Added by Björn Zettergren almost 6 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Difficulty:
Triaged:
No
Found in Releases:

Description

When importing facts from a host via API request we get


NoMethodError
undefined method `[]' for nil:NilClass

and the following trace:


---
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/foreman/host/import_facts.rb:28:in
  `block in run'" 
- "/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:94:in `as'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/foreman/host/import_facts.rb:27:in
  `run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action.rb:538:in
  `block (3 levels) in execute_run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/middleware/rails_executor_wrap.rb:14:in
  `block in run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.6/lib/active_support/execution_wrapper.rb:85:in
  `wrap'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/middleware/rails_executor_wrap.rb:13:in
  `run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in
  `call'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action/progress.rb:30:in
  `with_progress_calculation'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action/progress.rb:16:in
  `run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in
  `call'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/middleware/keep_current_user.rb:15:in
  `block in run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/middleware/keep_current_user.rb:43:in
  `restore_curent_user'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/middleware/keep_current_user.rb:15:in
  `run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in
  `call'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:31:in
  `run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in
  `call'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/middleware/world.rb:30:in
  `execute'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action.rb:537:in
  `block (2 levels) in execute_run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action.rb:536:in
  `catch'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action.rb:536:in
  `block in execute_run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action.rb:451:in
  `block in with_error_handling'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action.rb:451:in
  `catch'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action.rb:451:in
  `with_error_handling'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action.rb:531:in
  `execute_run'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/action.rb:278:in
  `execute'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in
  `block (2 levels) in execute'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract.rb:162:in
  `with_meta_calculation'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in
  `block in execute'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:30:in
  `open_action'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:15:in
  `execute'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/director.rb:43:in
  `execute'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/executors/parallel/worker.rb:12:in
  `block in on_message'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/executors.rb:12:in
  `run_user_code'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/executors/parallel/worker.rb:11:in
  `on_message'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/context.rb:46:in
  `on_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/executes_context.rb:7:in
  `on_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/dynflow-1.1.0/lib/dynflow/actor.rb:26:in
  `on_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/awaits.rb:15:in
  `on_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in
  `on_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:38:in
  `process_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:31:in
  `process_envelopes?'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:20:in
  `on_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/termination.rb:55:in
  `on_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/removes_child.rb:10:in
  `on_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in
  `pass'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in
  `on_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:161:in
  `process_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:95:in
  `block in on_envelope'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:118:in
  `block (2 levels) in schedule_execution'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
  `block in synchronize'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
  `synchronize'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
  `synchronize'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:115:in
  `block in schedule_execution'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:18:in
  `call'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:96:in
  `work'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:77:in
  `block in call_job'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in
  `run_task'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in
  `block (3 levels) in create_worker'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in
  `loop'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in
  `block (2 levels) in create_worker'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in
  `catch'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in
  `block in create_worker'" 
- "/usr/share/foreman/vendor/ruby/2.3.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in
  `block in create_with_logging_context'" 

I've tried sending very small custom jsons with only one fact in them, and they also fail in the same manner.

How can we troubleshoot this further?


Related issues 3 (1 open2 closed)

Related to foreman-tasks - Bug #24872: Update ImportFacts to not pass proxy_id to @import_host@ClosedIvan NecasActions
Related to Chef - Bug #24888: NoMethodError when importing facts from hostClosedIvan NecasActions
Related to Templates - Bug #37234: Template import fails in Foreman 3.9.1NewActions
Actions #1

Updated by Ondřej Pražák over 5 years ago

Could you add the exact request you are using please?

Actions #2

Updated by Björn Zettergren over 5 years ago

Ondřej Pražák wrote:

Could you add the exact request you are using please?

Well, almost exact:


curl -v -d @data.txt -X POST -H "User-Agent: Ruby" -H "X-Foreman-Client: <REDACTED>" -H "X-Foreman-Signature: <REDACTED>" -H "Host: <REDACTED>" -H  "Content-Type: application/json" http://<REDACTED>/api/hosts/facts

where data.txt is:


{"name":"<REDACTED>","facts":{"dmi::system::manufacturer":"QEMU"}}

I've also tried with other facts:


{"name":"<REDACTED>","facts":{"platform":"ubuntu"}}

Our foreman installation has been upgraded numerous times, don't even know which version we started out with 3-4-5 years ago, maybe there's something missing in the database. I'm just not really sure on how to debug rails/ruby to find out what it tries to do. Thoughts? :-)

Actions #3

Updated by Björn Zettergren over 5 years ago

Just found that this is related to having FOREMAN_TASKS_MONKEYS=true for foreman_tasks.

Fact uploads work without the environment variable, but fails in said manner with it.

Also, i verified on a clean install of Foreman 1.18.1, and added foreman-tasks + FOREMAN_TASKS_MONKEYS=true, and it fails in the same way.

Is it possible to move this issue to foreman-tasks, or do i have to create a new issue there?

Actions #4

Updated by Björn Zettergren over 5 years ago

  • Tracker changed from Support to Bug
  • Found in Releases 1.19.0 added
  • Found in Releases deleted (1.18.1)

There are two parts to this issue, both contained in "/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/foreman/host/import_facts.rb", and triggers when "FOREMAN_TASKS_MONKEYS=true".

  1. The "input" variable can contain differently named keys, I've encountered "managed" (if foreman manages the host) and "host" (if foreman does NOT manage the host), but the script only ever uses "host", like line 28 where it tries to access "input[:host][:id]", which causes NoMethodError in case of a managed host.
  2. the "input" variable "fact"-key gets converted from its original Hash to String, I don't know where this is done though. When "input[:facts]" is a string, this causes the following error:
    undefined method `with_indifferent_access' for #<String:0x00000008ecc658> (NoMethodError)
     | /usr/share/foreman/app/models/host/base.rb:136:in `import_facts' 
    

I have written a real ugly hack that works around these issues in the following branch:
https://github.com/bjozet/foreman-tasks/tree/debug_import_facts

Specifically my changes are https://github.com/bjozet/foreman-tasks/blob/b29735edf9dbc37d93a77ed447eedfdb06451169/app/lib/actions/foreman/host/import_facts.rb#L28-L35:

            # output contents of 'input' variable in debug log.
            ::Foreman::Logging.logger('foreman-tasks').debug "'input' value: #{input}" 
            # input[:facts] gets converted to an escaped String, fix by "eval":ing back to hash
            input[:facts] = eval(input[:facts])
            # input contains key-name "managed" if host is managed, or "host" 
            # if unmanaged by foreman.
            host           = ::Host.find(input[:managed][:id]) if input.has_key?(:managed)
            host           = ::Host.find(input[:host][:id]) if input.has_key?(:host)

However that is by no means a viable solution to this issue, but I don't have the overall picture or understanding of foreman/ruby/rails to take it any further at this point.

Questions:
  1. Are there more types, than "managed" and "host"?
  2. Should the input[:fact] really be a String or is this a bug (in foreman 1.16 it is a Hash)?
  3. Where is input mangled before it reaches the import_facts.rb-script?
  4. Can this issue be moved to foreman-tasks?
Actions #5

Updated by Ivan Necas over 5 years ago

  • Project changed from Foreman to foreman-tasks
Actions #6

Updated by Ivan Necas over 5 years ago

  • Status changed from New to Need more information

I was looking into this issue, and I suspect this commit in Chef plugin might be in charge: https://github.com/theforeman/foreman_chef/commit/0761b0ddb094b96da5b48868eea4e24a0149c713

@Bjorn: you are using the chef plugin, right?

Actions #7

Updated by Björn Zettergren over 5 years ago

  • Status changed from Need more information to Feedback

Ivan Necas wrote:

@Bjorn: you are using the chef plugin, right?

Yes we are using chef plugin, and I now realize that I did not mention that in my other post 5 days ago when i described my setup, sorry about that.

So I verified again on a (definitely) clean install with only foreman 1.19 and foreman-tasks 0.13.2, without foreman-chef. And that does not fail with the :managed / :host problem i wrote about, but it does fail with the second issue, "with_indifferent_acces":

2018-09-04T19:09:36 [D|for|] value of INPUT: {"host"=>{"id"=>1, "name"=>"test02.local.net"}, "facts"=>"{\"operatingsystemrelease\"=>\"1.0\", \"operatingsystem\"=>\"vos\"}"}
2018-09-04T19:09:36 [I|app|] Current user set to admin (admin)
2018-09-04T19:09:36 [E|bac|] undefined method `with_indifferent_access' for #<String:0x00000008d2a200> (NoMethodError)
 | /usr/share/foreman/app/models/host/base.rb:136:in `import_facts'
 | /usr/share/foreman/app/models/host/managed.rb:317:in `import_facts'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/foreman/host/import_facts.rb:30:in `block in run'
 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:100:in `as'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/foreman/host/import_facts.rb:28:in `run'

and this is due to the string-ification of input[:facts], if convert it to Hash importing works, but it feels like the wrong approach

--- /usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/foreman/host/import_facts.rb.original    2018-09-04 19:40:12.986764008 +0000
+++ /usr/share/foreman/vendor/ruby/2.3.0/gems/foreman-tasks-0.13.2/app/lib/actions/foreman/host/import_facts.rb    2018-09-04 19:41:54.894764008 +0000
@@ -24,7 +24,9 @@
         end

         def run
+          ::Foreman::Logging.logger('foreman-tasks').debug "value of INPUT: #{input}" 
           ::User.as :admin do
+            input[:facts] = eval(input[:facts])
             host           = ::Host.find(input[:host][:id])
             state          = host.import_facts(input[:facts])
             output[:state] = state

Unfortunately I know way to little about ruby/rails to understand what that foreman-chef commit you linked to actually does :/
Should I file a new ticket against foreman_chef for the :managed/:host part of the input variable, and we focus on fixing the input[:facts] thing?

Actions #8

Updated by Björn Zettergren over 5 years ago

To send facts, use curl and a json:

curl -u admin:password -v -d @/etc/foreman/test-tasks.json -H "Content-Type: application/json" http://localhost:3000/api/hosts/facts

Where /etc/foreman/test-tasks.json is:

{
    "facts": {
        "chef_node_name": "ubuntu-xenial",
        "cookbooks::git::version": "0.1.0",
        "operatingsystem": "Ubuntu",
        "operatingsystemrelease": "16.04" 
    },
    "name": "test01.local.net" 
}

Edit for appropriate names/timestamps

Actions #9

Updated by Ivan Necas over 5 years ago

  • Related to Bug #24872: Update ImportFacts to not pass proxy_id to @import_host@ added
Actions #10

Updated by Ivan Necas over 5 years ago

  • Subject changed from NoMethodError when importing facts from host to undefined method `with_indifferent_access' for #<String:0x00000008ecc658> (NoMethodError)
  • Status changed from Feedback to Assigned
  • Assignee set to Ivan Necas

I've renamded the subject of this issue to refer to the `with_indifferent_access` issue, which is something we need to fix in the foreman-tasks. The undefined method is caused by chef plugin, and we will fix it there in separate issue.

Actions #11

Updated by The Foreman Bot over 5 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman-tasks/pull/359 added
Actions #12

Updated by Björn Zettergren over 5 years ago

Tested the PR and it works for me. I don't get the "with_indifferent_access" error with the patch.

Actions #13

Updated by Ivan Necas over 5 years ago

  • Related to Bug #24888: NoMethodError when importing facts from host added
Actions #14

Updated by Ivan Necas over 5 years ago

  • Status changed from Ready For Testing to Closed
Actions #15

Updated by Ivan Necas over 5 years ago

  • Fixed in Releases foreman-tasks-0.14.1 (Foreman 1.20) added
Actions #16

Updated by Nathan Zachary 2 months ago

  • Related to Bug #37234: Template import fails in Foreman 3.9.1 added
Actions

Also available in: Atom PDF