Foreman » Installer

Is this the same as #2976 do you think? Does the host show the error, but the report metrics don't?

The host has a green zero and "No changes", so it's not set into error state...
I recall that it worked with Puppet 2.7 and the behaviour changed with puppet 3.x (but I can't recall if it still worked with 2.7 clients against a 3.x master).
Also, I just was tinnkering around and the error does correctly set the error state, wenn I do a puppet run with "puppet agent --test" on the shell of such a client, but if the normal 30 minute puppet runs as daemon run into such an error, it does not set the error state.

The difference between daemon and --test mode might be "usecacheonfailure" which is enabled by default I think and disabled with --test. It might be in daemon mode that it's falling back to a copy of the old catalog.

ah, right... if the case of the non-cached catalog I also have the error "err Puppet Could not retrieve catalog; skipping run"

Puppet itself doesn't include these types of errors in it's summary anymore, that seems to be the root cause:

--- !ruby/object:Puppet::Transaction::Report
  metrics: 
    resources: !ruby/object:Puppet::Util::Metric
      name: resources
      label: Resources
      values: 
        - - total
          - Total
          - 73
        - - skipped
          - Skipped
          - 0
        - - failed
          - Failed
          - 0
        - - failed_to_restart
          - "Failed to restart" 
          - 0
        - - restarted
          - Restarted
          - 0
        - - changed
          - Changed
          - 0
        - - out_of_sync
          - "Out of sync" 
          - 0
        - - scheduled
          - Scheduled
          - 0

  logs: 
    - !ruby/object:Puppet::Util::Log
      level: !ruby/sym err
      tags: 
        - err
      message: "Could not retrieve catalog from remote server: Error 400 on SERVER: Could not parse for environment production: Syntax error at 'mcollective::server::setting'; expected '}' at /etc/puppet/manifests/site.pp:35 on node xxx" 
      source: Puppet
      time: 2014-01-05 22:49:11.379313 +01:00
    - !ruby/object:Puppet::Util::Log
      level: !ruby/sym notice
      tags: 
        - notice
      message: "Using cached catalog" 
      source: Puppet
      time: 2014-01-05 22:49:16.229270 +01:00
    - !ruby/object:Puppet::Util::Log
      level: !ruby/sym notice
      tags: 
        - notice
      message: "Finished catalog run in 10.71 seconds" 
      source: Puppet
      time: 2014-01-05 22:49:29.766312 +01:00

This seems to be the relevant epic on the Puppet side: https://tickets.puppetlabs.com/browse/PUP-283

Since this seems to have low priority on the Puppet end (some of the relevant issues are targetted for 4.x), is there anything that can be done on the Foreman side of things? It's a little user unfriendly to see hosts with no activity, that when you click into their reports you see compilation and other errors.

A simple implementation could be to alter the Foreman report processor to add its own Foreman-internal flag into the report if it detects 'error' log lines in the report. The Foreman UI could then pick up on this flag and display it. (Though I'm not sure what the precedent is for altering a report to add custom data)