h1. Upgrading from Puppet 3 to 4

_This wiki page is a work in progress for the release of Foreman 1.12 and should be used +at your own risk+._

{{toc}}

This wiki page is a rough guide on how to upgrade from Puppet 3 to Puppet 4 when using Foreman. It doesn't replace Puppet's own documentation - you need to take both into account, and it's not for the faint-hearted.

Remember, Puppet is a separate piece of software to Foreman. Foreman integrates with Puppet in only a few places (e.g. reports, ENC and smart proxy class imports) but the Foreman installer may have set it up for you initially. Most of the work is changing the Puppet installation and then updating paths and configs in Foreman to suit.

If you're unfamiliar with how Puppet works, then you should consider setting up a new installation and migrating hosts instead.

h2. Planning

Ensure you are running Foreman 1.12 or higher, previous versions are not compatible with Puppet 4, while 1.12 is compatible with both 3 and 4, so carry out that upgrade first. See "Upgrading to Foreman 1.12":http://theforeman.org/manuals/1.12/index.html#3.6Upgrade for more information.

*Start with "Puppet 3.x to 4.x: Get upgrade-ready":https://docs.puppet.com/puppet/latest/reference/upgrade_major_pre.html* - it has many excellent points, including:

* As with any upgrade, the smaller the step, the easier it will be. Ensure you've already upgraded to the latest 3.x release and fixed any deprecations from the release notes before moving to 4. This will save time later.

* Ensure your Puppet modules are going to be compatible with Puppet 4's new ("future") parser

* Plan to upgrade your masters before your agents, because the master can serve older agents, but not the other way around.

* Back up everything, especially SSL keys and certificates.

* Ensure you have enough RAM, Puppet Server defaults to requiring at _least_ 2GB

The guide will assume you're using regular 'puppet' packages either from your OS (or EPEL) or from Puppet Labs repositories. Puppet 4 packages are All-In-One (AIO) packages and work quite differently, introducing lots of new paths for config files and binaries. More information on these at:

* "Welcome to Puppet Collections":https://puppet.com/blog/welcome-to-puppet-collections

* "About Puppet collections and packages":https://docs.puppet.com/puppet/latest/reference/puppet_collections.html

h2. Upgrading

h3. Install new PC1 packages

# Configure the new PC1 repositories with the "Using Puppet Collections":https://docs.puppet.com/guides/puppetlabs_package_repositories.html#using-puppet-collections instructions.

# On EL, run @yum remove puppet-server@ to prevent later conflicts.

# Install the @puppetserver@ package, which should replace @facter@, @puppet@ and @puppet-server@ with @puppetserver@ and @puppet-agent@

h3. Move configs and files to new structure

This section is based on "Puppet 3.x to 4.x: Upgrade Puppet Server":https://docs.puppet.com/puppet/latest/reference/upgrade_major_server.html which goes into far more detail.

h4. Environments, SSL and Apache

# Move or copy any environments from @/etc/puppet/environments@ to @/etc/puppetlabs/code/environments@

# Move or copy all SSL keys and certificates from @/var/lib/puppet/ssl@ to @/etc/puppetlabs/puppet/ssl@

# Remove the Puppet master VirtualHost from Apache at @/etc/httpd/conf.d/25-puppet.conf@ (EL) or @a2dissite 25-puppet@ (Debian/Ubuntu)

# Remove 8140 from the Apache ports in @/etc/httpd/conf/ports.conf@ or @/etc/apache2/ports.conf@

# Update SSL paths in @/etc/httpd/conf.d/05-foreman-ssl.conf@ or @/etc/apache2/sites-available/05-foreman-ssl.conf@, changing @/var/lib/puppet/ssl@ to @/etc/puppetlabs/puppet/ssl@

# Restart httpd/apache2 to free up the port

h4. Config files

# @mv /etc/puppet/autosign.conf /etc/puppetlabs/puppet/@

# @cp /etc/puppet/puppet.conf /etc/puppetlabs/puppet/puppet.conf@ and change:

#* in the 'main' section:

#*# @vardir = /opt/puppetlabs/puppet/cache@

#*# @logdir = /var/log/puppetlabs/puppet@

#*# @rundir = /var/run/puppetlabs@

#*# @ssldir = /etc/puppetlabs/puppet/ssl@

#*# @environmentpath = /etc/puppetlabs/code@

#*# @basemodulepath = /etc/puppetlabs/code/environments/common:/etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules@

#* in the 'agent' section:

#*# remove @configtimeout@

#* in the 'master' section:

#*# @autosign = /etc/puppetlabs/puppet/autosign.conf { mode = 0644 }@

#*# @external_nodes = /etc/puppetlabs/puppet/node.rb@

#*# @ssldir = /etc/puppetlabs/puppet/ssl@

# edit @/etc/puppetlabs/puppetserver/conf.d/puppetserver.conf@

## change @master-var-dir@ to @/opt/puppetlabs/puppet/cache@

## uncomment/set @use-legacy-auth-conf: false@ 

If using a smart proxy to import classes, edit @/etc/puppetlabs/puppetserver/conf.d/auth.conf@, search for @/puppet/v3/environments@ and add a new section below it:

<pre>

        {

            match-request: {

                path: "/puppet/v3/resource_type"

                type: path

                method: [get, post]

            }

            allow: "*"

            sort-order: 500

            name: "puppetlabs resource type"

        },

</pre> 

If you will still support Puppet 3 clients against the server running Puppet 4, see "auth.conf rules for Puppet 3 and 4 agents":https://docs.puppet.com/puppet/latest/reference/upgrade_major_server.html#example-authconf-rules-for-puppet-3-and-4-agents for additional rules. Using foreman-installer (below) will also configure these by default.

Start and enable the puppetserver service with: @/opt/puppetlabs/bin/puppet resource service puppetserver ensure=running enable=true@

h4. ENC files

# @cp /etc/puppet/foreman.yaml /etc/puppetlabs/puppet/foreman.yaml@ and change:

## replace @/var/lib/puppet/ssl@ with @/etc/puppetlabs/puppet/ssl@

## @:puppetdir: /opt/puppetlabs/puppet/cache@

# @mv /etc/puppet/node.rb /etc/puppetlabs/puppet/@

h4. Foreman settings

# edit @/etc/foreman/settings.yaml@ and change @:puppetssldir: /etc/puppetlabs/puppet/ssl@

# change @websockets_*@ settings to use @/etc/puppetlabs/puppet/ssl@ and also @ssl_*@ if specified

# restart Foreman by running @touch ~foreman/tmp/restart.txt@

# check in _Administer > Settings > Auth_ in the Foreman UI that SSL certificate, private key and CA file all use @/etc/puppetlabs/puppet/ssl@, else change them

h4. Smart proxy settings

# edit @/etc/foreman-proxy/settings.d/puppet.yml@ and set @:puppet_version@ to the version of Puppet currently installed, e.g. 4.5.0

#* look up the version of puppet-agent (@rpm -q puppet-agent@ or @dpkg -l puppet-agent@) and check "Release contents":https://docs.puppet.com/puppet/4.5/reference/about_agent.html

# edit @/etc/foreman-proxy/settings.d/puppet_proxy_puppet_api.yml@ and change @/var/lib/puppet/ssl@ to @/etc/puppetlabs/puppet/ssl@

# edit @/etc/foreman-proxy/settings.d/puppetca.yml@ and change:

## @:ssldir: /etc/puppetlabs/puppet/ssl@

## @:puppetdir: /etc/puppetlabs/puppet@

# edit @/etc/foreman-proxy/settings.yml@ and change @/var/lib/puppet/ssl@ to @/etc/puppetlabs/puppet/ssl@

# restart foreman-proxy

h2. Re-running foreman-installer

Foreman installer stores many paths in its answers file, so before it's safe to re-run it, these need to be changed.

_This section relies on #15071, currently unreleased._

You can either reset all of the affected parameters to their defaults and let them be recalculated using the AIO Puppet agent, or edit the answers file at @/etc/foreman-installer/scenarios.d/foreman-answers.yaml@. To reset them, run:

<pre>

foreman-installer --noop -v \

  --puppet-server-implementation=puppetserver \

  --reset-foreman-client-ssl-ca \

  --reset-foreman-client-ssl-cert \

  --reset-foreman-client-ssl-key \

  --reset-foreman-puppet-home \

  --reset-foreman-puppet-ssldir \

  --reset-foreman-server-ssl-ca \

  --reset-foreman-server-ssl-cert \

  --reset-foreman-server-ssl-chain \

  --reset-foreman-server-ssl-crl \

  --reset-foreman-server-ssl-key \

  --reset-foreman-websockets-ssl-cert \

  --reset-foreman-websockets-ssl-key \

  --reset-foreman-proxy-puppet-ssl-ca \

  --reset-foreman-proxy-puppet-ssl-cert \

  --reset-foreman-proxy-puppet-ssl-key \

  --reset-foreman-proxy-puppetdir \

  --reset-foreman-proxy-ssl-ca \

  --reset-foreman-proxy-ssl-cert \

  --reset-foreman-proxy-ssl-key \

  --reset-foreman-proxy-ssldir \

  --reset-puppet-autosign \

  --reset-puppet-codedir \

  --reset-puppet-configtimeout \

  --reset-puppet-dir \

  --reset-puppet-logdir \

  --reset-puppet-rundir \

  --reset-puppet-ssldir \

  --reset-puppet-vardir \

  --reset-puppet-server-common-modules-path \

  --reset-puppet-server-dir \

  --reset-puppet-server-envs-dir \

  --reset-puppet-server-external-nodes \

  --reset-puppet-server-jruby-gem-home \

  --reset-puppet-server-manifest-path \

  --reset-puppet-server-puppetserver-dir \

  --reset-puppet-server-ruby-load-paths \

  --reset-puppet-server-ssl-dir

</pre>

Keep the @--noop -v@ flags on the first run to check if there are any unexpected changes, then remove it to perform the actual changes. Note that there will be many more small changes to the contents of config files (particularly around Puppet Server) that the installer will change, but which shouldn't affect the operation.

h2. Further reading

* "Puppet reference manual":https://docs.puppet.com/puppet/latest/reference/index.html

* "Puppet Server documentation":https://docs.puppet.com/puppetserver/latest/

* "Puppet Server vs Apache/Passenger Puppet master":https://docs.puppet.com/puppetserver/latest/puppetserver_vs_passenger.html

* "Where did everything go in Puppet 4.x?":https://docs.puppet.com/puppet/latest/reference/whered_it_go.html

* "puppet-agent: What is it, and what's in it?":https://docs.puppet.com/puppet/4.5/reference/about_agent.html