Project

General

Profile

Upgrading from Puppet 3 to 4 » History » Version 18

Martin Schurz, 12/12/2016 11:09 AM

1 1 Dominic Cleal
h1. Upgrading from Puppet 3 to 4
2
3 7 Dominic Cleal
_This wiki page is a work in progress for the release of Foreman 1.12 and should be used +at your own risk+._
4 1 Dominic Cleal
5 8 Dominic Cleal
{{toc}}
6
7 1 Dominic Cleal
This wiki page is a rough guide on how to upgrade from Puppet 3 to Puppet 4 when using Foreman. It doesn't replace Puppet's own documentation - you need to take both into account, and it's not for the faint-hearted.
8
9
Remember, Puppet is a separate piece of software to Foreman. Foreman integrates with Puppet in only a few places (e.g. reports, ENC and smart proxy class imports) but the Foreman installer may have set it up for you initially. Most of the work is changing the Puppet installation and then updating paths and configs in Foreman to suit.
10
11
If you're unfamiliar with how Puppet works, then you should consider setting up a new installation and migrating hosts instead.
12
13
h2. Planning
14
15
Ensure you are running Foreman 1.12 or higher, previous versions are not compatible with Puppet 4, while 1.12 is compatible with both 3 and 4, so carry out that upgrade first. See "Upgrading to Foreman 1.12":http://theforeman.org/manuals/1.12/index.html#3.6Upgrade for more information.
16
17
*Start with "Puppet 3.x to 4.x: Get upgrade-ready":https://docs.puppet.com/puppet/latest/reference/upgrade_major_pre.html* - it has many excellent points, including:
18
19
* As with any upgrade, the smaller the step, the easier it will be. Ensure you've already upgraded to the latest 3.x release and fixed any deprecations from the release notes before moving to 4. This will save time later.
20
* Ensure your Puppet modules are going to be compatible with Puppet 4's new ("future") parser
21
* Plan to upgrade your masters before your agents, because the master can serve older agents, but not the other way around.
22
* Back up everything, especially SSL keys and certificates.
23 15 Anonymous
* Ensure you have enough RAM, Puppet Server defaults to requiring at _least_ 2GB (or set @--puppet-server-max-active-instances=1 --puppet-server-jvm-min-heap-size=1G --puppet-server-jvm-max-heap-size=1G@ to reduce it)
24 1 Dominic Cleal
25
The guide will assume you're using regular 'puppet' packages either from your OS (or EPEL) or from Puppet Labs repositories. Puppet 4 packages are All-In-One (AIO) packages and work quite differently, introducing lots of new paths for config files and binaries. More information on these at:
26
27
* "Welcome to Puppet Collections":https://puppet.com/blog/welcome-to-puppet-collections
28
* "About Puppet collections and packages":https://docs.puppet.com/puppet/latest/reference/puppet_collections.html
29 2 Dominic Cleal
30 10 Dominic Cleal
h2. 1. Beginning the upgrade
31 1 Dominic Cleal
32 10 Dominic Cleal
These steps should be carried out whether using the installer or updating by hand.
33 2 Dominic Cleal
34 10 Dominic Cleal
h3. 1a. Install new PC1 packages
35
36 1 Dominic Cleal
# Configure the new PC1 repositories with the "Using Puppet Collections":https://docs.puppet.com/guides/puppetlabs_package_repositories.html#using-puppet-collections instructions.
37 11 Dominic Cleal
38
On EL:
39
40
# run @yum remove puppet-server@ to prevent later conflicts
41
# run @yum install puppetserver@ which should replace @facter@, @puppet@ and @puppet-server@ with @puppetserver@ and @puppet-agent@
42
43
On Debian/Ubuntu:
44
45
# run @apt-get install puppetserver puppet-agent@ which should  @facter@, @puppet@ and @puppetmaster@ with @puppetserver@ and @puppet-agent@
46 2 Dominic Cleal
47 10 Dominic Cleal
h3. 1b. Environments, SSL and Apache
48 2 Dominic Cleal
49 8 Dominic Cleal
This section is based on "Puppet 3.x to 4.x: Upgrade Puppet Server":https://docs.puppet.com/puppet/latest/reference/upgrade_major_server.html which goes into far more detail.
50
51 2 Dominic Cleal
# Move or copy any environments from @/etc/puppet/environments@ to @/etc/puppetlabs/code/environments@
52
# Move or copy all SSL keys and certificates from @/var/lib/puppet/ssl@ to @/etc/puppetlabs/puppet/ssl@
53 1 Dominic Cleal
# Remove the Puppet master VirtualHost from Apache at @/etc/httpd/conf.d/25-puppet.conf@ (EL) or @a2dissite 25-puppet@ (Debian/Ubuntu)
54
# Remove 8140 from the Apache ports in @/etc/httpd/conf/ports.conf@ or @/etc/apache2/ports.conf@
55
# Update SSL paths in @/etc/httpd/conf.d/05-foreman-ssl.conf@ or @/etc/apache2/sites-available/05-foreman-ssl.conf@, changing @/var/lib/puppet/ssl@ to @/etc/puppetlabs/puppet/ssl@
56
# Restart httpd/apache2 to free up the port
57
58 10 Dominic Cleal
h3. 1c. Choose foreman-installer or by hand
59 1 Dominic Cleal
60 10 Dominic Cleal
Choose _one_ of the two following sections - either update all configuration files using foreman-installer or by hand.
61
62
h2. 2. Upgrading with foreman-installer
63
64
If your installation is fairly standard, re-running the Foreman installer is the most reliable way to install and configure Puppet Server correctly. All parameters containing paths that change based on an AIO/non-AIO layout will be reset back to their defaults, which will match the paths files were moved to in the previous section.
65
66 12 Dominic Cleal
First install the @puppet-agent-oauth@ package to prevent warnings running in no-op mode.
67
68 10 Dominic Cleal
<pre>
69
foreman-installer --noop -v \
70
  --puppet-server-implementation=puppetserver \
71
  --reset-foreman-client-ssl-ca \
72
  --reset-foreman-client-ssl-cert \
73
  --reset-foreman-client-ssl-key \
74
  --reset-foreman-puppet-home \
75
  --reset-foreman-puppet-ssldir \
76
  --reset-foreman-server-ssl-ca \
77
  --reset-foreman-server-ssl-cert \
78
  --reset-foreman-server-ssl-chain \
79
  --reset-foreman-server-ssl-crl \
80
  --reset-foreman-server-ssl-key \
81
  --reset-foreman-websockets-ssl-cert \
82
  --reset-foreman-websockets-ssl-key \
83
  --reset-foreman-proxy-puppet-ssl-ca \
84
  --reset-foreman-proxy-puppet-ssl-cert \
85
  --reset-foreman-proxy-puppet-ssl-key \
86 16 Dominic Cleal
  --reset-foreman-proxy-puppetca-cmd \
87 10 Dominic Cleal
  --reset-foreman-proxy-puppetdir \
88
  --reset-foreman-proxy-ssl-ca \
89
  --reset-foreman-proxy-ssl-cert \
90
  --reset-foreman-proxy-ssl-key \
91
  --reset-foreman-proxy-ssldir \
92
  --reset-foreman-puppet-home \
93
  --reset-puppet-autosign \
94 13 Dominic Cleal
  --reset-puppet-client-package \
95 10 Dominic Cleal
  --reset-puppet-codedir \
96
  --reset-puppet-configtimeout \
97
  --reset-puppet-dir \
98
  --reset-puppet-logdir \
99
  --reset-puppet-rundir \
100
  --reset-puppet-ssldir \
101
  --reset-puppet-vardir \
102
  --reset-puppet-server-common-modules-path \
103 12 Dominic Cleal
  --reset-puppet-server-default-manifest-path \
104 10 Dominic Cleal
  --reset-puppet-server-dir \
105
  --reset-puppet-server-envs-dir \
106
  --reset-puppet-server-external-nodes \
107
  --reset-puppet-server-jruby-gem-home \
108 14 Dominic Cleal
  --reset-puppet-server-package \
109 10 Dominic Cleal
  --reset-puppet-server-puppetserver-dir \
110 18 Martin Schurz
  --reset-puppet-server-puppetserver-vardir \
111 10 Dominic Cleal
  --reset-puppet-server-ruby-load-paths \
112
  --reset-puppet-server-ssl-dir
113
</pre>
114
115
* Note: in noop mode, you can expect an error of @Could not find a suitable provider for foreman_smartproxy@ to be logged, but this should be fixed on the live run, or install the @puppet-agent-oauth@ package manually.
116
117
Keep the @--noop -v@ flags on the first run to check if there are any unexpected changes, then remove it to perform the actual changes.
118
119
h2. 3. Upgrading by hand
120
121
If you have many manual customisations to your Foreman or Puppet installations that you wish to preserve, then you may wish to update config files by hand for new paths.
122
123
h3. 3a. Puppet Config files
124
125 3 Dominic Cleal
# @mv /etc/puppet/autosign.conf /etc/puppetlabs/puppet/@
126
# @cp /etc/puppet/puppet.conf /etc/puppetlabs/puppet/puppet.conf@ and change:
127
#* in the 'main' section:
128 6 Dominic Cleal
#*# @vardir = /opt/puppetlabs/puppet/cache@
129 1 Dominic Cleal
#*# @logdir = /var/log/puppetlabs/puppet@
130 6 Dominic Cleal
#*# @rundir = /var/run/puppetlabs@
131
#*# @ssldir = /etc/puppetlabs/puppet/ssl@
132 3 Dominic Cleal
#*# @environmentpath = /etc/puppetlabs/code@
133
#*# @basemodulepath = /etc/puppetlabs/code/environments/common:/etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules@
134 6 Dominic Cleal
#* in the 'agent' section:
135
#*# remove @configtimeout@
136
#* in the 'master' section:
137
#*# @autosign = /etc/puppetlabs/puppet/autosign.conf { mode = 0644 }@
138
#*# @external_nodes = /etc/puppetlabs/puppet/node.rb@
139
#*# @ssldir = /etc/puppetlabs/puppet/ssl@
140
# edit @/etc/puppetlabs/puppetserver/conf.d/puppetserver.conf@
141
## change @master-var-dir@ to @/opt/puppetlabs/puppet/cache@
142
## uncomment/set @use-legacy-auth-conf: false@ 
143
144
If using a smart proxy to import classes, edit @/etc/puppetlabs/puppetserver/conf.d/auth.conf@, search for @/puppet/v3/environments@ and add a new section below it:
145
146
<pre>
147
        {
148
            match-request: {
149
                path: "/puppet/v3/resource_type"
150
                type: path
151
                method: [get, post]
152
            }
153
            allow: "*"
154
            sort-order: 500
155 1 Dominic Cleal
            name: "puppetlabs resource type"
156 6 Dominic Cleal
        },
157
</pre> 
158
159
If you will still support Puppet 3 clients against the server running Puppet 4, see "auth.conf rules for Puppet 3 and 4 agents":https://docs.puppet.com/puppet/latest/reference/upgrade_major_server.html#example-authconf-rules-for-puppet-3-and-4-agents for additional rules. Using foreman-installer (below) will also configure these by default.
160
161 1 Dominic Cleal
Start and enable the puppetserver service with: @/opt/puppetlabs/bin/puppet resource service puppetserver ensure=running enable=true@
162 6 Dominic Cleal
163 10 Dominic Cleal
h3. 3b. ENC files
164 6 Dominic Cleal
165
# @cp /etc/puppet/foreman.yaml /etc/puppetlabs/puppet/foreman.yaml@ and change:
166
## replace @/var/lib/puppet/ssl@ with @/etc/puppetlabs/puppet/ssl@
167 1 Dominic Cleal
## @:puppetdir: /opt/puppetlabs/puppet/cache@
168 6 Dominic Cleal
# @mv /etc/puppet/node.rb /etc/puppetlabs/puppet/@
169 9 Dominic Cleal
170 10 Dominic Cleal
h3. 3c. Foreman settings
171 6 Dominic Cleal
172
# edit @/etc/foreman/settings.yaml@ and change @:puppetssldir: /etc/puppetlabs/puppet/ssl@
173
# change @websockets_*@ settings to use @/etc/puppetlabs/puppet/ssl@ and also @ssl_*@ if specified
174
# restart Foreman by running @touch ~foreman/tmp/restart.txt@
175
# check in _Administer > Settings > Auth_ in the Foreman UI that SSL certificate, private key and CA file all use @/etc/puppetlabs/puppet/ssl@, else change them
176
177 10 Dominic Cleal
h3. 3d. Smart proxy settings
178 6 Dominic Cleal
179
# edit @/etc/foreman-proxy/settings.d/puppet.yml@ and set @:puppet_version@ to the version of Puppet currently installed, e.g. 4.5.0
180
#* look up the version of puppet-agent (@rpm -q puppet-agent@ or @dpkg -l puppet-agent@) and check "Release contents":https://docs.puppet.com/puppet/4.5/reference/about_agent.html
181
# edit @/etc/foreman-proxy/settings.d/puppet_proxy_puppet_api.yml@ and change @/var/lib/puppet/ssl@ to @/etc/puppetlabs/puppet/ssl@
182
# edit @/etc/foreman-proxy/settings.d/puppetca.yml@ and change:
183
## @:ssldir: /etc/puppetlabs/puppet/ssl@
184
## @:puppetdir: /etc/puppetlabs/puppet@
185
# edit @/etc/foreman-proxy/settings.yml@ and change @/var/lib/puppet/ssl@ to @/etc/puppetlabs/puppet/ssl@
186 17 Dominic Cleal
# @service foreman-proxy restart@
187 6 Dominic Cleal
188 10 Dominic Cleal
h3. 3e. Foreman installer
189 6 Dominic Cleal
190 10 Dominic Cleal
Foreman installer stores many paths in its answers file, so if you plan to run it again in future, these need to be changed. See the earlier section on upgrading with Foreman installer to update them.
191 3 Dominic Cleal
192
h2. Further reading
193
194
* "Puppet reference manual":https://docs.puppet.com/puppet/latest/reference/index.html
195
* "Puppet Server documentation":https://docs.puppet.com/puppetserver/latest/
196
* "Puppet Server vs Apache/Passenger Puppet master":https://docs.puppet.com/puppetserver/latest/puppetserver_vs_passenger.html
197
* "Where did everything go in Puppet 4.x?":https://docs.puppet.com/puppet/latest/reference/whered_it_go.html
198
* "puppet-agent: What is it, and what's in it?":https://docs.puppet.com/puppet/4.5/reference/about_agent.html