Project

General

Profile

Fetch boot files via http instead of TFTP » History » Revision 20

Revision 19 (Lukas Zapletal, 06/22/2018 07:28 AM) → Revision 20/26 (Lukas Zapletal, 07/02/2018 08:04 AM)

h1. PXELinux chainboot into iPXE 

 {{toc}} 

 TFTP is a slow protocol on high-latency networks, but if your hardware is supported by iPXE (http://ipxe.org/appnote/hardware_drivers) or if UNDI driver of the NIC is compatible with iPXE, it is possible to configure PXELinux to chainboot iPXE and continue booting via HTTP protocol which is fast and reliable. 

 In this scenario, a system is PXE-booted into PXELinux which chainloads iPXE which continue booting via HTTP. The scenario is: 

 * hardware is turned on 
 * PXE driver gets network credentials from DHCP 
 * PXE driver gets PXELinux firmware from TFTP (pxelinux.0) 
 * PXELinux searches for configuration file on TFTP 
 * PXELinux chainloads iPXE (undionly-ipxe.0 or ipxe.lkrn) 
 * iPXE gets network credentials from DHCP again 
 * iPXE gets HTTP address from DHCP 
 * iPXE chainloads the iPXE template from Foreman 
 * iPXE loads kernel and init RAM disk of the installer 

 Requirements: 

 * a host entry is created in Foreman 
 * MAC address of the provisioning interface matches 
 * provisioning interface of the host has a valid DHCP reservation 
 * the host has special PXELinux template (below) associated 
 * the host has iPXE template associated 
 * hardware is capable of PXE booting 
 * hardware NIC is compatible with iPXE 

 The iPXE project offers two options: using PXE interface (UNDI) or using built-in linux network card driver. Both options have pros and cons and each gives different results with different hardware cards. Some NIC adapters can be slow with UNDI, some are actually faster. Not all network cards will work with either or both ways. 

 h2. A. Chainbooting iPXE directly 

 In this setup, iPXE uses build-in driver for network communication. Therefore this will only work on supported cards (see above)! 

 h3. TFTP setup 

 Copy the iPXE firmware to the TFTP root directory: 

   cp /usr/share/ipxe/ipxe.lkrn /var/lib/tftpboot/ 

 The source directory can be different on linux distributions, this is for Red Hats. The file is shipped in ipxe-bootimgs package. 

 Not all hardware is supported by iPXE drivers. In case of troubles, use latest development version build of iPXE: 

   wget -O /var/lib/tftpboot/ http://boot.ipxe.org/ipxe.lkrn 

 Do not use symbolic links as TFTP runs in chroot. When using SELinux, remember to correct file contexts: 

   restorecon -RvF /var/lib/tftpboot/ 

 h3. Foreman setup - PXELinux template 

 Configuration involves associating PXELinux and iPXE templates. 

 In your Foreman instance, go to "Provisioning templates" and create new template of PXELinux kind with the following contents: 

 <pre><code> 
 DEFAULT linux 
 LABEL linux 
 KERNEL ipxe.lkrn 
 APPEND dhcp && chain <%= foreman_url('iPXE') %> 
 IPAPPEND 2 
 </code></pre> 

 Recent version of Foreman (1.11) ships with this template already under name "PXELinux chain iPXE". 

 h3. Foreman setup - iPXE template 

 Associate iPXE template which ships with Foreman which is named 'Kickstart default iPXE' or 'Preseed default iPXE' containing something like: 

 <pre><code>#!ipxe 
 kernel <%= "#{@host.url_for_boot(:kernel)}" %> ks=<%= foreman_url("provision")%> 
 initrd <%= "#{@host.url_for_boot(:initrd)}" %> 
 boot 
 </code></pre> 

 If there was a host associated with PXELinux templates, you may need to exit and re-enter Build state for the TFTP configuration to be redeployed. Recent versions of Foreman (1.11) do this automatically on template save. 

 h2. B. Chainbooting iPXE via UNDI 

 In this setup, iPXE uses UNDI for network communication. The hardware must support that. 

 h3. TFTP setup 

 Copy the iPXE firmware to the TFTP root directory and rename it: 

   cp /usr/share/ipxe/undionly.kpxe /var/lib/tftpboot/undionly-ipxe.0 

 The source directory can be different on linux distributions, this is for Red Hats. The file is shipped in ipxe-bootimgs package. 

 Do not use symbolic links as TFTP runs in chroot. When using SELinux, remember to correct file contexts: 

   restorecon -RvF /var/lib/tftpboot/ 

 h3. TFTP setup (gPXELinux alternative) 

 This is alternative approach if none of the above configurations work or packages are not available. 

 Copy the gPXE firmware to the TFTP root directory: 

   cp /usr/share/syslinux/gpxelinuxk.0 /var/lib/tftpboot/ 

 The source directory can be different on linux distributions, this is for Red Hats. The file is shipped in syslinux package. 

 Do not use symbolic links as TFTP runs in chroot. When using SELinux, remember to correct file contexts: 

   restorecon -RvF /var/lib/tftpboot/ 

 h3. Foreman setup - PXELinux template 

 In your Foreman instance, go to "Provisioning templates" and create new template of PXELinux kind with the following contents: 

 <pre><code> 
 DEFAULT undionly-ipxe 
 LABEL undionly-ipxe 
 MENU LABEL iPXE UNDI 
 KERNEL undionly-ipxe.0 
 IPAPPEND 2 
 </code></pre> 

 Recent version of Foreman (1.11) ships with this template already under name "PXELinux chain iPXE UNDI". 

 h3. Foreman setup - iPXE template 

 Associate iPXE template which ships with Foreman which is named 'Kickstart default iPXE' or 'Preseed default iPXE' containing something like: 

 <pre><code>#!ipxe 
 kernel <%= "#{@host.url_for_boot(:kernel)}" %> ks=<%= foreman_url("provision")%> 
 initrd <%= "#{@host.url_for_boot(:initrd)}" %> 
 boot 
 </code></pre> 

 If there was a host associated with PXELinux templates, you may need to exit and re-enter Build state for the TFTP configuration to be redeployed. Recent versions of Foreman (1.11) do this automatically on template save. 

 h3. DHCP setup 

 The above configuration will lead to an endless loop of chainbooting iPXE firmware. To break this loop, configure DHCP server to hand over correct URL to iPXE to continue booting. In the /etc/dhcp/dhcpd.conf file change the "filename" global or subnet configuration as follows: 

 <pre><code> 
 if exists user-class and option user-class = "iPXE" { 
   filename "http://foreman/unattended/iPXE"; "http://foreman:443/unattended/iPXE"; 
 } else { 
   filename "pxelinux.0"; 
 } 
 </code></pre> 

 On isolated networks, use Smart Proxy URL instead of Foreman when templates feature is enabled. If there are existing leases on the DHCP server, let them expire and restart the DHCP service. This can be also forced with 

 <pre><code> 
 truncate /var/lib/dhcpd/dhcpd.leases 
 service dhcpd restart 
 </code></pre> 

 h2. C. Chainbooting virtual machines 

 Since most virtualization hypervisors use iPXE as the primary firmware for PXE booting, the above configuration will directly work without TFTP and PXELinux involved. This is known to work with libvirt, oVirt and RHEV. If the hypervisor is capable of replacing PXE firmware, it will work too (e.g. VMWare is documented at http://ipxe.org/howto/vmware). 

 This workflow will only work if *tokens are disabled*, to do that go to Administer - Settings and set "Token duration" to 0. There are two options. 

 h3. Foreman setup - iPXE template 

 Associate iPXE template which ships with Foreman which is named 'Kickstart default iPXE' or 'Preseed default iPXE'. The contents is the same as in the workflows above. 

 h3. DHCP setup - managed server 

 In this case, Foreman manages DHCP and reservation (IP address) is known in advance, therefore iPXE script can be found in Foreman DB and returned to a client: 

 * VM is turned on 
 * iPXE gets network credentials from DHCP 
 * iPXE gets filename option from DHCP (URL) 
 * iPXE chainloads the iPXE template from Foreman 
 * Foreman renders iPXE template for given host found by remote IP 
 * iPXE executed the script, loads kernel and init RAM disk of the installer 

 In the /etc/dhcp/dhcpd.conf file change the "filename" global or subnet configuration as follows: 

 <pre><code> 
 if exists user-class and option user-class = "iPXE" { 
   filename "http://foreman/unattended/iPXE"; "http://foreman:443/unattended/iPXE"; 
 } else { 
   filename "pxelinux.0"; 
 } 
 </code></pre> 

 Certain iPXE builds might not be able to access via SSL/TLS, in that case use HTTP URL. On isolated networks, use Smart Proxy URL instead of Foreman but keep in mind templates smart-proxy feature must be enabled. If there are existing leases on the DHCP server, let them expire and restart the DHCP service (see above). 

 h3. DHCP setup - unmanaged server 

 In this case, Foreman cannot create DHCP reservation in advance. But an intermediate iPXE script can be deployed to report MAC address to find proper host: 

 * VM is turned on 
 * iPXE gets network credentials from DHCP 
 * iPXE gets filename option from DHCP (URL) 
 * iPXE loads the intermediate iPXE template from a HTTP server 
 * iPXE executes the intermediate script 
 * iPXE chainloads the iPXE template from Foreman with MAC address provided as a parameter 
 * Foreman renders iPXE template for given host found by remote IP 
 * iPXE executed the script, loads kernel and init RAM disk of the installer 

 Create the following script and put it somewhere on the network via HTTP so iPXE clients can access it. 

 <pre><code> 
 #!ipxe 
 # Intermediate iPXE script to report MAC address to Foreman 
 isset ${net0/mac} || goto no_nic 
 dhcp net0 || goto net1 
 chain http://foreman/unattended/iPXE?mac=${net0/mac} http://foreman:443/unattended/iPXE?mac=${net0/mac} || goto net1 
 exit 0 

 :net1 
 isset ${net1/mac} || goto no_nic 
 dhcp net1 || goto net2 
 chain http://foreman/unattended/iPXE?mac=${net1/mac} http://foreman:443/unattended/iPXE?mac=${net1/mac} || goto net2 
 exit 0 

 :net1 
 # Create as many copies as necessary (this will work up to 2 NICs) 

 :no_nic 
 echo Failed to chainload from any network interface 
 sleep 30 
 exit 1 
 </code></pre> 

 Let's use httpd on Foreman server for this purpose: 

 <pre><code> 
 # scp intermediate.ipxe root@foreman:/var/www/htdocs/pub/ 
 </code></pre> 

 On an *unmanaged DHCP server* change filename option to be "http://foreman/pub/intermediate.ipxe". "http://foreman:443/pub/intermediate.ipxe". Instructions are different for various DHCP servers (like MS DHCP, Infoblox, Bluecoat) but if this was ISC DHCP, then simply change /etc/dhcp/dhcpd.conf file as follows: 

 <pre><code> 
 if exists user-class and option user-class = "iPXE" { 
   filename "http://foreman/pub/intermediate.ipxe"; "http://foreman:443/pub/intermediate.ipxe"; 
 } else { 
   filename "pxelinux.0"; 
 } 
 </code></pre> 

 Certain iPXE builds might not be able to access via SSL/TLS, in that case use HTTP URL. On isolated networks, use Smart Proxy URL instead of Foreman but keep in mind templates smart-proxy feature must be enabled. 

 Warning: Foreman 1.17 and older will not accept "mac" HTTP option, unless bootdisk plugin is installed, so make sure it is present. Foreman 1.18 already accepts the "mac" option even if bootdisk plugin is not installed.