This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

NetBoot

Most computers come with something called firmware. This is just software written semi-permanently on a chip in the PC. This is the first thing loaded at power-on. Modern versions include a GUI interface to change settings and test hardware. But it’s most important job is to load the main operating system from storage.

The main OS is normally loaded from a local disk, but it can also be loaded over the network. There are usually two goals;

  • Provision New Systems
  • Thin Clients or Disk-less Workstations

Provisioning New Systems

This saves you from carrying around a boot disk. But most importantly, it allows unattended deployment. You can remotely re-image workstations without a tech on-site. As great as this sounds, it’s not easy and requires solid user management to make it useful. So there hasn’t been wide adoption outside of line-of-business situations. Though it does make updating install sources easier.

Thin or Disk-less Stations

This seems like a good idea, and it works at scale. But the cost of a thin client is usually more than an equivalent PC so it’s mostly limited to line-of-business again. And the cost of storage is so cheap that a technical investment in netboot rarely pays off. But it can work when repurposing old PCs with worn out drives for kiosk use.

Boot Server

Either way, you’ll need a boot server

1 - Boot Server

NetBoot systems rely on the network DHCP (Dynamic Host Configuration Protocol) to request info on how to load an OS. So your first step is a DHCP server. Since you probably already have one, and would prefer not to break it while testing out netboot, let’s use a feature that allows the netboot server to work alongside your existing DHCP server; PXE Proxy

This server adds additional DHCP services and HTTP/TFTP related to PXE without interfering with your main IP allocation services.

Installation

dnsmasq supports both DHCP, for telling clients about itself, and TFTP, for getting the files to them. We’ll also add lighttpd for HTTP support. That protocol is much faster for systems that can use it.

sudo apt install dnsmasq lighttpd

Configuration

Server

Use a static IP and a hostname for the server that resolves correctly. We use the server name netboot.lan.

Lighttpd

No configuration is needed. It serves up content from /var/www/html folder by default.

Dnsmasq

When configured in proxy dhcp mode: “…dnsmasq simply provides the information given in –pxe-prompt and –pxe-service to allow netbooting”. So only certain settings are available. This is a bit vague, but testing reveals that you must set the boot file name with the dhcp-boot directive, rather than setting it with the more general DHCP option ID 67, for example.

# Add a file in the drop folder
sudo vi /etc/dnsmasq.d/netboot.conf 

# Disable DNS
port=0

# Set for DHCP PXE Proxy mode. It will only answer request from this range.
dhcp-range=192.168.1.0,proxy

# Respond to clients that use 'HTTPClient' or 'PXEClient' to identify themselves.
dhcp-pxe-vendor=PXEClient,HTTPClient 

# Send the BOOTP information for the clients using HTTP
dhcp-boot="http://netboot.lan/debian.iso" 

# Specify a boot menu option for PXE clients. If there is only one, it's booted immediately.
pxe-service=x86-64_EFI,"Network Boot"
pxe-service=x86-64_EFI,"Network Boot (UEFI)",boot/bootmgfw.efi
pxe-service=x86-64_EFI,"iPXE (UEFI)", "ipxe.efi"

# Enable TFTP for the PXE clients. 
enable-tftp 
tftp-root=/var/www/html

# Restart DNSMasq to enable
sudo systemctl restart dnsmasq.service

Installation Source

The simplest thing possible is to just drop an ISO on the web server. Take a look at the current debian ISO (the numbering changes) at https://www.debian.org/CD/netinst and download.

sudo wget  https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-13.1.0-amd64-netinst.iso -P /var/www/html -O debian.iso

Client

Clients may need to have the “Enable UEFI Network Stack” enabled in thier firmware setup. The debian boot loader is signed and works with secure boot.

Next Steps

You didn’t get any choices when booting. A menu with some options is a good thing to add next.

Troubleshooting

dnsmasq

A good way to see what’s going on is to enable dnsmasq logging.

# Add these to the dnsmasq config file
log-queries
log-dhcp

# Restart and follow to see what's happening
sudo systemctl restart dnsmasq.service
sudo journalctl -u dnsmasq -f

If you’ve enabled logging in dnsmasq and it’s not seeing any requests, you may need to look at your networking. Some virtual environments suppress DHCP broadcasts when they are managing the IP range. You can also have an issue with the client resolving DNS. Test with IPs if needed.

PXEClient:Arch:00007:UNDI:003016

If you see this the client isn’t attempting HTTPBoot. Dells specifically lack this feature even circa 2019

lighttpd

You can also see what’s being requested from the web server if you enable access logs.

cd /etc/lighttpd/conf-enabled
sudo ln -s ../conf-available/10-accesslog.conf
sudo systemctl restart lighttpd.service
sudo cat /var/log/lighttpd/access.log

Notes

In addition to ISOs, you can also specify .efi binaries like grubx64.efi. This allows you to extract the files and add a pre-seed. Some distributions support this, though Debian itself may have issues.

2 - menu

For a simple menu use the pxe-service built into dnsmasq.

Configuration

dnsmasq

Configure dnsmasq to serve up the ipxe.efi binary for both types of clients.

# add to the settings from the last example

pxe-service=X86-64_EFI,"Network Boot UEFI x86_64",grub/grubx64.efi

3 - netboot.xyz

You can add netboot.xyz to your iPXE menu to run Live CDs, OS installers and utilities they provide. This can save a lot of time and their list is always improving.

Installation

You’re going to connect to the web for this, so there’s nothing to install. You can download their efi bootloader manually if you’d like to keep things HTTPS, but they update it regularly so you may fall behind.

Configuration

Autoexec.ipxe

Add a menu item to your autoexec.ipxe. When you select it, iPXE will chainload (in their parlance) the netboot.xyz bootloader.

#!ipxe

echo ${cls}

:MAIN
menu Local Netboot Menu
item --gap Local Network Installation
item WINDOWS ${space} Windows 11 LTSC Installation
item DEBIAN ${space} Debian Installation
item --gap Connect to Internet Sources
item NETBOOT ${space} Netboot.xyz
choose selection && goto ${selection} || goto ERROR

:WINDOWS
echo Some windows things here
sleep 3
goto MAIN

:DEBIAN
dhcp
imgfree
set base http://netboot/debian-installer/amd64
kernel ${base}/linux 
initrd ${base}/initrd.gz
boot || goto ERROR

:NETBOOT
dhcp
chain --autofree http://boot.netboot.xyz || goto ERROR

:ERROR
echo There was a problem with the selection. Exiting...
sleep 3
exit

Local-vars

Netboot.xyz detects that it’s working with a Proxy PXE server and behaves a little differently. For example, you can’t insert your own local menu.ipxe. One helpful addition is a local settings file to speed up boot.

sudo vi /var/www/html/local-vars.ipxe

#!ipxe
set use_proxydhcp_settings true

Operation

You can choose the new menu item and load netboot.xyz. It will take you out the web for more selections. Not everything will load on every client, of course. But it gives you a lot of options.

Next Steps

We glossed over how to install Windows. That’s a useful item.

Troubleshooting

Wrong TFTP Server

tftp://192.168.0.1/local-vars.ipxe....Connection timed out
Local vars file not found... attempting TFTP boot...
DHCP proxy detected, press p to boot from 192.168.0.2...

If your boot client is attempting to connect to the main DHCP server, that server is probably sending value next server: 192.168.0.1 in it’s packets. This isn’t a DNS option per say, but it affects netboot. Dnsmasq does this though Kea doesn’t.

sudo systemctl -u dnsmasq -f

...
...
next server: 192.168.0.1
...
...

The boot still works, it’s just annoying. You can usually ignore the message and don’t have to hit ‘p’.

Exec Format Error

Could not boot: Exec format error (https://ipxe.org/2e008081)

You may see this flash by. Check your menus and local variables file to make sure you’ve in included the #!pxe shebang.

No Internet

You can also host your own local instance.

4 - menu-old

It would be useful to have some choices when you netboot. You can use the pxe-service built into dnsmasq but a more flexible option is the menu system provided by the iPXE project.

Installation

Set up a http/pxe net-boot server if you haven’t already.

Configuration

dnsmasq

Configure dnsmasq to serve up the ipxe.efi binary for both types of clients.

# Disable DNS
port=0 
 
# Use in DHCP PXE Proxy mode
dhcp-range=192.168.0.0,proxy 
 
# Tell dnsmasq to provide proxy PXE service to both PXE and HTTP clients
dhcp-pxe-vendor=PXEClient,HTTPClient 
 
# Send the BOOTP information for the clients using HTTP
dhcp-boot="http://netboot/ipxe.efi" 

# Specify a boot menu option for PXE clients. If there is only one, it's booted immediately.
pxe-service=x86-64_EFI,"iPXE (UEFI)", "ipxe.efi"
  
# We also need to enable TFTP for the PXE clients  
enable-tftp 
tftp-root=/var/www/html

Custom Menu

Change the autoexec.ipxe to display a menu.

sudo vi /var/www/html/autoexec.ipxe

#!ipxe

echo ${cls}

:MAIN
menu Local Netboot Menu
item --gap Local Network Installation
item WINDOWS ${space} Windows 11 LTSC Installation
item DEBIAN ${space} Debian Installation
choose selection && goto ${selection} || goto ERROR

:WINDOWS
echo Some windows things here
sleep 3
goto MAIN

:DEBIAN
dhcp
imgfree
set base http://netboot/debian-installer/amd64
kernel ${base}/linux 
initrd ${base}/initrd.gz
boot || goto ERROR


:ERROR
echo There was a problem with the selection. Exiting...
sleep 3
exit

Operation

You’ll doubtless find additional options to add. You may want to add the netboot.xyz project to your local menu too.

5 - windows

To install windows, have iPXE load wimboot then WinPE. From there you can connect to a samba share and start the Windows installer. Just like back in the gold-ole administrative installation point days.

Getting a copy of WinPE the official way is a bit of a hurdle, but definitely less work than setting up a full Windows imaging solution.

Installation

Samba and Wimboot

On the netboot server, install wimboot and Samba.

sudo wget https://github.com/ipxe/wimboot/releases/latest/download/wimboot -P /var/www/html
sudo apt install samba

Window ADK

On a Windows workstation, download the ADK and PE Add-on and install as per Microsoft’s ADK Install Doc.

Configuration

Samba

Prepare the netboot server to receive the Windows files.

sudo vi /etc/samba/smb.conf

[global]
  map to guest = bad user
  log file = /var/log/samba/%m.log

[install]
  path = /var/www/html
  browseable = yes
  read only = no
  guest ok = yes
  guest only = yes

sudo mkdir /var/www/html/winpe
sudo mkdir /var/www/html/win11
sudo chmod o+w /var/www/html/win*
sudo systemctl restart smbd.service

Window ADK Config

On the Windows workstation, start the deployment environment as an admin and create the working files as below. More info is in Microsoft’s Create Working Files document.

  • Start -> All Apps -> Windows Kits -> Deployment and Imaging Tools Environment (Right Click, More, Run As Admin)
copype amd64 c:\winpe\amd64

Add the required additions for Windows 11 with the commands below. These are the optional components WinPE-WMI and WinPE-SecureStartup and more info is in Microsoft’s Customization Section.

mkdir c:\winpe\offline

dism /mount-Image /Imagefile:c:\winpe\amd64\media\sources\boot.wim /index:1 /mountdir:c:\winpe\offline

dism /image:c:\winpe\offline /add-package /packagepath:"..\Windows Preinstallation Environment\amd64\WinPE_OCs\WinPE-WMI.cab" /packagepath:"..\Windows Preinstallation Environment\amd64\WinPE_OCs\WinPE-SecureStartup.cab"

dism /unmount-image /mountdir:c:\winpe\offline /commit

Make the ISO in case you want to HTTP Boot from it later and keep the shell open for later.

MakeWinPEMedia /ISO C:\winpe\amd64 C:\winpe\winpe_amd64.iso

WinPE

Now that you’ve got a copy of WinPE, copy it to the netboot server.

net use q: \\netboot\install
xcopy /s c:\winpe\* q:\winpe

Also create some auto-start files for setup. The first is part to the WinPE system and tells it (generically) what to do after it starts up.

notepad q:\winpe\amd64\winpeshl.ini

[LaunchApps]
"install.bat"

This the second is more specific and associated with the thing you are installing. We’ll mix and match these in the PXE menu later so we can install different things.

notepad q:\win11\install.bat

wpeinit
net use \\netboot
\\netboot\install\win11\setup.exe
pause

Win 11

You also need to obtain the latest ISO and extract the contents.

Wimboot

Bck on the netboot server, customize the WINDOWS section of your autoexex.ipxe like this.

:WINDOWS
dhcp
imgfree
set winpe http://netboot/winpe/amd64
set source http://netboot/win11
kernel wimboot
initrd ${winpe}/media/sources/boot.wim boot.wim
initrd ${winpe}/media/Boot/BCD         BCD
initrd ${winpe}/media/Boot/boot.sdi    boot.sdi
initrd ${winpe}/winpeshl.ini           winpeshl.ini
initrd ${source}/install.bat           install.bat
boot || goto MAIN

You can add other installs by copying this block and changing the :WINDOWS header and source variable.

Next Steps

Add some more installation sources and take a look at the Windows zero touch install.

Troubleshooting

System error 53 has occurred. The network path was not found

A given client may be unable to connect to the SMB share, or it may fail once, but then connect on a retry a moment later. I suspect it’s because the client doesn’t have an IP yet, though I’ve not looked at it closely. You can usually just retry.

You can also comment out the winpeshl.ini line and you’ll boot to a command prompt that will let you troubleshoot. Sometimes you just don’t have an IP yet from the DHCP server and you can edit the install.bat file to add a sleep or other things. See then [zero touch deployment] page for some more ideas.

Access is denied

This may be related to the executable bit. If you’ve copied from the ISO they should be set. But if after that you’ve changed anything you could have lost the x bit from setup.exe. It’s hard to know what’s supposed to be set once it’s gone, so you may want to recopy the files.

6 - Legacy

Many older systems can’t HTTP Boot so let’s add PXE support with some dnsmasq options.

Installation

Dnsmasq

Install as in the httpboot page.

The Debian Installer

Older clients don’t handle ISOs well, so grab and extract the Debian netboot files.

sudo wget http://ftp.debian.org/debian/dists/bookworm/main/installer-amd64/current/images/netboot/netboot.tar.gz -O - | sudo tar -xzvf - -C /var/www/html

Grub is famous for ignoring proxy dhcp settings, so let’s start off the boot with something else; iPXE. It can do a lot, but isn’t signed so you must disable secure boot on your clients.

sudo wget https://boot.ipxe.org/ipxe.efi -P /var/www/html

Configuration

iPXE

Debian is ready to go, but you’ll want to create an auto-execute file for iPXE so you don’t have to type in the commands manually.

sudo vi /var/www/html/autoexec.ipxe

#!ipxe

set base http://netboot/debian-installer/amd64

dhcp
kernel ${base}/linux
initrd ${base}/initrd.gz
boot

Dnsmasq

HTTP and PXE clients need different information to boot. We handle this by adding a filename to the PXE service option. This will override the dhcp-boot directive for PXE clients.

sudo vi /etc/dnsmasq.d/netboot.conf 

# Disable DNS
port=0 
 
# Use in DHCP PXE Proxy mode
dhcp-range=192.168.0.0,proxy 
 
# Respond to both PXE and HTTP clients
dhcp-pxe-vendor=PXEClient,HTTPClient 
 
# Send the BOOTP information for the clients using HTTP
dhcp-boot="http://netboot/debian.iso" 

# Specify a boot menu option for PXE clients. If there is only one, it's booted immediately.
pxe-service=x86-64_EFI,"iPXE (UEFI)", "ipxe.efi"

# We also need to enable TFTP for the PXE clients
enable-tftp 
tftp-root=/var/www/html

Client

Both types of client should now work. The debian installer will pull the rest of what it needs from the web.

Next Steps

You can create a boot-menu by adding multiple pxe-service entries in dnsmasq, or by customizing the iPXE autoexec.ipxe files. Take a look at that in the menu page.

Troubleshooting

Text Flashes by, disappears, and client reboots

This is most often a symptom of secure boot still being enabled.

Legacy Clients

These configs are aimed at UEFI clients. If you have old BIOS clients, you can try the pxe-service tag for those.

pxe-service=x86-64_EFI,"iPXE (UEFI)", "ipxe.efi"
pxe-service=x86PC,"iPXE (UEFI)", "ipxe.kpxe"

This may not work and there’s a few client flavors so enable the dnsmasq logs to see how they identify themselves. You can also try booting pxelinux as in the Debian docs.

DHCP Options

Dnsmasq also has a whole tag system that you can set and use similar to this:

dhcp-match=set:PXE-BOOT,option:client-arch,7
dhcp-option=tag:PXE-BOOT,option:bootfile-name,"netboot.xyz.efi"

However, dnsmasq in proxy mode limits what you can send to the clients, so we’ve avoided DHCP options and focused on PXE service directives.

Debian Error

*ERROR* CPU pipe B FIFO underrun

You probably need to use the non-free firmware

No Boot option

Try entering the computers bios setup and adding a UEFI boot option for the OS you just installed. You may need to browse for the file \EFI\debian\grubx64.efi

Sources

https://documentation.suse.com/sles/15-SP2/html/SLES-all/cha-deployment-prep-uefi-httpboot.html https://github.com/ipxe/ipxe/discussions/569 https://linuxhint.com/pxe_boot_ubuntu_server/#8

It’s possible to use secure boot if you’re willing to implement a chain of trust. Here’s an example used by FOG to boot devices.

https://forums.fogproject.org/topic/13832/secureboot-issues/3