Configuration

Incus Initialization

Start the initialization on the node with the full install. Node 1 for us.

ssh debain1
sudo incus admin init

You’re safe to take the defaults for all but a couple questions.

Would you like to use clustering? (yes/no) [default=no]: yes

Would you like to use an existing empty block device (e.g. a disk or partition)? (yes/no) [default=no]: yes

Path to the existing block device: /dev/sdb

Post Initialization Checks

For Debian

The initializer uses the hostname by default. In some cases it resolves that as 127.0.1.1 based on the first line in the hosts file. That will cause problems for the other hosts and prevent more esoteric things like split-brain DNS. You can check and possibly correct this with the command incus admin cluster.

Web UI

If you have multiple NICs you must tell Incus to listen on them (or at least the one you want).

# Be careful about changing the port later on, as it's also used for cluster communication
sudo incus config set core.https_address 0.0.0.0:8443

Adding Cluster Members

Before you leave the 1st host, create a cluster join token.

# Assuming the next server you add will be NODENAME2
sudo incus cluster add debian2 

Repeat the config on successive hosts, adding the disk similar to before.

sudo incus admin init

Would you like to use clustering? (yes/no) [default=no]: yes

Are you joining an existing cluster? (yes/no) [default=no]: yes

The Web UI

You can now connect to the Web user interface. It will prompt you to generate a key pair. I suggest not using a password as it complicates the import. You can then copy the key from the server to your workstation and import it into your browser.

Managing Incus Networks

The Default Network

Incus creates a virtual network for your guests and provides DHCP and DNS. It does this by creating a linux bridge device incusbr0, wrapping an instance of dnsmasq, and creating a nftable to handle masquerade.

If you want to isolate guests, you create a new network and a profile that uses that, Then assign guests to that profile.

Create a LAN Network

If you have a cluster you probably want instances to talk to each other. The simplest way to do that is to put them on the LAN. Do that by installing bridge-utils and creating a bridge.

On Debian 13 use systemd-networkd for this.

# Install the bridge utilities
sudo apt install bridge-utils

# Create a network device file to define the bridge device
vi /etc/systemd/network/10-br0.netdev

[NetDev]
Name=br0
Kind=bridge

# Define the network details in a network file
vi /etc/systemd/network/10-br0.network

[Match]
Name=br0

[Network]
Address=192.168.250.100/24
Gateway=192.168.250.1
DNS=192.168.250.1

# And then reload
sudo systemctl restart systemd-networkd.service

# Tell Incus about it  
incus network create LAN-BRIDGE --type=physical parent=br0

# Change the default profile so instances use the bridge by default
incus profile device remove default eth0
incus profile device add default eth0 nic nictype=bridged parent=br0 name=eth0

Note: Incus may need a restart before it accepts br0

Create a Fully Isolated Network

An interesting edge case is testing netboot where you need to turn off Incus’s DHCP. For that you must create a network without those services at the command line.

incus network create test ipv4.address=none ipv6.address=none
incus profile copy default isolated

Managing Storage

Individual Instances

Incus uses the concept of a storage pool. This is a location to hold disk images, ISOs, profiles and such. You can see the default with:

sudo incus storage show default

This is a shared space but instances are kept separate. Each gets their own volume inside the pool. This is thin-provisioned so it only takes as much space as used, but there is a max size of 10G. In some cases this isn’t enough so you can change the default.

sudo incus storage set default size 100GiB

Container Access to Shares

The best way is to mount it on your host and pass it to your containers with a bind-mount. This keeps the kernel modules and caching out of the containers where you’d otherwise need escalated privileges and have less efficient caching.

User IDs are normally mis-matched because containers have their own name-space. But you can line them up easily with the shift option.

incus config device add syncthing srv disk shift=true source=/srv/media path=/srv/media

Passing Video Cards

If you’re transcoding media it helps to have access to a video card, or at least the integrated video card hardware in modern CPUs.

Here’s a good discussion on it.

https://discuss.linuxcontainers.org/t/sharing-a-gpu-with-multiple-containers/21913/2

If you don’t have an existing media or render group, you can use the GUI of the process owner. For a server like Jellyfin, you can use the GID of the jellyfin user.

Command Line Administration Users

You can sudo Incus commands, but you can also add users to the admin group.

sudo adduser YOUR-USERNAME incus-admin
incus admin init