Mesh

A mesh usually means anything with more than one way to get somewhere. For purposes of hosts, it often shows up as all the hosts being directly connected to each other.

When you have three hosts, you can do this when each one as an extra dual interface NIC. But you probably shouldn’t. It saves you the cost of a switch but you’ll have to fight with it later on.

Here’s an example from PVE.

The classic example was to put the same IP address on both interfaces and use destination-based routing differently on each host, to make sure they take the right path.

Modern Trixie however, uses systemd-networkd and it precludes putting the same IP address on two interfaces.

One solution is to use split-brain DNS where each node has two different IP addresses, and they have different entires in the /etc/hosts file so that they’ll use the right IP depending on where they are.

Another is to use a bridge address. This is probably better. According to Google AI:

The most straightforward way to use a single IP across multiple physical ports in a mesh is to create a bridge device. This treats all your interfaces as a single logical switch.

Create the Bridge Device (/etc/systemd/network/10-br0.netdev):

[NetDev]
Name=br0
Kind=bridge

Bind Physical Interfaces (e.g., eth0 and eth1 in /etc/systemd/network/20-mesh.network):

[Match]
Name=eth0 eth1

[Network]
Bridge=br0

Assign the IP to the Bridge (/etc/systemd/network/30-br0.network):

[Match]
Name=br0

[Network]
Address=172.31.1.x/24

And Possibly

To prevent ARP Flux (where the wrong interface responds to an ARP request because both interfaces “see” the same subnet), you need to tell the Linux kernel to only respond if the IP is on the specific interface receiving the packet.

Add these to /etc/sysctl.d/99-linstor-mesh.conf:

# Only respond if the target IP is configured on the incoming interface
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1

# Only use the address on the outgoing interface for ARP requests
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_announce = 2

# Enable strict reverse path filtering to prevent spoofing/asymmetric routing
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

sudo sysctl -p /etc/sysctl.d/99-linstor-mesh.conf