This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Internet

1 - DNS

Web pages today are complex. Your browser will make on average 401 DNS queries to find the various parts of an average web page, so implementing a local DNS system is key to keeping things fast.

In general, you can implement either a caching or recursive server with the choice between speed vs privacy.

Types of DNS Servers

A caching server accepts and caches queries, but doesn’t actually do the lookup itself. It forwards the request on to another DNS server and waits for the answer. If you have a lot of clients configured to use it, chances are someone else has already asked for what you want and it can supply the answer quickly from cache.

A recursive server does more than just cache answers. It knows how to connect to the root of the internet and find out itself. If you need to find some.test.com, it will connect to the .com server, ask where test.com is, then connect to test.com and ask it for some.

Comparison

Between the two, the caching server will generally be faster. If you connect to a large DNS service they will almost always have things cached. You will also get geographically relevant results as content providers work with DNS providers to direct you to the closest content cache.

With a recursive server, you do the lookup yourself and no single entity is able to monitor your DNS queries. You also aren’t dependant upon any upstream provider. But you make every lookup ’the long way’, and that can take many hundreds of milliseconds on some cases, a large part of a page load time.

Testing

In an ad hoc test on a live network with about 5,000 residential user devices, about half the queries were cached. The other half were sent to either quad 9 or a local resolver. Quad 9 took about half the time that the local resolver did.

Here are the numbers - with Steve Gibson’s DNS benchmarker against pi-hole forwarding to a local resolver vs pi-hole forwarding to quad 9. Cached results excluded.

    Forwarder     |  Min  |  Avg  |  Max  |Std.Dev|Reliab%|
  ----------------+-------+-------+-------+-------+-------+
  - Uncached Name | 0.015 | 0.045 | 0.214 | 0.046 | 100.0 |
  - DotCom Lookup | 0.015 | 0.019 | 0.034 | 0.005 | 100.0 |
  ---<O-OO---->---+-------+-------+-------+-------+-------+

    Resolver      |  Min  |  Avg  |  Max  |Std.Dev|Reliab%|
  ----------------+-------+-------+-------+-------+-------+
  - Uncached Name | 0.016 | 0.078 | 0.268 | 0.079 | 100.0 |
  - DotCom Lookup | 0.018 | 0.035 | 0.078 | 0.017 | 100.0 |
  ---<O-OO---->---+-------+-------+-------+-------+-------+

Selection

This test is interesting, but not definitive. While the DNS benchmark shows that the uncached average is better, page load perception is different than the sum of DNS queries. A page metric test would be good, but in general, faster is better.

Use a caching server.

One last point: use your ISP’s name server when possible. They will direct you to their local content caching systems for Netflix, Google (YouTube) and Akamai. If you use quad 9 like I did, you may get to a regional content location, but you miss out on things optimized specifically for your geographic location.

They are (probably) capturing all your queries for monetization, and possibly directing you to their own their own advertising server when you mis-key in a domain name. So you’ll need to decide;

Speed vs privacy.

  1. Informal personal checking of random popular sites. ↩︎

1.1 - Pi-hole

Pi-hole is reasonable choice for DNS service, especially if you want metrics and reporting. A single instance will scale to 1000 active clients with just 1 core and 500M RAM and do a good job showing what’s going on.

Performance is limited with version 5, however. When you get past 1000 active clients you can take some mitigating steps, but the main process is single-threaded so you’re unlikely to get past 1500.

But for smaller deployments, it’s hard to beat.

Preparation

Prepare and secure a Debian system

Set a Static Address

sudo vi /etc/network/interfaces

Change

# The primary network interface
allow-hotplug eth0
iface eth0 inet dhcp

to

auto eth0
iface eth0 inet static
    address 192.168.0.2/24
    gateway 192.168.0.1

Secure Access with Nftables

Nftables is the modern way to set netfilter (localhost firewall) rules.

sudo apt install nftables
sudo systemctl enable nftables
sudo vi /etc/nftables.conf

#!/usr/sbin/nft -f

flush ruleset

table inet filter {
        chain input {
                type filter hook input priority 0;

                # accept any localhost traffic
                iif lo accept

                # accept already allowed and related traffic
                ct state established,related accept

                # accept DNS and DHCP traffic from internal only
                define RFC1918 = { 192.168.0.0/16, 10.0.0.0/8, 172.16.0.0/12 }
                udp dport { domain, bootps } ip saddr $RFC1918 ct state new accept
                tcp dport { domain, bootps } ip saddr $RFC1918 ct state new accept

                # accept web and ssh traffic on the first interface or from an addr range
                iifname eth0 tcp dport { ssh, http } ct state new accept
                 # or 
                ip saddr 192.168.0.1/24 ct state new accept

                # Accept pings
                icmp type { echo-request } ct state new accept

                # accept neighbor discovery otherwise IPv6 connectivity breaks.
                ip6 nexthdr icmpv6 icmpv6 type { nd-neighbor-solicit,  nd-router-advert, nd-neighbor-advert } accept

                # count other traffic that does match the above that's dropped
                counter drop
        }
}

sudo nft -f /etc/nftables.conf
sudo systemctl start nftables.service

Add Unattended Updates

This an optional, but useful service.

apt install unattended-upgrades

sudo sed -i 's/\/\/\(.*origin=Debian.*\)/  \1/' /etc/apt/apt.conf.d/50unattended-upgrades
sudo sed -i 's/\/\/\(Unattended-Upgrade::Remove-Unused-Kernel-Packages "true";\)/  \1/' /etc/apt/apt.conf.d/50unattended-upgrades
sudo sed -i 's/\/\/\(Unattended-Upgrade::Remove-Unused-Dependencies\) "false";/  \1 "true";/' /etc/apt/apt.conf.d/50unattended-upgrades
sudo sed -i 's/\/\/\(Unattended-Upgrade::Automatic-Reboot\) "false";/  \1 "true";/' /etc/apt/apt.conf.d/50unattended-upgrades

Installation

sudo apt install curl
curl -sSL https://install.pi-hole.net | bash

Configuration

Upstream Provider

Pi-hole is a DNS Forwarder, meaning it asks someone else on your behalf and caches the results for other people when they ask too.

Assuming the installation messages indicate success, the only thing you need do is pick that upstream DNS provider. In many cases your ISP is the fastest, but you may want to run a [DNS benchmark] with those added to find the best.

  • Settings -> DNS -> Upstream DNS Servers

Also, if you have more than one network, you may need to allow the other LANs to connect.

  • Interface settings -> Permit all origins (needed if you have multiple networks)

Block Lists

One of the main features of Pi-hole is that it blocks adds. But you may need to disable this for google or facebook search results to work as expected. The top search results are often ads and don’t work as expected when pi-hole is blocking them.

  • Admin Panel -> Ad Lists -> Status Column

You might consider adding security only lists instead, such as Intel’s below

Search the web for other examples.

Operation

DNS Cache Size

When Pi-hole looks up a name, it get’s back an IP address and what’s called a TTL (Time To Live). The latter tells Pi-hole how long it should keep the results in cache for.

If you’re asking for names faster than they are expiring, Pi-hole will ’evict’ things from cache before they should be. You can check this at:.

settings -> System -> DNS cache evictions:

You’ll notice that insertions keep climbing as things are added to the cache, but the cache number itself represents only those entries that are current. If you do see evictions, edit CACHE_SIZE in /etc/pihole/setupVars.conf

You can also check this at the command line

dig +short chaos txt evictions.bind @localhost

   dig +short chaos txt cachesize.bind
   dig +short chaos txt hits.bind
   dig +short chaos txt misses.bind

Having more than you need is a waste, however, when it could be used for disk buffers, etc. So don’t add more unless it’s needed.

Use Stale Cache

If you have the spare memory, you can tell Pi-hole not to throw away potentially-useful data with the use-stale-cache flag. It will instead wait for someone to ask for it, serve it up immediately even though it’s expired, then go out and check to see if anything has changed. So TTLs are ignored but unused entries are removed after 24 hours to keep the cache tidy.

This increases performance at the expense of more memory use.

# Add this to pi-hole's dnsmasq settings file
echo "use-stale-cache" >> /etc/dnsmasq.d/01-pihole.conf

Local DNS Entries

You can enter local DNS and CNAME entries via the GUI, (Admin Panel -> Local DNS), but you can also edit the config file for bulk entries.

For A records

vim /etc/pihole/custom.list

10.50.85.2 test.some.lan
10.50.85.3 test2.some.lan

For CNAME records

vim /etc/dnsmasq.d/05-pihole-custom-cname.conf

cname=test3.some.lan,test.some.lan

Upgrading

Pi-hole wasn’t installed via a repo, so it won’t be upgraded via the Unattended Upgrades service. It requires a manual command.

sudo pihole -up

Troubleshooting

Dashboard Hangs

Very busy pi-hole installations generate lots of data and (seemingly) hang the dashboard. If that happens, limit the about of data being displayed.

vi /etc/pihole/pihole-FTL.conf 

# Don't import the existing DB into the GUI - it will hang the web page for a long time
DBIMPORT=no

# Don't import more than an hour of logs from the logfile
MAXLOGAGE=1

# Truncate data older than this many days to keep the size of the database down
MAXDBDAYS=1

sudo systemctl restart pihole-FTL.service

Rate Limiting

The system has a default limit of 1000 queries in a 60 seconds window for each client. If your clients are proxied or relayed, you can run into this. This event is displayed in the dashbaord1 and also in the logs2.

sudo grep -i Rate-limiting /var/log/pihole/pihole.log /var/log/pihole/pihole.log

You may find the address 127.0.0.1 being rate limited. This can be due to pi-hole doing a reverse of all client IPs every hour. You can disable this with:

# In the pihole-FTL.conf
REFRESH_HOSTNAMES=NONE

DNS over HTTP

Firefox, if the user has not yet chosen a setting, will query use-application-dns.net. Pi-hole respods with NXDOMAIN3 as a signal to use pi-hole for DNS.

/etc/pihole/pihole-FTL.conf

Apple devices offer a privacy enhancing service4 that you can pay for. Those devices queries mask.icloud.com as a test and Pi-hole blocks that by default. The user will be notified you are blocking it.

# Signal that Apple iCloud Private Relay is allowed 
BLOCK_ICLOUD_PR=false

sudo systemctl reload pihole-FTL.service

Searching The Query Log Hangs DNS

On a very busy server, clicking show-all in the query log panel will hang the server as pihole-FTL works through it’s database. There is no solution, just don’t do it. The best alternative is to ship logs to a Elasticsearch or similar system.

Ask Yourself

The system continues to use whatever DNS resolver was initially configured. You may want it to use itself, instead.

# revert if pi-hole itself needs fixed.
sudo vi /etc/resolv.conf

nameserver 127.0.0.1

Other Settings

Pi-hole can be configured via it’s two main config files, /etc/pihole/setupVars.conf and pihole-FTL.conf, and you may find other useful settings.

In the v6, currently in beta, the settings are all in the .toml file, or via the gui via:

  • All Settings — Resolver – resolver.resolveIPv4 – resolver.resolveIPv6 – resolver.networkNames

  1. https://pi-hole.net/blog/2021/09/11/pi-hole-ftl-v5-9-web-v5-6-and-core-v5-4-released/#page-content ↩︎

  2. https://discourse.pi-hole.net/t/include-log-entry-in-pihole-ftl-log-when-client-hits-rate-limit/46798/12A ↩︎

  3. https://www.reddit.com/r/pihole/comments/113qkp5/i_am_seeing_useapplicationdnsnet_being_blocked/ ↩︎

  4. https://docs.pi-hole.net/ftldns/configfile/?h=mask#icloud_private_relay [DNS benchmark]:https://www.grc.com/dns/benchmark.htm ↩︎

1.2 - Pi-hole Unbound

Pi-hole by itself is just a DNS forwarding and caching service. It’s job is to consolidate requests and forward them on to someone else. That someone else is seeing all your requests.

If that concerns you, add your own DNS Resolver like Unbound. It knows how to fetch answers without any one entity seeing all your requests. It’s probably slower1, but possibly more secure.

Installation

sudo apt install unbound

Configuration

Unbound

The pi-hole guide for [unbound]:(https://docs.pi-hole.net/guides/dns/unbound/) includes a config block to copy and paste as directed. You should also add a config file for dnsmasq while you’re at it, to set EDNS packet sizes. (dnsmasq comes as part of pi-hole)

sudo vi /etc/dnsmasq.d/99-edns.conf

edns-packet-max=1232

When you check the status of unbound, you can ignore the warning: subnetcache:... as it’s just reminding you that data in the subnet cache (if you were to use it) can’t pre-fetched. There’s some conversation2 as to why it’s warning us.

The config includes prefetch, but you may also wish to add serve-expired to it, if you’re not already using use-stale-cache in Pi-hole.

# serve old responses from cache while waiting for the actual resolution to finish.
# don't set this if you're already doing it in Pi-hole
serve-expired: yes

sudo systemctl restart unbound.service

No additional setup is needed, but see the unbound page for more info.

Pi-hole

You must tel Pi-hole about the Resolver you’ve just deployed.

  • Settings -> DNS -> Upstream DNS Servers -> Custom 1 (Check and add 127.0.0.1#5335 as shown in the unbound guide linked above)

  1. DNS Speed Testing Recursive vs Forwarding ↩︎

  2. https://www.reddit.com/r/pihole/comments/11xb7pt/unbound_warning_subnetcache_prefetch_and/ ↩︎

1.3 - Pi-hole DHCP

Pi-hole can also do DHCP. However, the GUI only allows for a single range. On a large network you’ll need multiple ranges. You do this by editing the config files directly.

Interface-Based Ranges

In this setup, you have a separate interface per LAN. Easy to do in a virtual or VLAN environment, but you’ll have to define each in the /etc/network/interfaces file.

Let’s create a range from 192.168.0.100-200 tied to eth0 and a range of 192.168.1.100-200 tied to eth1. We’ll also specify the router and two DNS servers.

vim /etc/dnsmasq.d/05-dhcp.conf

dhcp-range=eth0,192.168.0.100,192.168.0.200,24h
dhcp-option=ens161,option:router,192.168.0.1
dhcp-option=ens161,option:dns-server,192.168.0.2,192.168.0.3

dhcp-range=eth1,192.168.1.100,192.168.1.200,24h
dhcp-option=ens161,option:router,192.168.1.1
dhcp-option=ens161,option:dns-server,192.168.1.2,192.168.1.3

# Shared by both
dhcp-option=option:netmask,255.255.0.0

# Respond immediately without waiting for other servers 
dhcp-authoritative

# Don't try and ping the address before assigning it
no-ping

dhcp-lease-max=10000
dhcp-leasefile=/etc/pihole/dhcp.leases

domain=home.lan

These settings can be implicit - i.e. we could have left out ethX in the range, but explicit is often better for clarity.

Note - the DHCP server (dnsmasq) is not enabled by default. You can do that in the GUI under Settings –> DHCP

Relay-Based Ranges

In this setup, the router relays DHCP requests to the server. Only one system network interface is required, though you must configure the router(s).

When configured, the relay (router) sets the relay-agent (giaddr) field, sends it to dnsmasq, which (I think) understands it’s a relayed request when it sees that field, and looks at it’s available ranges for a match. It’s also sets a tag that to be used for assigning different options, such as the gateway, per range.

dhcp-range=tag0,192.168.0.100,192.168.0.250,255.255.255.0,8h
dhcp-range=tag1,192.168.1.100,192.168.1.250,255.255.255.0,8h
dhcp-range=tag2,192.168.2.100,192.168.2.250,255.255.255.0,8h

dhcp-options=tag0,3,192.168.0.1
dhcp-options=tag1,3,192.168.1.1
dhcp-options=tag2,3,192.168.2.1

Sources

https://discourse.pi-hole.net/t/more-than-one-conditional-forwarding-entry-in-the-gui/11359

Troubleshooting

It’s possible that the DHCP part of dnsmasq doesn’t scale to many thousands of leases1

  1. https://news.ycombinator.com/item?id=33895187 ↩︎