I have been using opnsense on a very cheap celeron nuc for a few years, very happy with it

I personally would flick through the OpenWRT supported devices and pick the best supported device with 802.11ax.

How much wifi and open-source do you really want?

If you are willing to go with commercial hardware + open source firmware (OpenWrt) you might want to check the table of hardware of OpenWrt at https://openwrt.org/toh/views/toh_available_16128_ax-wifi and https://openwrt.org/toh/views/toh_available_864_ac-wifi. One solid pick for the future might be the Netgear WAX2* line or the GL.iNet GL-MT6000. One of those models is now fully supported the others are on the way. If you don’t mind having older wifi a Netgear R7800 is solid.

For a full open-source hardware and software experience you need a more exotic brand like this https://www.banana-pi.org/en/bananapi-router/. The BananaPi BPi R3 and here is a very good option with a 4 core CPU, 2GB of RAM Wifi6 and two 2.5G SFP ports besides the 4 ethernet ports. There’s also an upcoming board the BPI-R4 with optional Wifi 7 and 10G SPF.

Both solutions will lead to OpenWRT when it comes to software, it is better than any commercial firmware but be aware that it only support wifi hardware with open-source drives such as MediaTek. While MediaTek is good and performs very well we can’t forget that the best performing wifi chips are Broadcom and they use hacks that go behind the published wifi standards and get it go a few megabytes/second faster and/or improve the range a bit.

DD-WRT is another “open-source” firmware that has a specific agreement with Broadcom to allow them to use their proprietary drivers and distribute them as blob with their firmware. While it works don’t expect compatibility with newer hardware nor a bug free solution like OpenWRT is.

There are also alternatives like OPNsense and pfSense that may make sense in some cases you most likely don’t require that. You’ve a small network and OpenWRT will provide you with a much cleaner open-source experience and also allow for all the customization you would like. Another great advantage of OpenWRT is that you’ve the ability to install 3rd party stuff in your router, you may even use qemu to virtualize stuff like your Pi-Hole on it or simply run docker containers.

I use an entry level router ASUS RT-AX53U with OpenWrt. WiFi 6, IPv6, Guest VLAN, DNSCrypt (DoH), Adblock, Firewall are few things I have configured with OpenWrt.

Even if you don’t buy ASUS, make sure your router is supported by OpenWrt. It’s a Linux distribution that runs on routers and PCs to configure home networking.

Fritzboxes are rock stable, and support Wireguard from FritzOS 7.5 onwards, see https://avm.de/service/vpn/wireguard-vpn-zur-fritzbox-am-computer-einrichten/

(Apparently NOT the cable versions!)

What nags me most with them is that you have no separate Firewall controll over their WiFi, and the WiFi range is not really great. So probably consider going with dedicated APs instead.

i am happy to have a raspberry pi setup connected to a VLAN switch, internet is behind a modem (like bridged mode) connected with ethernet to one switchport while the raspi routes everything through one tagged physical GB switchport. the setup works fine with two raspi’s and failover without tcp disconnections during an actual failover, only few seconds delay when that happens, so basically voip calls recover after seconds, streaming is not affected, while in a game a second off might be too much already, however as such hardware failures happen rarely, i am running only one of them anyway.

for firewall i am using shorewall, while for some special routing i also use unbound dns resolver (one can easily configure static results for any record) and haproxy with sni inspection for specific https routing for the rather specialized setup i have.

my wifi is done by an openwrt but i only use it for having separate wifis bridged to their own vlans.

thus this setup allows for multi-zone networks at home like a wifi for visitors with daily changing passwords and another fror chromecast or home automation, each with their own rules, hardware redundancy, special tweaking, everything that runs on gnu/linux is possible including pihole, wireguard, ddns solutions, traffic statistics, traffic shaping/QOS, traffic dumps or even SSL interception if you really want to import your own CA into your phone and see what data your phones apps (those that don’t use certificate pinning) are transfering when calling home, and much more.

however regarding ddns it sometimes feels more safe and reliable to have a somehow reserved IP that would not change. some providers offer rather cheap tunnels for this purpose. i once had a free (ipv6) tunnel at hurricane electronic (besides another one for IPv4) but now i use VMs in data centers.

i do not see any ready product to be that flexible. however to me the best ready router system seems to be openwrt, you are not bound to a hardware vendor, get security updates longer than with any commercial product, can 1:1 copy your config to a new device even if the hardware changes and has the possibility to add packages with special features to it.

“openwrt” is IMHO the most flexible ready solution for longtime use. same as “pfsense” is also very worth looking at and has some similarities to openwrt while beeing different.

Wireguard and DNS filtering (albeit not as fine tuned and automatic as pihole) can all be done on OpnSense

I recommend OpnSense on whatever modern low-power hardware you can get your hands on, ThinkCentre, NUC or whatever, if you are okay with a separate device for WiFi or do not need WiFi. WiFi APs can be had for as low as 20 bucks and are usually straight forward to set up, but you gotta shell out more if you want the latest and greatest connectivity.

There is also the possibility for adding WiFi directly to OpnSense but I have not even bothered touching it. If you love tinkering and suffering, that’s a route you can go.

For the love of God, if you’re going to install PfSense, just get OpnSense instead. It’s just better.

If you want to start small, I’d go with one supported by Asuswrt-Merlin, “a third party alternative firmware for Asus routers, with a special emphasis on tweaks and fixes rather than radical changes or collecting as many features as possible.” Keeps it close to stock with minor upgrades, and a faster release cycle for fixes. The RT-AX88U_PRO is one of the higher end routers that is supported by Merlin.

When I reached your situation, I started rackmounting which has saved me a lot of time.

I got a 1u dell poweredge r210 and slapped in a 10Gb network card. Loaded up OPNsense onto it. OPN sense was not easy to learn how to use, for me at least. Struggled to get everything running smoothly. But I am very happy I went with rack mounting instead of adding to the rat’s nest.

I recently bought an x86 passive cooled box from Topton, an aliexpress merchant, that was recommended by ServeTheHome, a great youtube channel/blog that reviews all kinds of networking equipment for homelabs. Since it’s x86, you can pretty much install anything on it, in my case OPNSense. I recommend you watch some of their videos/read their blogs and see what fits!

I guess maybe too mainline for everyone here but I use an Asus router flashed with the Merlin OS (a painless easy process) and it works excellently. No issues setting up all the things you mentioned.

you can convert really any computer into a little router using the help of an ethernet card. I’m planning to do exactly that for my homeserver

I find DrayTek devices to work quite well.

Hi-end Xiaomi router, they have WiFi ax and enough ram and support open wrt so you can host your things on it, better yet, do DIY router on orange pi board, there’s tutorials you can follow