Wireguard config error causes no outbound traffic

[saghaulor]

First, thank you for your amazing tool! I've been happily using it for a few years. However, I recently upgraded my UDM-Pro to 3.x release, and noted that Wireguard was available to use. I've read great things about it, so I wanted to give it a try. Unfortunately I couldn't get it to work and I'm afraid that I can't figure out what I'm doing wrong. I would really appreciate if you could point me in the right direction. Thank you in advance!

A little background:
I have a vlan configured (br666) and mapped to a port on the UDM-Pro that I used to forward all traffic for a device to my VPN. br666 is running DHCP and leasing addresses for 192.168.2.0/24

I have a pi-hole running on br0, DHCP is running and leasing addresses for 192.168.1.0/24, the pi-hole is on 192.168.1.94.

Below is my wg0.conf:

[Interface]
# Key for udm-pro
# Bouncing = 11
# NetShield = 1
# Moderate NAT = off
# NAT-PMP (Port Forwarding) = off
# VPN Accelerator = on
PrivateKey = XxX=
Address = 10.2.0.2/32
# DNS = 10.2.0.1
PostUp = sh /etc/split-vpn/vpn/updown.sh %i up
PreDown = sh /etc/split-vpn/vpn/updown.sh %i down
Table = 101

[Peer]
# BR#45
PublicKey = xXx=
AllowedIPs = 0.0.0.0/1,128.0.0.0/1,::/1,8000::/1
Endpoint = 149.102.251.97:51820

And here is my vpn.conf

### SPLIT VPN OPTIONS ###
# Enter multiple entries separated by spaces.
# Do not enter square brackets around the entries.

# Force these sources through the VPN.
# Format: [brX] for interface. [IP/nn] for IP. [xx:xx:xx:xx:xx:xx] for mac.
FORCED_SOURCE_INTERFACE="br666"
FORCED_SOURCE_IPV4=""
FORCED_SOURCE_IPV6=""
FORCED_SOURCE_MAC=""

# Format: [tcp/udp/both]-[IP/MAC Source]-[port1,port2:port3,port4,...]
# Maximum 15 ports per entry.
FORCED_SOURCE_IPV4_PORT=""
FORCED_SOURCE_IPV6_PORT=""
FORCED_SOURCE_MAC_PORT=""

# Force these destinations through the VPN.
# These destinations will be forced regardless of source.
# Format: [IP/nn]
FORCED_DESTINATIONS_IPV4=""
FORCED_DESTINATIONS_IPV6=""

# Force local UDM traffic going out of these WAN interfaces to go through the
# VPN instead for both IPv4 and IPv6 traffic.
# This does not include routed traffic, only local traffic generated by the UDM.
# Do not enable this unless you want to force UDM local traffic through the VPN.
# For UDM-Pro, set to "eth8" for WAN1/Ethernet port, or "eth9" for WAN2/SFP+ port,
# or "eth8 eth9" for both. For UDM Base, set to "eth1" for the WAN port.
# This option might cause unintended problems, so disable it if you encounter any issues.
FORCED_LOCAL_INTERFACE=""

# Exempt these sources from the VPN.
# Format: [IP/nn] for IP. [xx:xx:xx:xx:xx:xx] for mac.
EXEMPT_SOURCE_IPV4=""
EXEMPT_SOURCE_IPV6=""
EXEMPT_SOURCE_MAC=""

# Format: [tcp/udp/both]-[IP/MAC Source]-[port1,port2:port3,port4,...]
# Maximum 15 ports per entry.
EXEMPT_SOURCE_IPV4_PORT=""
EXEMPT_SOURCE_IPV6_PORT=""
EXEMPT_SOURCE_MAC_PORT=""

# Exempt these destinations from the VPN.
# Format: [IP/nn]
EXEMPT_DESTINATIONS_IPV4=""
EXEMPT_DESTINATIONS_IPV6=""

# Force/exempt these IP sets
# IP sets need to be created before this script is run or the script will error.
# IP sets can be updated externally and will be matched dynamically.
# Each IP set entry consists of the IP set name and whether to match on source
# or destination. src/dst needs to be specified for each IP set field.
#
# Enable NAT hairpin by exempting UBIOS_ADDRv4_ethX:dst for IPv4 or
# UBIOS_ADDRv6_ethX:dst for IPv6 (where X = 8 for RJ45, or 9 for SFP+ WAN).
# For IPv6 prefix delegation, exempt UBIOS_ADDRv6_brX, where X = VLAN number (0 = LAN).
#
# To allow communication with your VLAN subnets without hardcoding the subnets,
# exempt the UBIOS_NETv4_brX:dst ipset for IPv4 or UBIOS_NETv6_brX:dst for IPv6.
#
# Format: [IPSet Name]:[src/dst,src/dst,...]
FORCED_IPSETS=""
EXEMPT_IPSETS=""

# VPN port forwards.
# Format: [tcp/udp/both]-[VPN Port]-[Forward IP]-[Forward Port]
PORT_FORWARDS_IPV4=""
PORT_FORWARDS_IPV6=""

# Redirect IPv4 and IPv6 DNS to these addresses for VPN-destined traffic.
# Note that many VPN providers redirect DNS going through their VPN network
# to their own DNS servers. Redirection to other IPs might not work on all providers,
# except for DNS redirects to a local address, or rejecting DNS traffic completely.
#
# IPV4 Format: [IP] to redirect to IP, "DHCP" if using OpenVPN or OpenConnect to obtain
# DNS from DHCP options, or "REJECT" to reject all DNS traffic. "DHCP" is not supported on
# other VPN types like wireguard/external.
#
# Example: Get DNS from DHCP
DNS_IPV4_IP="192.168.1.94"
DNS_IPV4_PORT=53
# Set this to the interface (brX) the DNS is on if it is a local IP. Leave blank for
# non-local IPs. Local DNS redirects will not work without specifying the interface.
DNS_IPV4_INTERFACE="br0"
# IPV6 Format: [IP] to redirect to IP, or "REJECT" to reject IPv6 DNS traffic completely.
# IPV6 Format: [IP] to redirect to IP, "DHCP" if using OpenConnect to obtain DNS from DHCP
# options, or "REJECT" to reject all DNS traffic. "DHCP" is not supported on
# other VPN types.
DNS_IPV6_IP=""
DNS_IPV6_PORT=53
DNS_IPV6_INTERFACE=""

# Bypass masquerade (SNAT) for these source IPs. This option should only be used if your
# VPN server is setup to know how to route the subnet you do not want to masquerade
# (e.g.: the "iroute" option in OpenVPN).
# Set these options to ALL to disable masquerading completely.
# Format: [IP/nn] or "ALL"
BYPASS_MASQUERADE_IPV4=""
BYPASS_MASQUERADE_IPV6=""

# Enabling kill switch drops VPN-destined traffic that doesn't go through the VPN.
KILLSWITCH=0

# Enable this only if you are testing or you don't care about your real IP leaking
# when the vpn client restarts or exits.
REMOVE_KILLSWITCH_ON_EXIT=1

# Enable this if you added blackhole routes in the Unifi Settings to prevent Internet
# access at system startup before the VPN script runs. This option removes the blackhole
# routes to restore Internet access after the killswitch has been enabled.
# If you do not set this to 1, openvpn will not be able to connect at startup, and your
# Internet access will never be enabled until you manually remove the blackhole routes.
# Set this to 0 only if you did not add any blackhole routes.
REMOVE_STARTUP_BLACKHOLES=1

# Set the VPN provider.
# "openvpn" for OpenVPN (default), "openconnect" for OpenConnect, "external" for wireguard,
# or "nexthop" for an external VPN client.
VPN_PROVIDER="external"

# If using "external" for VPN_PROVIDER, set this to the VPN endpoint IP so that the
# gateway route can be automatically added for the VPN endpoint.
# OpenVPN passes the VPN endpoint IP to the script and will override these values.
# These must be defined if using VPN_PROVIDER="nexthop".
VPN_ENDPOINT_IPV4="149.102.251.97"
VPN_ENDPOINT_IPV6=""

# Set this to the route table that contains the gateway route, "auto", or "disabled".
# The Ubiquiti route table is "201" if you're using Ethernet, "202" for SFP+, and
# "203" for U-LTE.
# Default is "auto" which works with WAN failover and automatically changes the endpoint
# via gateway route when the WAN or gateway routes changes.
# Set to "disabled" if you are using the nexthop option to connect to a VPN on your LAN.
GATEWAY_TABLE="auto"

# Set the MSS clamping on packets going out the VPN tunnel. Usually, it is not needed to
# set this manually, but some VPN connections stall if the MSS clamping is not set correctly.
# Typical values range from 1240 to 1460, but it could be lower.
MSS_CLAMPING_IPV4=""
MSS_CLAMPING_IPV6=""

# Set this to the timer to use for the rule watcher (in seconds).
# The script will wake up every N seconds to re-add rules if they're deleted by
# the system, or change gateway routes if they changed. Default is 1 second.
WATCHER_TIMER=1

# Options for custom table and chains.
# These options need to be unique for each instance of openvpn if running multiple.
ROUTE_TABLE=101
MARK=0x169
PREFIX="VPN_"
PREF=99
DEV=wg0

[peacey]

Hi @saghaulor,

Your configuration looks correct. When you say there's no outbound traffic from a client on br666, have you tested if it's just DNS resolution or even pinging IPs?

Can you try to ping 1.1.1.1 from a client on br666 after you bring the VPN up?

[saghaulor]

@peacey

Thanks for taking a look at my configs. I just tested on the device and it does seem to be a DNS issue. In a browser I loaded https://1.1.1.1 just fine, but https://ifconfig.co would not resolve. I'm not sure what the problem is, especially given that you said that my configs look correct.

Is there anything else that I can try to address the issue?

Thanks again for the help!

[peacey]

Are you able to ping the pihole IP from the br666 client when the VPN is enabled?

The issue is likely pihole itself is blocking inter-VLAN requests that are more than one hop away. In pihole web admin, go to Settings -> DNS -> under Inyergace Settings click "Permit all origins". Save then see if DNS is working from br666 clients on VPN.

[saghaulor]

Thanks again for your help @peacey

I changed the setting on my pi-hole to "Permit all origins" as you suggested. DNS is still not resolving for clients on br666. I also made sure to turn on standard mode instead of guest mode on br666, and made sure to disable a firewall rule that I had created to block LAN traffic from br666 to br0. It could still not resolve DNS.

I also verified that clients on br666 were pointing to the pi-hole for DNS.

I'm really at a loss for what could be the problem.

[saghaulor]

I changed the vpn.conf to:

DNS_IPV4_IP="1.1.1.1"
DNS_IPV4_PORT=53
DNS_IPV4_INTERFACE=""

It still didn't work.

Of note, when I try to use the VPN providers DNS in the wg0.conf, I receive an error.

root@UDMPRO:/etc/split-vpn/wireguard/vpn# wg-quick up ./wg0.conf
Warning: `/data/split-vpn/wireguard/vpn/wg0.conf' is world accessible
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.2.0.2/32 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] resolvconf -a wg0 -m 0 -x
/usr/bin/wg-quick: line 32: resolvconf: command not found
[#] ip link delete dev wg0
root@UDMPRO:/etc/split-vpn/wireguard/vpn#

I'm guessing this is because resolvconf can't be found?

[peacey]

@saghaulor Your wireguard didn't start correctly (it ended up exiting due a configuration error and then reverting the changes). And the script didn't even start because of that.

You shouldn't have the resolvconf line in your wg0.conf and the, can you please show me your wg0.conf?

[saghaulor]

Thanks for looking again. I'm aware the VPN didn't start. I was trying anything I could think to figure out how to get the DNS to resolve. So I changed the config to use the DNS entry that was in the original config from the VPN provider. This is what I used when I received the error.

[Interface]
# Key for udm-pro
# Bouncing = 11
# NetShield = 1
# Moderate NAT = off
# NAT-PMP (Port Forwarding) = off
# VPN Accelerator = on
PrivateKey = XxX=
Address = 10.2.0.2/32
DNS = 10.2.0.1
PostUp = sh /etc/split-vpn/vpn/updown.sh %i up
PreDown = sh /etc/split-vpn/vpn/updown.sh %i down
Table = 101

[Peer]
# BR#45
PublicKey = xXx=
AllowedIPs = 0.0.0.0/1,128.0.0.0/1,::/1,8000::/1
Endpoint = 149.102.251.97:51820

[peacey]

Oh I see. Sorry I was confused since I forgot I saw config from the first post. Yes that DNS line should be commented out or removed. Can you also please set DNS_IPV4_IP to empty (DNS_IPV4_IP=) in your vpn.conf just so we can get DNS working first before trying to force it.

Also the br666 network must not be a guest network (neither in LAN settings or WiFi if using a WiFi SSID) and cannot have any content filtering enabled on it in the settings. The UDM will try to reroute the DNS itself and mess things up if you enable that on the network.

Do you have any client on br666 you can run ping and dig from so we can do some testing to a DNS IP (maybe a Mac or Linux host)?

It would be ideal if you could run the following tests on a br666 client when the VPN is enabled:

ping 1.1.1.1
dig @1.1.1.1 google.com A
ping 192.168.1.94
dig @192.168.1.94 google.com A

This way we can see if you can ping the IP and then do a DNS lookup. At the least DNS to 1.1.1.1 should be working.

[saghaulor]

Thanks again for the help.

I verified that content-filtering is disabled on br666. I also changed the DNS_IPV4_IP entry to be empty.

I attempted the ping and dig requests that you mentioned on a device connected to br666.

✗ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=58 time=182.681 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=58 time=181.479 ms
^C
--- 1.1.1.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 181.479/182.080/182.681/0.601 ms


✗ dig @1.1.1.1 google.com A

; <<>> DiG 9.10.6 <<>> @1.1.1.1 google.com A
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached


✗ ping 192.168.1.94
PING 192.168.1.94 (192.168.1.94): 56 data bytes
64 bytes from 192.168.1.94: icmp_seq=0 ttl=64 time=2.272 ms
64 bytes from 192.168.1.94: icmp_seq=1 ttl=64 time=28.682 ms
^C
--- 192.168.1.94 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.272/15.477/28.682/13.205 ms


✗ dig @192.168.1.94 google.com A

; <<>> DiG 9.10.6 <<>> @192.168.1.94 google.com A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3200
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;google.com.			IN	A

;; ANSWER SECTION:
google.com.		175	IN	A	142.251.32.46

;; Query time: 47 msec
;; SERVER: 192.168.1.94#53(192.168.1.94)
;; WHEN: Sun May 28 09:52:18 PDT 2023
;; MSG SIZE  rcvd: 55

✗ ping google.com
ping: cannot resolve google.com: Unknown host

[grapeslush]

Did you ever get this figured out? I'm having this same exact problem.

[grapeslush]

I had to enable systemd-resolved to get it to work. Works beautifully now.