Arch Linux Tailscale and sing-box Global Networking with DNS Stub

Problem

The archlinux host needed to satisfy both of these at the same time:

Tailscale must keep working normally
- tailnet connectivity must remain available
- *.ts.net / MagicDNS resolution must still work
sing-box must provide true global proxying
- not just HTTP_PROXY/HTTPS_PROXY/ALL_PROXY
- browser and normal system traffic should go through a tun-based global route

Initial symptoms during migration to global mode included:

sing-box startup failure after adding tun
Google failing in browser / curl even after tun appeared active
DNS resolution flipping between success, timeouts, and polluted answers
GitHub working while Google still failed
Tailscale DNS and system DNS behavior interfering with each other

Environment

Host:

Arch Linux
Tailscale installed and connected
sing-box installed as a systemd service
NetworkManager managing the main network interface

Relevant components:

tailscaled
systemd-resolved
NetworkManager
sing-box

Investigation

1. Verified the target machine

Tailscale status confirmed the online host was:

archlinux

The host was reachable over Tailscale SSH and had:

sing-box installed
sing-box.service enabled and running

2. Confirmed sing-box was initially not yet true global mode

The live config originally had only a local mixed inbound:

{
  "type": "mixed",
  "tag": "mixed-in",
  "listen": "127.0.0.1",
  "listen_port": 5780
}

That allowed CLI / app proxying when explicitly configured, but not full system interception.

3. Added a `tun` inbound

A tun inbound was added so traffic could be routed through:

singtun0
auto_route: true
strict_route: true

This was the right direction, but the first rollout failed.

First Failure: duplicate inbound tag

After applying the new config, sing-box failed with:

FATAL unmarshal merged config: duplicate inbound tag: tun-in

Root cause

The systemd service uses:

/usr/bin/sing-box -D /var/lib/sing-box -C /etc/sing-box run

-C /etc/sing-box means sing-box merges config files from the whole directory.

At the time, both of these existed:

/etc/sing-box/config.json
/etc/sing-box/sing-box-config-global.json

Both defined:

"tag": "tun-in"

Fix

Removed the duplicate effective config from the directory merge set so only one active config defined tun-in.

After that, sing-box.service started successfully and singtun0 came up.

Second Failure: TUN was active, but Google still failed

After the duplicate config issue was fixed:

sing-box.service was active
singtun0 existed
routing policy rules were present
logs showed traffic entering inbound/tun[tun-in]

So the global proxy path itself was active.

However, browser and curl still failed for Google.

Key observation

At different points:

curl https://github.com succeeded
curl https://www.google.com failed
DNS sometimes returned results, sometimes timed out
some answers looked suspicious / polluted

That meant the remaining problem was no longer “traffic not entering sing-box”, but rather:

DNS ownership was unclear
address selection was unstable
IPv6 behavior and node capability did not match

DNS Conflict Analysis

Tailscale was initially taking over system DNS

tailscale debug prefs showed:

"CorpDNS": true

And /etc/resolv.conf initially looked like:

nameserver 100.100.100.100
search taila6b1f7.ts.net

That meant Tailscale was acting as the system DNS authority.

This is often fine on its own, but in this setup it conflicted with:

sing-box TUN global interception
application DNS behavior
NetworkManager DNS defaults
the need to keep tailnet DNS separate from public DNS

tailscaled logs confirmed DNS forwarding trouble

Observed repeatedly:

dns udp query: waiting for response or error from [185.222.223.224 180.76.76.76]: context deadline exceeded

That indicated Tailscale itself was struggling to reach the configured upstream DNS servers under the new routing environment.

Fix: stop Tailscale from owning system DNS

Applied:

sudo tailscale up --accept-dns=false

Afterwards:

"CorpDNS": false

This preserved Tailscale connectivity while removing Tailscale’s role as global system resolver.

Third Failure: NetworkManager became the new system DNS owner

After disabling Tailscale DNS takeover, /etc/resolv.conf reverted to NetworkManager-managed DNS, specifically:

nameserver 185.222.223.224
nameserver 180.76.76.76

This still caused unreliable behavior for public name resolution in the new global-proxy environment.

At that stage:

Tailscale tailnet resolution still worked
public DNS remained unstable

Intermediate DNS Test

As a temporary validation step, /etc/resolv.conf was manually rewritten to:

nameserver 1.1.1.1
nameserver 8.8.8.8

This proved an important point:

DNS resolution itself could be stabilized
but Google still sometimes failed with TLS EOF after DNS was fixed

That showed DNS was not the last remaining issue.

Fourth Failure: IPv6 path broke Google

Once DNS was cleaner, the remaining failure was:

curl: (35) TLS connect error: error:0A000126:SSL routines::unexpected eof while reading

Further checks showed:

curl -4 https://www.google.com succeeded
curl -6 https://www.google.com failed

sing-box logs showed the decisive clue:

outbound connection to [2607:f8b0:4007:805::2004]:443
remote error: no IPv4 address available

Root cause

The selected proxy path was not reliably handling the IPv6 route chosen for Google.

So even after DNS was healthy:

applications still preferred IPv6 in some cases
the chosen proxy node/path was fine for IPv4
but not for this IPv6 traffic pattern

Final Architecture

The final working architecture was:

1. Tailscale keeps tailnet connectivity

Tailscale remains responsible for:

tailnet transport
*.ts.net / MagicDNS
Tailscale peer resolution

But not for system-wide public DNS.

2. systemd-resolved becomes the local DNS stub

Enabled systemd-resolved and changed /etc/resolv.conf to the stub mode:

nameserver 127.0.0.53

This gives normal applications one stable local resolver.

3. Global public DNS goes to clean resolvers

Configured global DNS through resolved as:

1.1.1.1
8.8.8.8

with global domain routing:

~.

4. tailscale0 gets split DNS only

Configured tailscale0 with:

DNS server: 100.100.100.100
domains:
- ~taila6b1f7.ts.net
- ~ts.net

That means only tailnet domains are resolved by Tailscale.

5. IPv4 is preferred system-wide

Added to /etc/gai.conf:

precedence ::ffff:0:0/96  100

This causes address selection to prefer IPv4-mapped candidates before generic IPv6 ones.

That was necessary because the current sing-box outbound path handled IPv4 traffic correctly but was unreliable for the Google IPv6 path.

6. sing-box selector switched to `auto`

Using the Clash controller API, the active selector was changed from a manually pinned node to:

select -> auto

This let urltest select a more stable current route.

Observed successful egress later came through a different outbound than the original manual selection.

Effective End State

DNS

systemd-resolved status:

active
/etc/resolv.conf is the stub symlink / stub file content
global DNS:
- 1.1.1.1
- 8.8.8.8
tailscale0 DNS:
- 100.100.100.100
tailscale0 domains:
- ~taila6b1f7.ts.net
- ~ts.net

Tailscale

Verified:

getent hosts archlinux.taila6b1f7.ts.net

Result: success.

Public network through sing-box global TUN

Verified:

curl -4 -I https://www.google.com
curl -I https://www.google.com
curl -I https://github.com

Results:

Google over IPv4: HTTP/2 200
Google with default address selection: HTTP/2 200
GitHub: HTTP/2 200

Commands Used / Worth Remembering

Check Tailscale DNS takeover

tailscale debug prefs | grep CorpDNS

Disable Tailscale system DNS takeover

sudo tailscale up --accept-dns=false

Check resolver mode

cat /etc/resolv.conf
resolvectl status

Test tailnet DNS

getent hosts archlinux.taila6b1f7.ts.net

Test public network

curl -I https://github.com
curl -I https://www.google.com
curl -4 -I https://www.google.com
curl -6 -I https://www.google.com

Check current sing-box selector

curl http://127.0.0.1:9090/proxies/select
curl http://127.0.0.1:9090/proxies/auto

Lessons Learned

Tailscale and sing-box can coexist cleanly, but DNS ownership must be explicit.
Do not let multiple components compete to be the default resolver.
A local DNS stub is the cleanest place to implement split DNS.
When Google fails but GitHub works, check IPv6 selection before assuming DNS is still broken.
With sing-box using -C /etc/sing-box, extra JSON files in that directory are not harmless backups; they are part of the merged config unless moved out of the load set.

Final Summary

The final stable arrangement is:

Tailscale handles tailnet networking and *.ts.net
systemd-resolved handles local DNS stub and split DNS routing
sing-box handles full-system public traffic via TUN
IPv4 preference avoids broken IPv6 paths on the current proxy node set

That combination restored both:

Tailscale functionality
global VPN / proxy functionality

without forcing one system to break the other.

Explorer

Table of Contents

Arch Linux Tailscale and sing-box Global Networking with DNS Stub

Problem

Environment

Investigation

1. Verified the target machine

2. Confirmed sing-box was initially not yet true global mode

3. Added a tun inbound

First Failure: duplicate inbound tag

Root cause

Fix

Second Failure: TUN was active, but Google still failed

Key observation

DNS Conflict Analysis

Tailscale was initially taking over system DNS

tailscaled logs confirmed DNS forwarding trouble

Fix: stop Tailscale from owning system DNS

Third Failure: NetworkManager became the new system DNS owner

Intermediate DNS Test

Fourth Failure: IPv6 path broke Google

Root cause

Final Architecture

1. Tailscale keeps tailnet connectivity

2. systemd-resolved becomes the local DNS stub

3. Global public DNS goes to clean resolvers

4. tailscale0 gets split DNS only

5. IPv4 is preferred system-wide

6. sing-box selector switched to auto

Effective End State

DNS

Tailscale

Public network through sing-box global TUN

Commands Used / Worth Remembering

Check Tailscale DNS takeover

Disable Tailscale system DNS takeover

Check resolver mode

Test tailnet DNS

Test public network

Check current sing-box selector

Lessons Learned

Final Summary

Table of Contents

Backlinks

Graph View

3. Added a `tun` inbound

6. sing-box selector switched to `auto`