By Published on

Accidentally Securing DNS for Millions

Photo by JOSHUA COLEMAN on Unsplash

I setup NextDNS on a server. The idea was to test blocking all tracking when using a vpn. As far as the test, nextdns is quick and accurate. It's a nice system and works well. In order to get the vpn server to work correctly, I left the firewall fairly permissive. It all worked well for months. I pay $5/mo and get a full vpn server I control and know what's being tracked. nextdns runs as a local daemon on the system and provides dns resolution for the host and anything connected to it. We'll return to that last statement in the sentence, in a bit.

Nextdns is great at DNS over HTTPS (DoH) and requesting DNSSEC validated responses. In nextdns, you can simply use it as a straight, secure dns resolver. Or through their control panel, you can block/allow domain resolution by name. If I want to block simplyevil.example.com, I add it to a list and dns resolves to NXDOMAIN, meaning the name doesn't exist. This works great for stopping tracking and security threats. If the client cannot resolve the domain, then there's nothing to track or visit. 

With a permissive firewall and open dns resolver sitting on the Internet, it was a few weeks before someone found it. They probably didn't know it was nextdns, but they knew it was an open resolver. I first noticed it while analyzing a tcpdump traffic capture for an odd routing issue with IPv6 and the vpn. The routing issue was resolved, but the sheer quantity of dns queries from all over the world intrigued me. As I started to dig through it, I setup a tcpdump just to capture dns queries from outside the server. There are thousands of IP addresses querying my server. Domains like 67b dot org, hitnslab dot cn, yahoo dot com, taobao, vk, vimeo, and various continents of pool.ntp.org. It seems nextdns defines a device by IP address if the nextdns software isn't installed. It's clear there are many devices behind some of the IPs. 

There are thousands of IP addresses in just my sample. Behind these IPs are more clients. Here are some of the devices as identified by nextdns.

Turns out, there are millions of clients making millions of queries. 

The 1.68 million number is just from the last 30 days before I setup a more restrictive firewall blocking all external dns queries. The nextdns configuration is locked down by adding every known dns blocklist. Not sure what the clients were expecting, but they sure got a ton of responses as NXDOMAIN. 

Ironically, one of the blocklists loaded into the nextdns config was for known and suspected botnets. Unfortunately, nextdns doesn't have an API for their analytics portal. I'd love to pull down all the data and do some more analysis on it. 

And that's how I accidentally provided secure and private DNS for 1.68 million machines on the internet.