Cricket on DNS

25 Years of .COM

2010-03-14T08:03:38Z

According to Wired, Symbolics.COM was registered on March 15, 1985. Symbolics.COM was the very first subdomain of COM, making today the silver anniversary of, well, something. The first delegation from .COM, I guess. Since then, there have been tens of millions more, of course, so the very first ought to be significant.

I had the privilege of managing the 9th-oldest subdomain of .COM, HP.COM, for several years back in the late 1980s and early 1990s. That job set me on the path I've been on for the last twenty-something years, and for that I'm very grateful.

What else has happened during those 25 years? Countless versions of the BIND name server were released, from BIND 4.8 to the current 9.7.0. For that, we owe the Internet Systems Consortium an enormous debt of gratitude. BIND still powers, by our last measure, almost 75% of the authoritative name servers serving subzones of .COM, .NET and .ORG. Commercial ventures with that kind of market share make people rich; the folks at ISC chose instead to pursue the nobler goal of producing the reference implementation of the Domain Name System, thereby facilitating the remarkable growth and success of the Internet.

All around them, though, DNS commercialized. Custody of .COM and the other generic top-level domains (gTLDs) moved from staid monopolist InterNIC to (one-time) .COM darling VeriSign and a legion of registrars. Domain names - particularly those under .COM - became hot virtual property. Entrepreneurs partnered with sovereign states that happened to own country-code top-level domains with useful mnemonics (such as .TV and .FM) to offer competition to the gTLDs.

We also saw threats to DNS infrastructure evolve from relatively straightforward exploits of implementation flaws to sophisticated attacks against fundamental limitations in the design of DNS. Early exploits, including the Kashpureff cache poisoning attack and the worm that used the TSIG buffer overrun to infect name servers, were quickly addressed with improved code. More recently, the Kaminsky vulnerability capitalized on weaknesses in the Domain Name System itself, and we've only managed to buy time by implementing clever mechanisms to make the exploit more difficult to carry out.

We now face the most daunting - but also the most pressing - upgrade to the Domain Name System we've ever undertaken: The addition of long-needed cryptographic security with DNSSEC, the DNS Security Extensions. If successful, DNSSEC offers the promise of a secure, ubiquitous distributed naming service, which could act as the foundation for securing other Internet services, including email and the web. But while DNSSEC's adoption is accelerating, there's still a very long way to go, and numerous doubters and detractors to win over.

Despite this challenge, though, we ought to acknowledge .COM and DNS as the extraordinary successes they are. .COM has grown from that single delegation into the biggest domain in the world's largest distributed database system; and DNS has kept pace with both the jaw-dropping expansion of the Internet and the transformation of services from Telnet to telephony, from VT100s to video streaming.

So, to .COM and DNS, a very happy 25th, and (at the risk of sounding self-serving), many happy returns!

Paul Vixie on DNSSEC vs. DNSCurve

2010-03-12T10:03:06Z

When I wrote my recent blog posting on DNSSEC vs. DNSCurve, I wasn't aware that Paul Vixie had already written his own blog entry on the same subject. It also explains ISC's stance on DNSCurve. Recommended reading.

Slides from our Recent DNSSEC Webinar

2010-03-11T10:03:40Z

If you're interested in the slides from the recent Infoblox/F5 DNSSEC webinar with Dan Kaminsky, Nate Meyer and Scott Rose, they're available here. Thanks to everyone who listened in!

If you're having trouble with the link above, here's a PDF of the slides.

DNSSEC vs. DNSCurve

2010-02-27T10:02:20Z

With the recent announcement that OpenDNS will support DNSCurve, I've begun hearing more questions about it. In particular, people wonder whether DNSCurve is a viable alternative to DNSSEC. They've generally heard that DNSCurve is simpler to set up than DNSSEC and involves less overhead.

Unfortunately, DNSCurve isn't an alternative to DNSSEC - although it could conceivably complement DNSSEC, in ways I'll discuss.

You can think of DNSCurve as a clever scheme for bootstrapping secure communications between name servers, a bit like automatically setting up a TSIG key for use between any two name servers. To communicate securely, a recursive name server can use public key information embedded in the domain name of an authoritative name server to derive a session key. The communications, then, can be encrypted for privacy, authenticated and integrity-checked.

One thing DNSCurve doesn't do is protect data sitting in a name server's cache. It secures the channel between two name servers, but if your name server retrieves records from the cache of an untrusted recursive name server, DNSCurve gives it no way to authenticate those records. DNSSEC, of course, does, as those records would be accompanied by an RRSIG record that digitally signed them.

Likewise, if a hacker compromises an authorized authoritative name server, DNSCurve can't protect name servers that query that name server: The responses continue to validate. With DNSSEC - assuming the hacker didn't gain access to the zone's private key - the hacker wouldn't be able to produce valid signatures for the zone data.

Finally, the difference between the amount of support for DNSCurve and DNSSEC is startling. OpenDNS's implementation of DNSCurve is, by their own admission, the only existing DNSCurve implementation so far. Despite their announcement of support for DNSCurve, their recursive name server currently doesn't use DNSCurve to secure any communications, because there are no authoritative name servers that speak DNSCurve. Today, an interested, committed DNS admin would have to write his own implementation of DNSCurve to deploy it.

DNSSEC is supported in the latest versions of BIND, the Microsoft DNS Server, NSD, and Unbound, as well as in many products based on those name servers.

But what frustrates many of us in the DNS community isn't what DNSCurve doesn't do, it's what it does. Specifically, DNSCurve is distracting DNS administrators from the important work of deploying DNSSEC. We've labored for 15 years to address misgivings about the DNSSEC specifications, provide a choice of implementations that support DNSSEC, pressure the administrators of the upper levels of the Internet's namespace to take on the hard work of signing their zones, and rally legions of DNS administrators. And, just as we gain traction and begin to make headway, DNSCurve comes along, claiming to offer an alternative, when in fact it solves different problems.

In an ideal world, DNSCurve could act as a powerful complement to DNSSEC, providing channel authentication and privacy that DNSSEC alone doesn't offer. We'd finish the roll out of DNSSEC, begin deploying DNSCurve, and enjoy the best of both. But DNSCurve's backers seem committed to derailing DNSSEC's progress in favor of DNSCurve. If they succeed, the Internet will be the worse off for it.

Securing DNSSEC's "Last Mile"

2010-02-11T09:02:50Z

I feel like at least half of my postings to this blog have been about DNSSEC (and for those of you uninterested in DNSSEC, I'm sorry). But one DNSSEC-related topic I haven't brought up is the "last mile."

In DNSSEC, the "last mile" refers to communications between the stub resolver and the recursive name server. The stub resolver is the piece of the Domain Name System that resides on nearly every computer and translates an application's request for data (say the address of www.infoblox.com) into a DNS query, and then sends that query to one or more name servers. The recursive name server receives a resolver's query, examines its cache for the answer, and if it doesn't find the answer there, may need to send one or more queries to remote name servers.

DNSSEC does describe some signaling between the stub resolver and the recursive name server. In particular, stub resolvers can set the CD bit in a query. The CD (Checking Disabled) bit tells the recursive name server not to validate DNSSEC-signed data on behalf of the resolver, but to return the various DNSSEC resource record types, such as RRSIG and DNSKEY RRs, so that the resolver can perform the validation. (It's also sometimes used when troubleshooting DNSSEC problems.)

But no common stub resolvers implement their own DNSSEC validation, so CD is rarely used.

If CD isn't used, the recursive name server will signal to the stub resolver that it has validated DNSSEC-signed data using the AD, or "Authenticated Data," bit. That's all very well and good, but the problem is that most communication between stub resolvers and recursive name servers is itself unauthenticated. A hacker who saw a resolver send a query to a recursive name server for a domain name in a signed zone could spoof a response to that stub resolver containing bogus records, and simply assert that the answer had been validated by setting the AD bit.

In firewalled corporate environments, this may be less of a concern than for home users. A company's firewall shouldn't allow DNS responses from arbitrary Internet addresses to internal stub resolvers, so the spoofing would have to happen internally. But most Internet-connected home computers either aren't protected with a firewall or use a firewall that permits responses from name servers on the Internet (because that's where their ISP's name servers are).

What's needed is some sort of authentication for the channel between the stub resolver and the recursive name server--as lightweight as possible, naturally, for the sake of busy recursive name servers. TSIG would seem like an ideal choice: It's lightweight, and provides both authentication and integrity checking. But TSIG requires configuration of a shared secret, so it suffers from a nasty key distribution problem. If your recursive name server handles hundreds or thousands of stub resolvers, you'd need to configure a shared secret with each of them.

Microsoft's GSS-TSIG would seem to solve that problem, at least for Active Directory environments. GSS-TSIG uses shared secrets already stored in AD to bootstrap the negotiation of a shared session key. Resolvers that run on computers in an AD domain, then, could automatically determine a key to use when querying a name server in that domain, and could validate its responses with the same key. Yet, inexplicably, Microsoft recommends configuring IPSec between resolvers and their name servers instead.

What's the answer to this problem? Lobby your vendor for a stub resolver that supports TSIG for queries and responses! Now if we could just come up with a catchy way to put that on a bumper sticker....

Quantifying DNSSEC Overhead, Part 2

2010-02-01T02:02:22Z

Last time, I compared the number and size of the response messages involved in resolving a record in an unsigned zone to those involved in resolving a record in a signed zone under a signed TLD. This time, I want to look at the actual computation involved.

This isn't really a comparison, of course, because in the case of an unsigned zone, there's no heavy computing involved: The name server simply reads responses from the network and unmarshals their content into discrete resource records--simple! In the case of a signed zone under a signed TLD, there's lots of work to do.

Let's start with a count of the unique records returned and their types. We need to be a little careful here, though, since sometimes identical records are returned in more than one response. Here's our census:

1. 6 NS RRs for org

2. 6 corresponding A RRs (the org name servers' addresses)

3. 6 corresponding AAAA RRs (the org name servers' IPv6 addresses)

4. 4 NS RRs for isc.org

5. 2 corresponding DS RRs

6. 1 RRSIG RR (covering the DS RRs)

7. 3 corresponding A RRs (the isc.org name servers' addresses)

8. 1 A RR (the answer!)

9. 2 RRSIG RRs (covering the A RR)

10. 4 NS RRs for isc.org (same as in 4.)

11. 2 RRSIG RRs (covering the NS RRs)

12. 3 corresponding A RRs (same as in 7.)

13. 1 corresponding AAAA RR

14. 8 RRSIG RRs, covering the name servers' 3 A RRs and 1 AAAA RR)

15. 4 DNSKEY RRs for isc.org

16. 4 RRSIG RRs (covering the DNSKEY RRs)

17. 4 NS RRs (same as in 4.)

18. 2 RRSIG RRs (same as in 11.)

19. 3 corresponding A RRs (same as in 12.)

20. 1 corresponding AAAA RR (same as in 13.)

21. 8 RRSIG RRs (same as in 14.)

22. 4 DNSKEY RRs for org

23. 2 RRSIG RRs (covering the DNSKEY RRs)

24. 6 NS RRs for org (same as in 1.)

25. 1 RRSIG RR (covering the NS RRs)

26. 2 corresponding A RRs (subset of 2.)

27. 2 corresponding AAAA RRs (subset of 3.)

Whew! Now, to keep things simple, let's just say:

Each DS RR requires one cryptographic hashing operation to check (because you run the corresponding DNSKEY RR though SHA-1 or SHA-256 to compare it to the value in the DS record)
Each RRSIG RR requires one hashing operation and one asymmetric decryption operation to validate (because you run the RRset you're validating through SHA-1 and then decrypt the ostensible signature from the RRSIG RR and make sure they match)

That means that, in total, we'll need 22 hashing operations and an even 20 asymmetric decryption operations to completely validate www.isc.org's A RR and its chain of trust. Yikes! (This is actually a worst case, since it assumes that we validate all the RRSIG records, when in fact we likely don't need to. For example, checking the signature of any one isc.org DNSKEY RR covering the full set of keys is enough to validate all four keys.)

Luckily, our name server will cache the results of the validation. If we were to subsequently look up ftp.isc.org's A RR, the name server would only have a single hashing operation and a single decryption to do, which now doesn't seem bad at all.

This is the reason I've recommended that organizations deploying DNSSEC watch the CPU load on their recursive name servers carefully: As the proportion of responses that are signed increases, so will the load on their recursors.

Ultimately, though, the ever-increasing speed of processors and networks will trump the burden DNSSEC adds. Years from now - assuming DNSSEC becomes widely deployed - we'll look back at our concerns about the overhead of DNSSEC and chuckle. I hope.

Quantifying DNSSEC Overhead

2010-01-14T10:01:46Z

I realized last week that I'd never actually traced all the queries sent and responses received by a recursive name server resolving a domain name in a zone signed with DNSSEC. I decided to trace the recursive resolution of an RRset in a signed top-level domain, since I wanted to see the "chain of trust" in action. I knew .org was signed and figured isc.org (the Internet Systems Consortium's domain) would probably already have a DS (Delegation Signer) record. Sure enough, a quick query using dig verified that:

% dig ds isc.org. +dnssec

I chose www.isc.org/IN/A as the RRset I'd look up.

I started by clearing the cache on my BIND name server (running BIND 9.7.0rc1), so that I could watch my name server query the .org name servers and see the records returned:

% sudo rndc flush

Then I fired up tcpdump to capture DNS traffic:

% sudo tcpdump -w www.isc.org.cap port 53

And then sent the query:

% dig www.isc.org. +dnssec

This turned out not to capture enough data when the responses were long, so I refined my tcpdump command by specifying the option "-s 4096". Then I found out I wasn't catching all fragments of really long responses, so I dumped the "port 53" argument (since fragments don't carry port information) and used Wireshark to filter out all the non-DNS traffic. Now I actually had a complete record of the queries my name server sent and the responses it received.

The results surprised me: I saw five queries and five resulting responses, when resolution of a domain name from an unsigned zone, such as www.cert.org, would usually require at most just three roundtrips. And, of course, the signed responses were much larger than most unsigned responses.

Here's a short comparison of resolution of www.isc.org and www.cert.org:

This isn't a perfect comparison--isc.org has four authoritative name servers, while cert.org only has two, which would make isc.org responses larger even without DNSSEC--but it gives you a sense of how much more traffic DNSSEC will use: In this case, over 6.5 times as much!

Now, not all resolutions of RRsets in signed zones will require this many roundtrips: Once our name server has validated the signature of www.isc.org's A record, as well as the public keys for isc.org and org, it'll cache those results. Still, even the individual response that contained www.isc.org's A record (Response 3) was a whopping 17 times as large as the response containing the address of www.cert.org, and that record's TTL is just ten minutes, so we might have to look it up frequently.

Of course, the workload involved in sending and receiving those UDP datagrams pales in comparison with all the hashing and decryption that needs to be done to validate the records received. Here's what those five responses contained:

Response 1: 6 NS records, 6 A records, 6 AAAA records
Response 2: 4 NS records, 2 DS records, 1 RRSIG record
Response 3: 4 A records, 1 AAAA record, 4 NS records, 12 RRSIG records
Response 4: 4 DNSKEY records, 14 RRSIG records, 4 NS records, 3 A records, 1 AAAA record,
Response 5: 4 DNSKEY records, 2 RRSIG records, 6 NS records, 1 RRSIG record, 2 A records, 2 AAAA records

Now, some of these are duplicates, and some RRSIG records aren't needed in the validation process, but each RRSIG record requires one cryptographic hashing operation and one asymmetric encryption operation to check, and each DS record requires one cryptographic hashing operation to check. Compare that with the resolution of www.cert.org, which would require all of, er, zero hashing operations and no asymmetric encryption operations.

As a consequence, we should expect to see CPU overhead go up dramatically as our name servers start handling more DNSSEC-signed responses. After you configure validation, you'd be well advised to begin monitoring your name server's CPU utilization, too, so you aren't caught by surprise when your recursive name server pegs the processor.

My Predictions for DNS Developments in 2010

2009-12-17T03:12:57Z

'Tis the season for new year's predictions, and my blog will be no exception. Some of these predictions are fairly safe bets, like the signing of the root zone and the introduction of internationalized top-level domains. Others are more speculative. Here they are, in no particular order:

The signing of the root zone, slated for July, will spur adoption of DNSSEC. A signed root zone will make configuring DNSSEC validation much easier: Administrators will only need to configure a single key in order to validate signed data in any of the currently-signed top-level zones. But the signed root zone will also point out shortcomings in DNSSEC support. In particular, ICANN and VeriSign have announced that they'll use RSA/SHA-256 to sign the root, and many DNSSEC implementations don't yet support RSA/SHA-256. Expect a scramble among vendors.
We'll see some nasty cache poisoning attacks against stragglers who haven't upgraded their recursive name servers to use random query ports or other anti-spoofing mechanisms. We've already seen some of these this year (see my "DNS Year in Review" posting); hackers have all the tools they need to mount more, and it costs them basically nothing to keep vulnerable recursive name servers constantly under attack.
As DNSSEC and IPv6 roll out, we'll see more organizations driven to outsource DNS. Without good tools, managing a DNSSEC-signed zone is almost embarrassingly complex (though ISC is doing its darnedest to make it easier). Many smaller organizations will throw in the towel and pay a service provider to do it for them. On the recursive side, the hassle of keeping name servers patched against cache poisoning and other threats will induce more organizations to use services like OpenDNS and Google's Public DNS.
We'll begin seeing the addition of internationalized country code top-level domains (ccTLDs). ICANN is eager to expand the top-level of the Internet's namespace and has instituted a new, fast-track process for applying for new internationalized ccTLDs. Expect to see the first of these introduced in 2010. To some of us, these top-level domains may be incomprehensible, either because we'll see them rendered in their encoded form, as opaque strings beginning with "xn--," or because we don't understand the script they're written in. But to others, these new ccTLDs will provide a more natural, understandable way to access Internet resources.

Let's see if I can remember to tick these off as they happen over the course of the year. Assuming that any of them happen, that is.

Hope you all had a happy, relaxed holiday season!

WALSYIB

2009-12-16T12:12:08Z

A system administrator I knew at HP Labs, Mike Rodriquez, named his personal workstation "walstib." Mike explained that it was an acronym for "What A Long, Strange Trip It's Been," which, he said, was a kind of motto among Deadheads. (I gather it's a line from one of the many indistinguishable Grateful Dead songs. Sorry, Mike.)

So "WALSYIB" is my acronym for "What A Long, Strange Year It's Been." (And yes, I realize that I used a similar title for a previous blog post.) 2009 was a productive year: We made more progress in deploying DNSSEC in the last 12 months than in the previous 10 years. But we saw more attacks on DNS infrastructure, including cache poisoning attacks in the wild. And we saw the discovery (and subsequent patching) of more vulnerabilities in BIND.

Here, then, is the DNS year in review:

January: My old friend Matt Larson and I announce The Ask Mr. DNS Podcast. (The high point of the DNS year, and it's only January!) The ISP UkrTeleGroup, which hosted many malicious open recursive name servers, is "de-peered" by their uplink provider.
February: IANA introduces the Interim Trust Anchor Repository (ITAR), a temporary clearinghouse for the trust anchors of top-level zones. VeriSign announces that they'll sign all top-level zones they operate in the next 24 months. This includes .com and .net. Matt Larson later refines the timeline: .net will be signed in 2010, .com in 2011. .gov, the U.S. government's top-level domain, is signed.
March: .th (Thailand) becomes the first DNSSEC-signed zone in Asia.
April: Hackers use an SQL injection attack against the main registrar in Puerto Rico to redirect users of the local versions of major web sites, while others use the same vector at a New Zealand registrar to redirect users of MSN's New Zealand web site, Sony, HSBC and others. A cache poisoning attack in Brazil causes a banking Trojan to be served to the customers of a broadband carrier.
May: A DDoS attack against a Chinese registrar takes out their name servers, and collateral damage caused by the response to the attack causes problems in many Chinese provinces.
June: PIR begins signing .org, the first generic top-level domain (gTLD) to be signed, and the largest top-level zone to be signed.
July: Eircom's customers (Eircom is the largest broadband provider in Ireland) fall victim to a DNS cache poisoning attack which incidentally causes a DNS outage. BIND is patched for a DDoS vulnerability in which a specially crafted dynamic update can crash the name server
September: .na (Namibia) becomes the first top-level zone in Africa to be signed. SWITCH begins a DNSSEC trial for the .ch (Switzerland) and .li (Liechtenstein) top-level zones. Niue's .nu is signed.
October: ICANN and VeriSign announce details of their arrangement to jointly administer the signed root zone, as well as the timetable for signing the root. A flaw in the generation of the .se zone data causes a massive outage of Swedish domain names. Infoblox and The Measurement Factory release the results of their fifth annual DNS Survey of the Internet's DNS infrastructure. Michael Sinatra (of UC Berkeley--w00t!) discovers a flaw in BIND's processing of DNSSEC-signed responses.
November: Turkmenistan's .tm is signed.
December: Google announces their Public DNS service, offering free recursive name service. Neustar signs the .us zone.

What do we have to look forward to in 2010? The signing of the root zone by July 1, .net by the end of the year, and likely many more top-level zones. Internal U.S. government zones signed by the summer. Undoubtedly more vulnerabilities and more attacks, too. But if we see the amount of progress next year that we've already seen this year, the Internet will definitely be a safer place.

On Neustar's DNS Real-time Directory

2009-12-14T02:12:06Z

Last week, Neustar announced that they've added an interesting new feature to their zone hosting service, called the DNS Real-time Directory. In an effort to address some of the shortcomings of DNS's loose coherence, Neustar is publishing changes to the zones they host on their constellation of authoritative name servers through Amazon's EC2 service. Subscribers, including OpenDNS, are notified of those changes and can remove outdated resource records from their recursive name servers' caches in response. This would help avoid the recent mess caused by the accidental appending of an extra ".SE" to domain names in Sweden's .SE zone: While the problem was fixed on the authoritative name servers right away, the operational effects lingered for up to a day--the TTL on resource records in the .SE zone, and hence the maximum time recursive name servers would cache the bogus records. Had Neustar been hosting .SE and had the DNS Real-time Directory up and running, presumably they could have invalidated the cached, bogus records--on OpenDNS's name servers, at least.

Neustar says it's working with other providers of recursive name service, too. We'll see who that includes. Google's Public DNS service and DynDNS seem like obvious candidates, though DynDNS also runs services that compete with Neustar's.

I'm also interested to see if Neustar is willing to open the publishing end of the service up to rivals in the business of hosting authoritative zones. While that would reduce the competitive advantage they gain from the Real-time Directory, it could hugely increase the incentive for providers of recursive DNS services and products to support their fledgling feature. That would be a bold move, indeed.

On Google's Public DNS Service

2009-12-06T10:12:11Z

With the press frenzy over Google's announcement of their Public DNS Service, you'd think that they'd announced that they had taken over running the root name servers. At the very least, the press is presenting it as a power grab, a way for Google to insert themselves into still more Internet transactions. Others have suggested that Google's looking to replace the Internet's DNS infrastructure entirely, and possibly introduce new, private top-level domains. (I'm skeptical about this.)

What is Google really doing? Put simply, they're offering recursive name service from their cloud, based on their own implementation of a recursive name server. From the writeup, they included nearly every anti-spoofing mechanism known to man in their name server, which means it should be highly resistant to cache poisoning. I say "nearly" because they don't support the DNS Security Extensions yet, so they can't take advantage of the long-term solution to cache poisoning, which is being deployed either "soon" or "now," depending on what part of the namespace you live in. They also pre-fetch information about popular domain names, which should provide better performance than your average recursive name server.

They pointedly don't do NXDOMAIN redirection, which is intercepting responses that would normally return a "No such domain name" reply and replacing them with the address of a web server. Once you're there, the web server typically tries to guess which domain name you meant to type, and probably displays some ads, too. Companies including OpenDNS use this technique, ostensibly to try to help users find what they're after, but also to generate cash to fund their operations.

Google stops short of calling NXDOMAIN redirection evil, but they plainly don't like it. Others have reservations about NXDOMAIN redirection, too: Many Internet services count on DNS to return those "No such domain name" responses. For example, mail servers often check to see whether the domain name used in an email address really exists to help decide whether the email is spam or not. But NXDOMAIN redirection makes every domain name look like it exists.

Does that mean that you should dump OpenDNS and move to Google's Public DNS service? That depends on your needs and your priorities. OpenDNS does more than NXDOMAIN redirection: They maintain a dynamic list of domain names associated with different kinds of malicious (or simply unproductive) activity, and if you inadvertently try to look up one of these, they'll head you off. And they provide the ability to customize this behavior and choose, category by category, which types of domain names you don't want your users to resolve. (For their part, Google's blog suggests they disapprove of this kind of blacklisting, too.) Plus OpenDNS runs the same kind of anycast infrastructure Google does, and they have their own tricks for improving performance.

If you don't need or want NXDOMAIN redirection or OpenDNS's blacklisting capabilities, why wouldn't you use Google's service? Well, there's no SLA, first of all - Google's refreshingly candid about that. And there's no such thing as a free lunch: Google will undoubtedly analyze your DNS "query stream" to their advantage, though they've published a data privacy policy that says that they'll anonymize the record of your queries before they throw it in their big hopper. But heck, if you use your ISP's name servers, you're giving them your query stream, too, and your ISP probably has no published privacy policy.

So the upshot is that Google looks like another worthy entrant into the world of cloud-based recursive name service, but it's by no means a juggernaut. I think it's great that they offer a functionally and philosophically different flavor of DNS to users - choice is generally good, after all - but I also think there are lots of folks who find what OpenDNS does useful. Even the purists who are morally opposed to NXDOMAIN redirection might be uneasy using Google's name servers, since those same purists are more likely to worry about the privacy of their query stream.

And we should keep in mind that all of this is really a tempest in a very modest teapot, since the user base we're so worried about consists only of the small percentage of people capable and motivated enough to reconfigure their computers' resolvers to use name servers other than the defaults.

Various Varieties of Vulnerabilities

2009-11-25T10:11:38Z

Just yesterday, ISC announced the release of several versions of BIND to address a new vulnerability. The vulnerability could allow unsigned data to be cached on a recursive name server configured to perform DNSSEC validation.

While that's alarming, it's not a systemic problem with DNSSEC; it's simply a flaw in BIND's implementation of DNSSEC. (How could it be anything else if it was addressed by releasing new versions?) Implementations of the latest incarnation of DNSSEC are still relatively new, so it should come as no surprise that we're still finding flaws. (I'm proud to say that this particular defect was found by Michael Sinatra, who works for my alma mater, Berkeley.)

If you're curious as to the nature of the vulnerability, it works like this: A querier sends a recursive query to our validating name server with the DO and CD bits set. The DO (DNSSEC OK) bit says, "Please return DNSSEC records," while the CD (Checking Disabled) bit says, "You don't need to validate any DNSSEC records in the response before sending them to me, because I'm going to do that myself." The problem is that our validating name server would add records from the response's Additional Data section to its cache without validating them. That's not the behavior prescribed by the RFCs; that data should be passed through to the querier but only added to the cache if it validates.

ISC's advisory suggests that restricting access to recursion on the vulnerable name server is a workaround, but it's not a complete fix, since it's possible an authorized querier could send a query that would induce a vulnerable name server to cache unvalidated records, or that someone with access to an authorized querier acting in collusion with a hacker could do the same. If you do DNSSEC validation, the real solution is to upgrade.

Giving Thanks for Good News in DNS

2009-11-19T01:11:01Z

Most of the results of our recent DNS Survey were pretty scary, especially the news that nearly 80% of the name servers we found in our sweep of 5% of the Internet's address space were open to recursion. But the results contained some good news, too, and for that we should be thankful.

- Of those open recursors we found, almost 90% used source-port randomization, the most common mechanism used to combat cache poisoning in general, and the Kaminsky vulnerability in particular. Last year we found that a hair less than 24% of the open recursors we identified exhibited poor source-port randomization (e.g., none), and my friend Matt Larson's study from the vantage point of the root name servers estimated that roughly 30% hadn't been patched. Any decrease in the population of name servers vulnerable to cache poisoning is good news.

- The percentage of zones with name servers open to zone transfers dropped by almost half, from 31% last year to 16% this year. While not as serious a threat as open recursive name servers, name servers that are open to zone transfers are sometimes at risk of denial of service attacks.

- The percentage of name servers we fingerprinted as running some version of the Microsoft DNS Server dropped to 0.37% from 2.74% just two years ago. That's really remarkable, given how pervasive the use of Windows operating systems is. This suggests that administrators realize that the Microsoft DNS Server, while useful "behind the firewall," lacks features necessary to secure it when directly exposed to the Internet.

- The number of signed subzones of com, net, and org is increasing. In percentage terms, the rise is dramatic: about 270%. But in absolute terms, the increase is much less impressive, from 45 signed subzones last year to 167 this year. Still, that's heartening evidence of broader deployment of DNSSEC. With the .org zone recently signed and .net due to be signed next year, as well as the availability of products that make the management of DNSSEC-signed zones easier, I'll bet that we'll see adoption accelerate by next year's survey. (I'm also grateful to the Public Interest Registry for agreeing to participate in this year's survey, and to VeriSign for their continued participation.)

I'm even hopeful that our alarming percentage of open recursors can and will be addressed quickly. If these open recursors are, in fact, customer premises equipment that carriers are deploying, there's a good chance that those carriers have some kind of centralized control of those devices, or at the very least can change the default configuration the devices ship with. And there's every reason to believe that the carriers want to do the right thing - after all, who would risk finding coal in his stocking?

The Dominoes Keep Falling

2009-11-10T01:11:11Z

To celebrate the 20^th anniversary of the fall of the Berlin Wall today, Berliners and other reunified Germans toppled a set of 1000 giant dominoes at the Brandenburg Gate—a metaphor for the fall of Communism in states throughout the Eastern Bloc.

While not as dramatic or significant as the fall of European Communism, several dominoes on the path to broad implementation of the DNS Security Extensions fell recently:

Switzerland’s top-level zone, .ch, was signed. I found this out in the nick of time: just before I gave a security-themed presentation in Zuerich, where I would have looked pretty dumb had I not known that .ch was signed just days before.
Since SWITCH, the organization that runs the Swiss registry, also runs Liechtenstein’s registry, Liechtenstein’s top-level zone, .li, was signed, too. While Liechtenstein’s adoption of DNSSEC may seem like a footnote, remember that the country has more registered companies than citizens, including many financial services firms.
Namibia became the first country in Africa with a DNSSEC-signed zone, .na. (And the upcoming movie adaptation of “The Prisoner” was filmed in the Namibian city of Swakopmund, so clearly big things are happening in the country.)
Niue’s top-level zone, .nu, is signed. Sure, Niue is a tiny Pacific island nation (it’s population is less than that of a big Midwestern high school), but because “nu” means “now” in Swedish, it boasts a substantial population of Swedish registrants.
Turkmenistan’s .tm zone is signed. Through a similar quirk, Turkmenistan attracts registrants because “tm” is commonly used as an abbreviation for “trademark.”

All of this is good news for those of us who believe that widespread deployment of DNSSEC will make the Internet a safer place.

There are still some very big dominoes that haven’t fallen, of course—though they’re teetering: the root zone, .com, .net, .uk, .de, and more. The U.S. Department of Commerce has already announced that the root will be signed by July 1, 2010, and VeriSign will sign .net next year and .com in 2011. Plus my old friend Matt works for VeriSign as their Vice President of DNS Research, and if I’m not mistaken, I owe him money. I promise I won’t pay him back until VeriSign makes good on their commitments.

When "Trick or Treat" Brings DDoS to Your Door

2009-10-26T03:10:23Z

Together with our partner The Measurement Factory, Infoblox has just competed our fifth annual DNS Survey. We're still poring over the results, but one number that stands out to me is our latest estimate of the total population of name servers on the Internet, which has jumped to 16.3 million name servers this year from 11.7 million in 2007. (2008's results were a little suspect.)

Two questions spring to mind: First: Are there really that many name servers on the Internet? And second: How could that number have grown so dramatically in two years?

There's a single answer to both questions: They're mostly CPE. As many of you know already, CPE stands for customer premises equipment. It's a carrier's term for a device on the far end (from their standpoint) of a subscriber's link to the Internet. They're often nominally DSL or cable routers, but increasingly they support functions like embedded DNS proxy servers. That's why our survey identifies them as name servers: They respond to our probes just as a recursive name server would.

Just like an open recursive name server would, to be more precise. For the most part, these babies don't have any kind of access control whatsoever. They'll accept queries from any old IP address on the Internet, resolve them, and return the answer. Now you might think that would make them good citizens, but in fact it's just the opposite: Open recursors can be and are often used in DDoS attacks. A hacker spoofs queries from the IP address of a host (web server, mail server, what-have-you) he'd like to target and sends those queries to open recursors around the Internet. The open recursors respond to the targetted host. Worse, the hacker generally chooses a query that will return as large a response as possible, near the 4096 byte limit advertised by most BIND name servers. He can do that with a very modest-sized query of less than 100 bytes, giving him access to tremendous amplification. Send 1000 queries per second to 1000 open recursors and you've got a formidable DDoS attack of 32 Gbps.

If these open recursive name servers were rare, these attacks wouldn't be a problem. Fielding a thousand or more open recursors might require too much effort to be worth the hacker's while. But our survey results show exactly the opposite trend: Not only is the number of name servers increasing, but the proportion of open recursors is now increasing, too. The percentage of open recursors rose in this year's survey to a whopping 79.6% of the name servers identified, up from 52% in 2007.

Geoff Sisson, who conducted this year's survey, lays the blame on CPE: "4.4% of all IP addresses in French address blocks replied to probes, more than five times as many as responded on average to all probes globally. This is due the the broad deployment of totd ("Trick or Treat Daemon") in France Telecom's network. totd is a DNS proxy that forwards queries from pure IPv6 networks. Over 99.9% of all hosts running totd in France and nearly 92% of all hosts running totd anywhere were in AS 3215, France Telecom's primary AS."

It's tempting to dismiss the spike in the percentage of open recursors as simple bad luck, since we choose the networks to probe randomly and this time drew some of France Telecom's networks. But Geoff saw much the same phenomenon on Spanish networks, too.

Our next order of business is to contact France Telecom to confirm our identification of totd and impress upon them the importance of shipping CPE that supports access control on queries, and with a configuration that only proxies their subscribers' queries by default.