Our friends at The Measurement Factory just completed the 2010 DNS Survey, and we’ll release the complete results soon.  For those of you who just can’t wait, though, here’s a preview of some of the results.

One of our datasets was a large (i.e., millions of zones), random sample of subzones of .COM, .NET and .ORG.  Looking at the IP addresses of the authoritative name servers for these zones, we found that nearly 20% seem to have all of their authoritative name servers on the same network. (They’re on the same /24, at least—in some cases, of course, that might be multiple networks, but it’s relatively unlikely.)

This is an accident waiting to happen.  Some of you might remember the embarrassing DNS outage Microsoft suffered several years ago when a technician misconfigured a border router in Redmond:  Because all of their authoritative name servers were on a single network behind that router, users couldn’t resolve most Microsoft domain names for an extended period.  Don’t let that happen to you.

Read more...

On Thursday of last week, Facebook went down.  Most of us in the Internet business know this; a frighteningly large proportion of the population actually experienced it first-hand.  I read more than one posting suggesting that worldwide productivity surged on Thursday.  It reminded me of the episode of “The Simpsons” in which Marge mounts a crusade against cartoon violence on television (read “Itchy and Scratchy”) and succeeds in getting the cartoon watered down.  (The revised theme song begins, “They love!/They share!/They share and love and share!”)  Kids all over the world go outside to swing on swings, blow bubbles, play tag.  My wife frequently cites that episode herself, wistfully, while looking alternately at our sunny backyard and then at our kids, plopped on the so fa.

Anyway, besides reminding me of old episodes of The Simpsons, Thursday’s news reminded me of how quick people are to blame DNS.  The initial reports of the problem had cited DNS as the cause.  After doing a little research into what had gone wrong (by reading the archives of the NANOG mailing list anda posting by Facebook’s engineering organization explaining the cause of the problem in that admirably candid way engineers do), it was obvious that, while the problem affected DNS, it wasn’t chiefly an issue with DNS.

Read more...

An hour or so ago, I tried to check a Wikipedia entry and my browser told me it couldn't find en.wikipedia.org.  Surely that's wrong, I thought, but pushed "Check Wikipedia" onto the stack and went on to something else.  Then, coincidentally, while searching for DNS-related news articles to inspire my next blog entry, I ran across this one from PC Magazine.  Turns out Wikipedia's European data center had an overheating problem that caused many of their servers to shut down in an act of self-preservation.  To shunt European traffic to their servers in Florida, they enacted their failure procedure, which modifies their DNS records.

Unfortunately, that failover mechanism was broken (they didn't specify how), and broken so badly that it interrupted DNS resolution for all Wikimedia sites globally.  While they quickly recognized and fixed the problem, it took as long as an hour for the corrected data to propagate because of TTLs.

Read more...