5 Years Later - What We've Learned from the Attack on Dyn
One of the most memorable distributed denial of service (DDoS) exploits of the Domain Name System (DNS) was the 2016 attack on the DNS provider Dyn. In this exploit a series of DDoS attacks severely affected connectivity for almost seventy major companies including giants in the e-commerce, financial, news/entertainment and retail spaces.1 The attack, which also introduced the Mirai botnet, served to introduce the world to the foundational importance of a smoothly functioning DNS infrastructure. While this attack could be considered old news, it bears consideration partly because of the lessons learned from it. Additionally, a look at this exploit helps to better understand what types of attacks on DNS are still happening today.
The DDoS attack on Dyn was not actually a single continuous attack but rather occurred in several waves. The first wave of the attack, which targeted three large Dyn datacenters, featured volumetric, network-related TCP SYN floods. This common attack method takes advantage of the three-part TCP handshake, in which the client (or attacker, in this case) starts the communication with a synchronize (SYN) packet, the server responds with an acknowledgement (SYN-ACK) packet, and the client finishes with another acknowledgement (ACK) packet and begins the transmission. In a TCP SYN flood, the attacker starts the transmission, then leaves the server holding the handshake session open waiting for a final ACK packet. To understand the magnitude of this attack, multiply this process tens of thousands of times; and remember, this is all on top of the normal traffic that the servers were already processing. Since there is a finite number of TCP connections a server can process at once, exhausting those slots with half open connections means system resources aren’t available to respond to legitimate requests.
The TCP SYN flood was aimed at port 53, the default port of DNS, on the authoritative servers. Part of the reason that the first wave succeeded is that bad traffic could be aimed directly at Dyn’s authoritative servers themselves. This was possible because at that time the live addresses behind most DNS servers’ anycast addresses were open. One of the lessons learned from the exploit is to only allow traffic to the anycast addresses and not directly to the IP addresses of the individual name servers, making it more difficult to target the servers directly.
Another item that the attackers exploited in the first wave was the congestion of the bandwidth that came from many of the peering relationships between Dyn and other internet service providers. The botnet traffic caused enough congestion that the internet exchange points (IXPs) that the providers and Dyn peered at were overloaded with enough packet loss to make them mostly unusable.
The second wave of the attack, which targeted approximately twenty Dyn datacenters, featured an attack more specific to DNS itself. This attack, known as a subdomain or prepend attack, begins with the attacker’s clients requesting an address on a domain that might exist, but prepending or adding a subdomain that does not. For example, an attacker might ask for resolution for “123456789012.example.com.” Rather than going directly to the authoritative server, this request is sent through the recursive resolver. The attacker knows that because this address does not exist, the resolver will not have the answer in its cache and will need to forward the attack to the auth server. The 2016 attack randomized prepended subdomains for the domains that Dyn was authoritative. Rather than the volumetric “terabits-per-second” bandwidth clogging of the first wave, the second wave of the attack also generated “millions of packets-per-second” in intensity, designed to exhaust the DNS application itself.
For this attack to succeed, many millions of requests were required to be sent simultaneously. At that time, this was possible because there were millions of unmonitored open recursive resolvers, any of which could send a request to any authoritative server. Such an attack would probably not succeed as dramatically today because the number of unmonitored open resolvers have decreased dramatically.
Highlights from Dyn - considerations
DNS is a very resilient protocol, and even though it is constantly subjected to DDoS attacks, there is rarely a major outage. That said, the attack on Dyn did serve to highlight several elements that bear consideration.
Overprovision – Make sure your DNS infrastructure (or that of a provider that you are considering) has overprovisioned their DNS infrastructure to deal with the regular ebbs and flows of internet traffic, as well as that during times of attack.
DDoS Protection – DDoS is a constant threat to DNS and the worst time to consider mitigation is when you are under attack. You should consider what options are available, as well as the details of mitigation. It is easy to let this go during “peacetime,” but a bad thing to have to research while under attack.
Throughput – this is more than just a simple number – it includes both elements below:
Robust and closely monitored anycast – Anycast is an important element of how modern DNS functions, and usually generates the best response for address resolution. Neustar’s recent edge node anycast upgrades were made based on analysis of where requests were coming from as well as what paths they were using. The goal was to bring down the number of hops required to get to authoritative servers, provide a means to avoid overloaded traffic exchange points, and reduce overall latency.
Monitored and tuned peering – It is essential that peering relationships be closely monitored and reviewed. Visualizing and monitoring peering points, whether via a public IXPs or in a colocation facility will ensure that your DNS will function optimally.
While the cautionary tale that Dyn provides has led to many changes within the industry, it is important to note that DDoS attacks on DNS still occur on a regular basis. That’s because DNS is an application that has to be able to accept requests for address resolution in order to function. A sufficiently large botnet can still slow responses or stop them altogether by clogging the pipes. Additionally, many simultaneous requests for addresses that are not in a resolver’s cache can still cause a server to run slowly. However, the changes that were made as a result of the attack on Dyn, including removing unmonitored open resolvers, DDoS mitigation, anycast, and monitored/tuned peering has reduced the risk of such exploits but, has not eliminated them all together. Nevertheless, by using Dyn as a cautionary tale, you can ensure that you do not risk falling victim or find yourself under in the same situation.