Big Data DDoS Solutions

Attackers tools are readily available making DDoS defence much harder than attack. It’s hard to blame anyone; the ISP are just transiting traffic and end users don’t know if they are compromised and part of a BotNet farm. There is no service of abuse or licence for the Internet making tracking and detection between independent service provider locations difficult. Recently, there has been a shift to application footprints. We now have multi-tiered applications dispersed across a variety of sites all requiring cross communication. New application architecture results in new types of attacks and with any application segment you are only as strong as the weakest link. The birth of the cloud and new technologies certainly increase the attack surface making quick and accurate DDoS¬†detection a prime focus.

Companies require mechanisms to stop and slow down DDoS attacks. The IETF introduced best practices with BCP38 and service providers started to incorporate ingress filtering to their designs, cross check incoming frames. However, ISPs are not forced by contract to implement these features.

The only way to adequately mitigate DDoS attacks is adequate detection. How long should this take? What timeframe is acceptable? All this depends on the traffic analysis solution you have in place. Initially, traffic analysis began monitoring up/down interfaces with basic statistics. They then moved to Syslog servers and single source basic flow capturing. What we need is a system that captures enriched flow data grouping both infrastructure and application information together. Enriching data from all elements allows the network and its traffic to be viewed as one holistic entity.


Enhanced Data Sources

DDoS traffic analysis solutions extract various types of flow data from network devices. Flow record consists of different fields from a variety of data types including NetFlow, IPFIX, and sFlow. DDoS Big Data solutions can enrich records at the ingest layer by performing the lookup on source and destination in the flow, BGP table, and GeoIP database. These values are added as additional volumes and fields stored with the original flow. The extra information allows administrators to slice the traffic at ingest enabling a great multi-dimensional view of network traffic.

Tools such has sFlOW, and IPFIX variants are critical pieces in the DDoS detection realm. Classic flow fields based on 5-tuples include for example IP address, source/destination port numbers are later expanded to include fields such as MAC address, MPLS, and application schematics like URLs, HTTP host headers, DNS queries and responses. The availability of advanced fields enables the detection of sophisticated attacks higher up the protocol stack. For example, access to the HTTP host header for each request allows precise identification down to the URL.

big data DDoS

Not all DDoS attacks are easily detected. Volumetric attacks such as a SynFlood are easier to catch than a SlowLoris and RUDY attacks. Layer 7 attacks usually don’t exceed the packet/sec threshold – a standard parameter for detecting volumetric based attacks. To combat this, we need to go deeper than the standard 5-tuple with augmented flows. Augmented flows contain additional fields to include a variety of advanced metrics such as connection counts, congestion windows and TCP RTT. Traditional flow data does not provide this level of detailed information.


Data Source Variations

Netflow and IPFIX flow record creation are based on packets sharing the same fields. Flow state is held hitting system resources. To save system resource, flows are exported at predefined times. As a result, traffic measurement is accurate, but it might not hit the detector for up to one minute. sFlow sends packet samples every 1 in N, streaming flows as soon as they are prepared. sFlOW has a reduced draw on system resources than its Netflow counterpart. It is considered to be faster and has better accuracy meaning it’s a great tool for DDoS detection.

sFlow is better at carrying the source MAC address than NetFlow and IPFIX. With NetFlow and IPFIX the source MAC is possible but not usually implemented by all vendors. NetFlow is useful for some requirements while IPFIX and sFlow for others. To get all the possible knobs, it’s better to extract from all data sources and combine into one database easily viewed with a single portal. By combining all data sources into one unified store, the type of protocol less relevant.


Irregularities with ASN Information

DDoS solutions can peer EBGP with customers which give them a copy of the BGP table. Customer route updates are reflected through standard BGP propagation procedure. It’s a non-intrusive peering agreement; BGP next hops are not altered meaning customers data plane traffic flows as normal. The contents of the BGP table provides access to customers control plane information enabling full visibility into data source and destination. The manual approach with BGP can be cumbersome. BGP does offer a string of information about DDoS source and destination but it can be hard to craft BGP regular expressions to extract this type of information. Not everyone can craft regular expressions, and it’s a skill for senior engineers.

Netflow does provide some BGP ASN information but you only have access to source and destination Peer or Origin ASN. Some high-end platforms do both, but it’s restricted to certain devices and vendor discretion. NetFlow should not hold all BGP type information; this would certainly be a suboptimal solution. Also, Netflow does have drawbacks and inaccuracies when determining the source ASN. The destination ASN is never usually a problem. To determine the source ASN and populate the FIB, the BGP process/daemon performs a REVERSE BGP Lookup. However, this type of BGP lookup does not guarantee result correctness. A REVERSE BGP Lookup primarily determines how to route back to the source, but this does not correlate how the source may route to you. The majority of networks out there are asymmetric in nature; meaning the source-destination path is not the same as the reverse direction. An IP packet traversing from source A to destination B will most certainly take a different return path. Traditional monitoring systems misrepresent the BGP table with inaccurate source ASN due to the common nature of asymmetric routing.

Legacy traffic analysis systems that don’t peer EBGP with their customers will report inaccurate source ASN. Not much good when troubleshooting a DDoS attack and the source ASN information is incorrect. The majority of legacy systems don’t offer accurate full AS-Path information leading to false positives and the inability to determine friend from foe. It’s far better for the solution to peer BGP with the customer, extract NetFlow/IPFIX/sFlow locally and then correlate the data to provide a unified source of truth. Origin ASN and Peer ASN provide the endpoints of the data flow and NetFlow is used to the middle. To go one step further, we can utilise GeoIP Information and analyse the county, region, and city. Correlate this with the full AS-Path list, and you now have a full view of source and destination paths with all the details of the middle points.





About Matt Conran

Matt Conran has created 150 entries.

Leave a Reply