A combined discussion of Big Data analytics and SDN is analogous to asking the question, “What came first, the chicken or the egg?” Is it SDN that makes big data analytics possible or is it advanced analytics that makes SDN possible? In actuality, it is hard to discern one without the other as both are dependent upon the other.
In our series of white papers, Software Defined Networking – The Next IT Paradigm of Promise and Managing the Software Defined Enterprise, we defined SDN as the separation of the control plane from the data plane. Rather than have network intelligence residing on the hardware devices themselves, (switches, routers, firewalls) all devices within the data plane are controlled by a centralized clustered controller. This controller integrates with an intelligence driven software orchestrator or application that then drives the network. This software driven network architecture allows the controller a bird’s eye view of the network to derive traffic patterns and data flows. This upper level perspective allows it to make immediate adaptive decisions in order to respond to real time access demands, resulting in a level of agility and scalability that was unimaginable a decade ago.
Organizations are inundated today with data to the degree that we went from discussing gigabytes to terabytes in only a couple of years. Even medium data centers now describe their data storage needs in the terms of petabytes and very soon the terms exabytes or zettabytes will be regularly tossed around with the emerging paradigm of the Internet of Things (IoT).
CONNECTING THE DISCONNECTED
One can argue that the greatest resource of an organization is no longer its human resources, but its data resources. It is data that creates actionable intelligence that drives innovation. Up until recently, data driven decision making has almost exclusively involved connected structured data. The problem is that organizations are inundated with disconnected or unstructured data as well. Examples of unstructured data can be videos, voice mail, saved documents and social network application data. There is a plethora of data within an organization that remains isolated or disconnected from its users, mainly because it’s time consuming and cumbersome to sort through the data that is being created by a range of different devices which is then saved to many locations in a variety of file formats. Analysts thus spend the vast majority of their time querying data, leaving a small minority of their time for analyzing data. For this reason, researchers find themselves deriving the questions they want to ask in advance, rather than relying on the data dictating the questions.
The goal of analytics is to liberate ALL of the data within an organization. In order to maximize data driven decisions, you must involve as many endpoints as possible in the data collection process. The data mining extraction process involves three primary steps:
- Splitting the data into multiple server nodes
- Analyzing each data block in parallel
- Merging the found results
Much of this process involves an east-west or parallel transfer of data rather than the traditional north-south communication flow characteristic of the classic client-server model. It is this east-west transfer of data that serves as the bottleneck for big data today as big data can only operate as fast as the network’s capability to transfer it between servers during the split and merge processes. This results in significant wait time for these large data transfers. This wait time is greater exemplified when data is distributed amongst global resources through the cloud which further reduces transfer rates. Besides traffic patterns, the greatest challenge is just the scope of everything.
The more data endpoints you can draw from, the better your intelligence will be and highly distributed data accentuates the need for highly distributive networks. Only a software defined network can deliver the level of agility and responsiveness required by big data. In an SDN environment, the controller can provision additional resources in real time to instantly meet data transfer demands. Without the responsiveness and adaptability of SDN, the true benefits of big data would not be possible which results in an ideal liaison between the two.
LEVERAGING ADVANCED ANALYTICS
Few organizations have learned how to harness the power of advanced data analysis better than Netflix. Netflix has access to almost unlimited information about their user subscriber base and how they watch programming through the Netflix app. Of course they know what the most popular movies, genres, plot types, actors and actresses of their subscribers are, but they can also analyze:
- The percentage of the time each show is watched in its entirety
- The ratings given and what viewers had to say
- How often a show is paused and when
- The date and time shows are watched
It was this immense granular web of data that enabled Netflix to purchase and produce original programming. Much has been written about their pre-analysis concerning the series, House of Cards, and how executives knew of its impending success even before the first scene was shot at the studio.1
There are so many examples of how advanced data analysis can be utilized. A customer in a department store is texted a coupon for the jeans she is currently looking at in real time. An insurance company uses big data in order to achieve better risk assessments and in turn enables the creation of better pricing models. And finally, a large enterprise network utilizing big data analytics in order to streamline traffic flows and achieve optimum efficiency in the network – what we call SDN.
SDN after all is directly correlated with information that circulates the network and drives the decisions of the orchestrator based controller. While it is Software Defined Storage (SDS) that adds the immense degree of agility and automation to the network, it is big data that is essentially the brains behind the streamlining. Just as big data doesn’t require dedicated data analysts to query the data, SDN doesn’t require network administrators to conduct packet sniffing sessions to examine traffic patterns over an elongated period in order to identify trends that will drive future decision making. For SDN, advanced analytics performs these actions in real time.
It is clear why the melding of SDN and advanced data analytics is helping to drive each of these technologies to new heights. One cannot completely thrive without the other and it is this working relationship that is transforming networks today into responsive entities that will improve the bottom line of the organizations that embrace this powerful combination.
