NetBouncer: Active Device and Link Failure Localization in Data Center Networks

Authors:
Cheng Tan, Ze Jin, Chuanxiong Guo, Tianrong Zhang, Haitao Wu, Karl Deng, Dongming Bi, and Dong Xiang

 Motivation:
Reliability and availability are crucial in data centers since loss of service is inexcusable. It is important that any failure in the network is identified and mitigated immediately. However, the process of accurately locating the point of failure is a non-trivial task in a network with millions of servers and network devices. The paper proposes NetBouncer, a failure location system which can detect both device and link failures. It does so through a combination of domain knowledge with machine learning, all the while ensuring that it is accurately able to find the path to the point of failure with minimal overhead.

Pros:

  • It manages to identify gray failures,(partial or subtle malfunctions where packets are dropped probabilistically ). Traditional monitoring systems have difficulty in observing gray failures since they are not easily perceived by  the end-hosts
  • Using IP-in IP tunneling enables NetBouncer to explicitly identify paths with minimal overhead. It probes a path without involving the CPUs of switches by encapsulating the entire path in nested IP packages. The assumption of the target network, Clos, states that there is  only a single path from a server to upper layer switch, greatly reducing the number of possible duplicate paths. This is crucial since in a data center network, a minute drop in throughput can have adverse effects.
  • Unlike most probing plans that try to minimize redundant paths, NetBouncer is not affected by redundant probing paths. Since redundant paths are considered a validation of each other, its accuracy is ultimately improved.  
  • It makes use of Coordinate Descent for optimization to solve the link failure inference problem. CD has certain advantages over SGD such as the ability to handle the sparse characteristic of the network data (since each link is included by only a few paths as compared to the total number of paths in the system), and also being able to handle large volumes of data. It is also said that CD converges more than an order of magnitude faster than SGD for link failure inference.
Cons:
  • NetBouncer assumes that the failures are independent. While this may be the case in most failures, in cases where the failures are correlated, The optimization problem could be significantly more complex since we would have to consider the interaction between different failure incidents. 
  • A number of the techniques and assumptions (such as existence of single path from server to switch, sufficient probing theorem) used depend on the underlying network being a Clos network. Given these assumptions, it is possible that the model may not work well with other networks.
  • NetBouncer assumes that both the sender and receiver for packet bouncing are the same server. If these servers are different, it would have to consider the case of failure of the sender or receiver, which may complicate failure handling. Although this assumption fits well with the Clos network, it may not hold be possible in certain more restrictive networks where sender and receiver cannot be the same server
  • It uses probes that originate from end-hosts rater than switches since the switches in their data centers are non-programmable. Using a switch based probing agent could offer more flexibility like individual link probes. As they claim that NetBouncer should support switch based approaches as well a comparison of switch based and end-host based probes would be interesting

-Karthik Unnikrishnan


Comments

  1. jon_weissmanMarch 15, 2019 at 12:25 PM

    Good points. As mentioned, there is also the issue of the rate at which dynamism is occurring relative to their prediction overhead.

    ReplyDelete

Post a Comment

Popular posts from this blog

A Machine Learning Approach to Live Migration Modeling

Authors:  Changyeon Jo, Youngsu Cho, Bernhard Egger Seoul National University Summary:  This paper proposed a machine learning approach to predict the system metrics for different VM migration algorithms. These system metrics are affected by migration algorithm, host machine status and user applications and feasible for machine learning approach. The prediction model takes 21 system/application input features (including which algorithm), and generates 6 metrics as the output. As a result, it can accurately estimate each metrics under different application, VM and algorithms. One of the application, the paper provide a semi-automated algorithm selection based on this machine learning model and user-provided SLA (service level agreements) constraints. Questions about Machine Learning:  Why machine learning is applied: The combination of algorithm, VM status and current running applications status, is a huge searching space. Machine learning features: which algor...
Read more

StormDroid: A Streaminglized Machine Learning-Based System for Detecting Android Malware

Introduction: The paper approaches the problem of Malware Detection using machine learning approach. In this paper, the authors propose considering new type of features along with two traditionally used set of features for developing techniques for malware detection. Along with this proposal, the paper makes other contributions as well. The authors provide a new framework which is able to process information from large data sets as streams, and efficiently perform malware detection. Motivation: Android is an open platform for development. It has certainly made easy for everyone to develop and publish apps.  The number of apps published on Google store is in millions. However, it has also made easier for hackers to reach out to a larger population for malicious attacks. Additionally, with surge in the number of attacks on devices, more data is available for analysis which can be used by the machine learning models to identify malicious apps. This is motivation to use machine ...
Read more

Coordinated Management of Multiple Interacting Resources in Chip Multiprocessors: A Machine Learning Approach

Authors: Ramazan Bitirgen, Engin Ipek, Jose F. Martinez Motivation: Sharing system resources efficiently on chip multiprocessors (CMPs) platforms is the key to obtain high utilization and enforce system-level performance objectives. Unrestricted sharing of microarchitectural resources will result in destructive interference and unacceptable performance. However, the works focus on coordinated management of multiple interacting resources on CMPs are very scarce , even though several proposals focusing on addressing the management of a single microarchitectural resource exist. Aiming to fill this gap, this paper proposes a framework with the machine learning technology to manage multiple shared CMP resources in a coordinated fashion to enforce higher-level performance objectives. This novel mechanism consists of per-application hardware performance models and a global resource manager. Positive Points: 1. The global resource manager is able to aggregate each app...
Read more