NetBouncer: Active Device and Link Failure Localization in Data Center Networks

Authors : Cheng Tan, Ze Jin, Chuanxiong Guo, Tianrong Zhang, Haitao Wu, Karl Deng, Dongming Bi, and Dong Xiang

Introduction :

Availability of large scale data centers ensures that all the operations within the center's network function smoothly. The biggest challenge for such data centers is the ability to accurately locate and contain device and link failures, across millions of servers and thousands of network devices. The authors introduce NetBouncer, a failure localization system which is capable of detecting both device and link failures.

The authors introduce the framework's algorithm for high accuracy link failure inference and use domain knowledge to make it robust to real-world data inconsistencies. 

Pros :

  • The NetBouncer framework makes use of Clos network architectures, which were introduced by Bell Labs in 1950s and made famous again with the onset of Ethernet switches. Many software companies now make use of multi-tier Clos network-based architectures. These networks reduce complexity in implementation and make it a simple network topology. 
  • The servers, which initiate the path probing, contain both the senders and receivers of the probing data. This prevents failures between senders or receivers or both in a distributed systems setting, which makes the failure handling a less complicated affair.
  • The authors come up with a theorem, the sufficient probing theorem, which helps in reducing the number of paths for the probing packets to traverse, thus saving a lot of computation resources. This saves a lot of resources in the switch CPUs.
  • The authors use a specialized regularization term and use coordinate descent, which takes the sparse nature of the path probing problem into consideration. It serves as a two-direction penalty and helps to resolve the false positive cases where the imperfect measurement involves a bad link and a good link. Coordinate descent is also considered useful when  computing the gradients is a tedious task.

Cons :

  • The framework produced a few false negatives over the years it has been deployed in Microsoft Azure's data centers. The real-world inconsistency seem to produce these values.
  • The framework also makes an assumption that the probing packets also experience the same failures as the real applications, which may not hold in all cases.
  • The objective function used is Least Squares with a specialized regularization term, based on the domain knowledge. The optimization method used here is coordinate descent which is a simple method to implement. There might be other optimization methods which can produce good results.
  • The authors reiterate the sentence "there are zero false positives and a few false negatives so far". This indicates that the authors are expecting some cases for the framework to produce false positives ? This has not been communicated clearly from the paper.
  • This particular framework works well with the Clos-based architectures. The framework doesn't generalize for other multi-stage interconnected networks.
- Aditya Bhat

Comments

  1. jon_weissmanMarch 14, 2019 at 6:49 AM

    Your first two cons are valid concerns: is the network so dynamic and noisy that it makes accurate inference dependent on long-term stable failure probabilities on links. Not sure what you mean by your next to last con?

    ReplyDelete
    Replies
    1. Aditya BhatMarch 14, 2019 at 6:59 AM

      Hello Prof,
      Let me clarify that. Throughout the entire paper, the authors seem to have this sentence recurring, "the model has been deployed for three years but hasn't produced any false positives yet." The emphasis I felt is on the fact that it works but it seems that the authors are expecting it to not play along well in the future? That's something which is not clear from the paper.

      Delete

Post a Comment

Popular posts from this blog

A Machine Learning Approach to Live Migration Modeling

Authors:  Changyeon Jo, Youngsu Cho, Bernhard Egger Seoul National University Summary:  This paper proposed a machine learning approach to predict the system metrics for different VM migration algorithms. These system metrics are affected by migration algorithm, host machine status and user applications and feasible for machine learning approach. The prediction model takes 21 system/application input features (including which algorithm), and generates 6 metrics as the output. As a result, it can accurately estimate each metrics under different application, VM and algorithms. One of the application, the paper provide a semi-automated algorithm selection based on this machine learning model and user-provided SLA (service level agreements) constraints. Questions about Machine Learning:  Why machine learning is applied: The combination of algorithm, VM status and current running applications status, is a huge searching space. Machine learning features: which algor...
Read more

StormDroid: A Streaminglized Machine Learning-Based System for Detecting Android Malware

Introduction: The paper approaches the problem of Malware Detection using machine learning approach. In this paper, the authors propose considering new type of features along with two traditionally used set of features for developing techniques for malware detection. Along with this proposal, the paper makes other contributions as well. The authors provide a new framework which is able to process information from large data sets as streams, and efficiently perform malware detection. Motivation: Android is an open platform for development. It has certainly made easy for everyone to develop and publish apps.  The number of apps published on Google store is in millions. However, it has also made easier for hackers to reach out to a larger population for malicious attacks. Additionally, with surge in the number of attacks on devices, more data is available for analysis which can be used by the machine learning models to identify malicious apps. This is motivation to use machine ...
Read more

Coordinated Management of Multiple Interacting Resources in Chip Multiprocessors: A Machine Learning Approach

Authors: Ramazan Bitirgen, Engin Ipek, Jose F. Martinez Motivation: Sharing system resources efficiently on chip multiprocessors (CMPs) platforms is the key to obtain high utilization and enforce system-level performance objectives. Unrestricted sharing of microarchitectural resources will result in destructive interference and unacceptable performance. However, the works focus on coordinated management of multiple interacting resources on CMPs are very scarce , even though several proposals focusing on addressing the management of a single microarchitectural resource exist. Aiming to fill this gap, this paper proposes a framework with the machine learning technology to manage multiple shared CMP resources in a coordinated fashion to enforce higher-level performance objectives. This novel mechanism consists of per-application hardware performance models and a global resource manager. Positive Points: 1. The global resource manager is able to aggregate each app...
Read more