ApDeepSense : Deep Learning Uncertainty Estimation Without the Pain for IoT Applications



Introduction :

The paper focuses on providing uncertainty estimates in deep learning outputs. Uncertainty estimates
are important for IoT applications which interact directly with the real world. The use of the existing
approaches requires intensive computing and testing which would be a bottleneck in resource
constrained IoT environments. To tackle this challenge, the authors propose ApDeepSense which proves
to reduce the execution time to 88.9% and 90% of the energy consumption when compared to the
state-of-the-art methods.

Method : The paper proposes novel layer wise approximations and builds over the previous results to prove the equivalence between the layers in neural networks and the layers in deep gaussian processes. The obtained posterior distribution is approximated as a multivariate gaussian process which is used to establish the uncertainty of predictions.

Thoughts/Comments :
1. The paper considers the resource challenges of IoT and brings attention to an overlooked area of
research.
2. It proposes a great improvement on inference time but there is little discussion on the actual training
required and the increase in training time for convergence using drop out training.
3. Authors discuss the challenge of figuring a closed-form output distribution as a potential road block to
extend to convolutional and recurrent neural networks.
4. The most important gain from the paper is that it removes the requirement of sampling and re-training of
the neural nets to provide the uncertainty estimates which could be really expensive in IoT environments.

-Aakhila Shaheen

Comments

  1. jon_weissmanApril 16, 2019 at 6:37 AM

    Good point about the impact of dropout on training time. The main idea seems to have been published elsewhere, that stochastic nets via dropout can give you Gaussian estimates, but they have tweaked it for the IoT environment. A hard paper to evaluate given the amount of mathematics!

    ReplyDelete

Post a Comment

Popular posts from this blog

A Machine Learning Approach to Live Migration Modeling

Authors:  Changyeon Jo, Youngsu Cho, Bernhard Egger Seoul National University Summary:  This paper proposed a machine learning approach to predict the system metrics for different VM migration algorithms. These system metrics are affected by migration algorithm, host machine status and user applications and feasible for machine learning approach. The prediction model takes 21 system/application input features (including which algorithm), and generates 6 metrics as the output. As a result, it can accurately estimate each metrics under different application, VM and algorithms. One of the application, the paper provide a semi-automated algorithm selection based on this machine learning model and user-provided SLA (service level agreements) constraints. Questions about Machine Learning:  Why machine learning is applied: The combination of algorithm, VM status and current running applications status, is a huge searching space. Machine learning features: which algorithm, 20 featu
Read more

NetBouncer: Active Device and Link Failure Localization in Data Center Networks

Authors : Cheng Tan, Ze Jin, Chuanxiong Guo, Tianrong Zhang, Haitao Wu, Karl Deng, Dongming Bi, and Dong Xiang Introduction : Availability of large scale data centers ensures that all the operations within the center's network function smoothly. The biggest challenge for such data centers is the ability to accurately locate and contain device and link failures, across millions of servers and thousands of network devices. The authors introduce NetBouncer, a failure localization system which is capable of detecting both device and link failures. The authors introduce the framework's algorithm for high accuracy link failure inference and use domain knowledge to make it robust to real-world data inconsistencies.  Pros : The NetBouncer framework makes use of Clos network architectures, which were introduced by Bell Labs in 1950s and made famous again with the onset of Ethernet switches. Many software companies now make use of multi-tier Clos network-based architecture
Read more

Deep Learning for Entity Matching: A Design Space Exploration

Authors - Sidharth Mudgal, Han Li, Theodoros Rekatsinas, AnHai Doan, Youngchoon Park, Ganesh Krishnan, Rohit Deep, Esteban Arcaute, Vijay Raghavendra Motivation/Abstract: Introduces entity matching, deep learning separately. Categorizes all the Deep Learning solutions in similar situations(matching tasks in NLP), and then works towards solving Entity Matching. Talks about the involvement of Deep Learning in EM in the past (one model). Finds out different fields where Deep Learning might perform better, does testing and reports results and conclusion. Main Points: There are three criteria/context where the discussion(comparing Deep Learning model vs traditional model) takes place - 1. Structured Data - SQL type stored data 2. Textual Data - Consists of text blobs which consist of huge sentences 3. Dirty Data - Erroneous structured data where text fields have values all mixed up Four representative models are picked and the Deep Learning process is explained through them. 
Read more