Device Placement Optimization with Reinforcement Learning


Motivation :

The main aim behind the paper is to learn which sets of operations on TensorFlow would perform best on which set of devices in order to optimize the size and computational requirements of the CPUs and GPUs for a neural network. Currently, the devices in the environment being used are heterogeneous and distributed where there is a mixture of hardware devices like CPUs and GPUs and the decision about which parts of the neural models to place on which devices is made by human experts.  These decisions are based on simple heuristics and intuitions, thus there is a limitation to this approach which the authors have tackled using Reinforcement Learning. 

Main points :

1.They use a sequence-to-sequence model where device placements is the output sequence and the operations in a neural network is the input sequence.

2. They use three Benchmark algorithms to make comparisons.(Recurrent Neural Network Language Model (RNNLM), Neural Machine Translation with attention mechanism (NMT) and Inception-V3) Thus giving us a clear level to make comparisons

3. The Reinforcement Learning - based placement of the Neural MT graph compared to the fine-tuned expert-designed placement shows that an improvement of 19.3% in running time. 

4. The Reinforcement Learning - based placement of Inception - V3 compared to the fine-tuned expert-designed placement shows that an improvement of 19.7% in running time. 

5. In an extension to their work where they increased memory  footprints (increased the LSTM sizes) such that one layer would not be possible to fit into a single device then the human-designed placement is not possible. But their approach managed to find a way to place the input models on devices efficiently.

Pros :

1. They have introduced us to a graph partitioning algorithm with an open source software library called the Scotch optimizer. It is very interesting how they model the inputs and outputs of the Neural Network as graphs. This idea is interesting and can be applied to other domains as well.

2. In the related works sections we can see that they have done a lot of background research. Thus they are able to clearly state what techniques they are using and why they have used it.

Cons :

1. The Devices used are very high end devices which are very fast and also very expensive. So we cannot relate this with regular machines that we students have access to.

2. They use only the execution time and the number of available devices, they do not use other information about the hardware configuration. I feel that in an environment where there were heterogeneous devices this information would be useful.

- Nishka Monteiro

Comments

  1. UnknownFebruary 21, 2019 at 6:26 AM

    Try to think about deeper issues. Have they justified the models they are using? Are there any obvious limitations or questionable assumptions? On the positive side, their approach seems to work for quite different input workloads (ML models).

    ReplyDelete

Post a Comment

Popular posts from this blog

A Machine Learning Approach to Live Migration Modeling

Authors:  Changyeon Jo, Youngsu Cho, Bernhard Egger Seoul National University Summary:  This paper proposed a machine learning approach to predict the system metrics for different VM migration algorithms. These system metrics are affected by migration algorithm, host machine status and user applications and feasible for machine learning approach. The prediction model takes 21 system/application input features (including which algorithm), and generates 6 metrics as the output. As a result, it can accurately estimate each metrics under different application, VM and algorithms. One of the application, the paper provide a semi-automated algorithm selection based on this machine learning model and user-provided SLA (service level agreements) constraints. Questions about Machine Learning:  Why machine learning is applied: The combination of algorithm, VM status and current running applications status, is a huge searching space. Machine learning features: which algor...
Read more

StormDroid: A Streaminglized Machine Learning-Based System for Detecting Android Malware

Introduction: The paper approaches the problem of Malware Detection using machine learning approach. In this paper, the authors propose considering new type of features along with two traditionally used set of features for developing techniques for malware detection. Along with this proposal, the paper makes other contributions as well. The authors provide a new framework which is able to process information from large data sets as streams, and efficiently perform malware detection. Motivation: Android is an open platform for development. It has certainly made easy for everyone to develop and publish apps.  The number of apps published on Google store is in millions. However, it has also made easier for hackers to reach out to a larger population for malicious attacks. Additionally, with surge in the number of attacks on devices, more data is available for analysis which can be used by the machine learning models to identify malicious apps. This is motivation to use machine ...
Read more

Coordinated Management of Multiple Interacting Resources in Chip Multiprocessors: A Machine Learning Approach

Authors: Ramazan Bitirgen, Engin Ipek, Jose F. Martinez Motivation: Sharing system resources efficiently on chip multiprocessors (CMPs) platforms is the key to obtain high utilization and enforce system-level performance objectives. Unrestricted sharing of microarchitectural resources will result in destructive interference and unacceptable performance. However, the works focus on coordinated management of multiple interacting resources on CMPs are very scarce , even though several proposals focusing on addressing the management of a single microarchitectural resource exist. Aiming to fill this gap, this paper proposes a framework with the machine learning technology to manage multiple shared CMP resources in a coordinated fashion to enforce higher-level performance objectives. This novel mechanism consists of per-application hardware performance models and a global resource manager. Positive Points: 1. The global resource manager is able to aggregate each app...
Read more