ECE656(00) Ubiquitous Networks

▣ Lecture outline

  In this semester, we will cover two different topics: IoT and deep learning. The first part of this lecture will focus on wireless adhoc network technology that provide low energy and low latency lightweight communication for WSN (wireless sensor networks) and MANET (mobile adhoc networks). The second part of this class will focus on deep learning techniques of artificial intelligence. Recently, deep learning techniques are used not only for machine learning but also applied to other fields of information technology as well as to the other fields of engineering and science in general. We will discuss the basic concepts of deep learning such as CNN and RNN by studying the classical papers of machine learning and deep learning. Then, we move to more recent topics of deep learning such as reinforcement learning (RL) and generative adversarial networks (GAN) by reviewing the recent research papers in deep learning.

 Professor : Lynn Choi( lchoi@korea.ac.kr, Engineering Bldg, #411, 3290-3249)

 Assistant : HanJun Bae(qwerty2901@korea.ac.kr, Engineering Bldg, #236, 3290-3896) 

 Time(Place) : Wednesday (1-2) Engineering Building, #162

 Textbook :
"Deep Learning: Adaptive Computation and Machine Learning", Ian Goodfellow, MIT Press, 2016 

 Reference book : A Collection of Research Papers 

 Class notice


1. Lecture Note 1 was updated on August 30th.

2. Lecture Note 1 was updated again on September 5th.

3. Lecture Note 2 was updated on September 7th.

4. Lecture Note 2 was updated again on September 12th.

5. Lecture Note 2 was updated again on September 13th.

6. Lecture Note 1 was updated again on September 13th.


▣ Lecture slide


1. IoT and WSN


3. WSN MAC and Power Management

4. Wakeup Scheduling



▣ Reference


 Paper Presentation

 Paper List :  Paper List.docx


 Convolutional Neural Network (12 papers)

 - Rethinking the inception architecture for computer vision (2016), C. Szegedy et al.pdf

 - Inception-v4, inception-resnet and the impact of residual connections on learning (2016), C. Szegedy et al.pdf

 - Identity Mappings in Deep Residual Networks (2016), K. He et al.pdf

 - Deep residual learning for image recognition (2016), K. He et al.pdf

 - Spatial transformer network (2015), M. Jaderberg et al.pdf

 - GoingDeeperwithConvolutions - GoogleNet.pdf


 - Return of the devil in the details, delving deep into convolutional nets (2014), K. Chatfield et al.pdf

 - OverFeat Integrated recognition, localization and detection using convolutional networks (2013), P. Sermanet et al.pdf

 - Maxout networks (2013), I. Goodfellow et al.pdf

 - NetworkInNetwork.pdf

 - ImageNet classification with deep convolutional neural networks (2012), A. Krizhevsky et al.pdf


 Natural Language Processing / Recurrent Neural Network (12 papers)

 - Conditional random fields as recurrent neural networks (2015), S. Zheng and S. Jayasumana.pdf

 - Effective approaches to attention-based neural machine translation (2015), M. Luong et al.pdf

 - Exploring the limits of language modeling (2016), R. Jozefowicz et al.pdf

 - Generating sequences with recurrent neural networks (2013), A. Graves.pdf

 - Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014), K. Cho et al..pdf

 - Memory networks (2014), J. Weston et al.pdf 

 - Neural Architectures for Named Entity Recognition (2016), G. Lample et al.pdf

 - Neural machine translation by jointly learning to align and translate (2014), D. Bahdanau et al.pdf

 - Neural turing machines (2014), A. Graves et al.pdf

 - Sequence to sequence learning with neural networks (2014), I. Sutskever et al.pdf

 - Teaching machines to read and comprehend (2015), K. Hermann et al.pdf

 - Training and analysing deep recurrentneuralnetworks.pdf


 Optimization / Training Techniques (10 papers)


 - Batch normalization Accelerating deep network training by reducing internal covariate shift.pdf

 - Delving deep into rectifiers, Surpassing human-level performance on imagenet classification (2015), K. He et al.pdf

 - Dropout A Simple Way to Prevent Neural Networks from Overfitting.pdf

 - Improving neural networks by preventing co-adaptation of feature detectors (2012), G. Hinton et al.pdf

 - Learning long-term dependencies with gradient descent is difficult.pdf

 - Learning representations by back-propagating errors.pdf

 - Random search for hyper-parameter optimization (2012) J. Bergstra and Y. Bengio.pdf

 - Training very deep networks (2015), R. Srivastava et al.pdf



 Uderstanding / Generalization / Transfer (7 papers)

 - Distilling the knowledge in a neural network (2015), G. Hinton et al..pdf

 - Deep neural networks are easily fooled, High confidence predictions for unrecognizable images (2015), A. Nguyen et al.pdf

 - How transferable are features in deep neural networks.pdf

 - CNN features off-the-Shelf, An astounding baseline for recognition (2014), A. Razavian et al.pdf

 - Learning and transferring mid-Level image representations using convolutional neural networks (2014), M. Oquab et al.pdf

 - Visualizing and understanding convolutional networks (2014), M. Zeiler and R. Fergus.pdf

 - Decaf, A deep convolutional activation feature for generic visual recognition (2014), J. Donahue et al.pdf




 Other papers

 Recent papers (12 papers)

 - Batch renormalization Towards reducing minibatch dependence in batch-normalized models (2017), S. Ioffe.pdf

 - Convolutional Sequence to Sequence Learning (2017), Jonas Gehring et al.pdf

 - Deep Photo Style Transfer (2017), F. Luan et al.pdf

 - Deep voice Real-time neural text-to-speech (2017), S. Arik et al.pdf

 - Deformable Convolutional Networks (2017), J. Dai et al.pdf

 - Evolution Strategies as a Scalable Alternative to Reinforcement Learning (2017), T. Salimans et al.pdf

 - Learning to discover cross-domain relations with generative adversarial networks (2017), T. Kim et al.pdf

 - Least squares generative adversarial networks (2016), X. Mao et al.pdf

 - PixelNet Representation of the pixels, by the pixels, and for the pixels (2017), A. Bansal et al.pdf

 - TACOTRON Towards end-to-end speech synthesis (2017), Y. Wang et al.pdf

 - Understanding deep learning requires rethinking generalization (2017), C. Zhang et al.pdf

 - Wasserstein GAN (2017), M. Arjovsky et al.pdf


 Classic papers (10 papers)

 - A fast learning algorithm for deep belief nets (2006), G. Hinton et al.pdf

 - A practical guide to training restricted boltzmann machines (2010), G. Hinton.pdf

 - An analysis of single-layer networks in unsupervised feature learning (2011), A. Coates et al.pdf

 - Deep sparse rectifier neural networks (2011), X. Glorot et al.pdf

 - Gradient-based learning applied to document recognition (1998), Y. LeCun et al.pdf

 - Greedy layer-wise training of deep networks (2007), Y. Bengio et al.pdf

 - Learning mid-level features for recognition (2010), Y. Boureau.pdf

 - Natural language processing (almost) from scratch (2011), R. Collobert et al.pdf

 - Understanding the difficulty of training deep feedforward neural networks (2010), X. Glorot and Y. Bengio.pdf

 - Why does unsupervised pre-training help deep learning (2010), D. Erhan et al.pdf



 Reading List