Deep Learning Tutorial at ICON 2015

Abstract

Deep learning techniques have demonstrated tremendous success in the natural language processing community in recent times. The goal is to move machine learning closer to one of its original goals: Artificial Intelligence. Deep learning involves learning multiple levels of representation and abstraction that help to make sense of data such as images, sound, and text. Various natural language processing applications have shown state­of­the­art results by using deep learning architectures. Much of these successes can be attributed to deep learning’s ability to lend itself to representation learning for words. The focus of this tutorial is to establish a strong foundation of deep learning, give an extensive overview of existing deep learning approaches, and introduce different word representation approaches. This tutorial will help the audience in solving problems in language or text processing. This tutorial will span over three parts. In the first part, we will discuss the basics of deep learning, it’s architectures, distributed representations and various approaches to build such representations. We will direct the audience towards ready­to­use pre­trained models, open source tools to train your own models, and methods to evaluate these models. In the second part of the tutorial, we place particular emphasis on several important applications, including (1) Named Entity Recognition, (2) Word Sense Disambiguation, (3) WordNet linking and (4) Sentiment Analysis. For each application, we will discuss what particular architectures of deep learning models are suitable given the nature of the application, and how learning can be performed efficiently and effectively using end­to­end optimization strategies. In the third part, we are planning to have hands­on­sessions on various popular deep learning tools, and how deep learning techniques can be employed in various applications. Depending on availability of time, we plan to guide the audience to build either one of the following sample applications: ● ‘Odd­one­out’ application (find the outlier word in a set of words) ● ‘Word origin annotation’ application (detecting transliterated words in running text)