Deep learning techniques have demonstrated tremendous success in the natural language processing community in recent times. The goal is to move machine learning closer to one of its original goals: Artificial Intelligence. Deep learning involves learning multiple levels of representation and abstraction that help to make sense of data such as images, sound, and text. Various natural language processing applications have shown state­of­the­art results by using deep learning architectures. Much of these successes can be attributed to deep learning’s ability to lend itself to representation learning for words. The focus of this tutorial is to establish a strong foundation of deep learning, give an extensive overview of existing deep learning approaches, and introduce different word representation approaches. This tutorial will help the audience in solving problems in language or text processing.\nThis tutorial will span over three parts. In the first part, we will discuss the basics of deep learning, it’s architectures, distributed representations and various approaches to build such representations. We will direct the audience towards ready­to­use pre­trained models, open source tools to train your own models, and methods to evaluate these models.\nIn the second part of the tutorial, we place particular emphasis on several important applications, including (1) Named Entity Recognition, (2) Word Sense Disambiguation, (3) WordNet linking and (4) Sentiment Analysis. For each application, we will discuss what particular architectures of deep learning models are suitable given the nature of the application, and how learning can be performed efficiently and effectively using end­to­end optimization strategies. In the third part, we are planning to have hands­on­sessions on various popular deep learning tools, and how deep learning techniques can be employed in various applications. Depending on availability of time, we plan to guide the audience to build either one of the following sample applications: \n● ‘Odd­one­out’ application (find the outlier word in a set of words) \n● ‘Word origin annotation’ application (detecting transliterated words in running text)

The focus of this tutorial is to cover the breadth of the literature on recent advances in Unsupervised Machine Translation. The tutorial will help the audience in getting started with unsupervised machine translation. The tutorial will span over three sections. In the first section, we will cover the fundamental concepts like cross-lingual embeddings, denoising auto-encoders, language model pre-training, Back Translation (BT), etc which are key to the success of Unsupervised Machine Translation. In the second section, the tutorial will provide a brief summary of recent works on unsupervised machine translation. The tutorial will cover both Phrase-Based Statistical Machine Translation systems as well as Neural Machine Translation systems. In the last section, we will talk about the limitations of the existing approaches for Unsupervised machine translation approaches and provide general guidelines for successful training of these systems. We also discuss case-studies from Indian languages and provide results obtained with U-MT over Indian language pairs. Finally, we talk about possible research directions.

This talk is aimed to discuss the challenges and the research opportunities in Indic Language Computing. The talk begins with an introduction to Large Language Models and the capabilities of these Large Language Models. Specifically, the talk covers auto-regressive language models like GPT2, GPT3, etc. The talk briefly describes the resource requirements for training such large language models. Specifically, we will look at the requirements in terms of training corpora, the model capacity, and the computation budget for training such large language models.\nRecently, there has been widespread interest in creating better large language models by aligning them with the human intention. These approaches include instruction-finetuning, learning with reinforcement learning with human feedback, etc. The talk covers some of the popular approaches for large language model alignment.\nThe talk later delves into the challenges of creating such large language models for Indic languages. We will first look at the efforts of AI4Bharat in creating large scale monolingual corpus for various Indic languages. We will discuss the potential challenges and research opportunities. We will discuss some of the evaluation benchmarks like IndicNLG and IndicXtreme for Indic languages. We will also discuss the Olive and instruction-following large language model for Odia language. The talk concludes by briefly summarizing the research opportunities in Indic Language NLP.