Yang, Yinchong (2018): Enhancing representation learning with tensor decompositions for knowledge graphs and high dimensional sequence modeling. Dissertation, LMU München: Faculty of Mathematics, Computer Science and Statistics 

PDF
Yang_Yinchong.pdf 4MB 
Abstract
The capability of processing and digesting raw data is one of the key features of a humanlike artificial intelligence system. For instance, realtime machine translation should be able to process and understand spoken natural language, and autonomous driving relies on the comprehension of visual inputs. Representation learning is a class of machine learning techniques that autonomously learn to derive latent features from raw data. These new features are expected to represent the data instances in a vector space that facilitates the machine learning task. This thesis studies two specific data situations that require efficient representation learning: knowledge graph data and high dimensional sequences. In the first part of this thesis, we first review multiple relational learning models based on tensor decomposition for knowledge graphs. We point out that relational learning is in fact a means of learning representations through onehot mapping of entities. Furthermore, we generalize this mapping function to consume a feature vector that encodes all known facts about each entity. It enables the relational model to derive the latent representation instantly for a new entity, without having to retrain the tensor decomposition. In the second part, we focus on learning representations from high dimensional sequential data. Sequential data often pose the challenge that they are of variable lengths. Electronic health records, for instance, could consist of clinical event data that have been collected at subsequent time steps. But each patient may have a medical history of variable length. We apply recurrent neural networks to produce fixedsize latent representations from the raw feature sequences of various lengths. By exposing a prediction model to these learned representations instead of the raw features, we can predict the therapy prescriptions more accurately as a means of clinical decision support. We further propose TensorTrain recurrent neural networks. We give a detailed introduction to the technique of tensorizing and decomposing large weight matrices into a few smaller tensors. We demonstrate the specific algorithms to perform the forwardpass and the backpropagation in this setting. Then we apply this approach to the inputtohidden weight matrix in recurrent neural networks. This novel architecture can process extremely high dimensional sequential features such as video data. The model also provides a promising solution to processing sequential features with high sparsity. This is, for instance, the case with electronic health records, since they are often of categorical nature and have to be binarycoded. We incorporate a statistical survival model with this representation learning model, which shows superior prediction quality.
Abstract
Item Type:  Thesis (Dissertation, LMU Munich) 

Subjects:  000 Computers, Information and General Reference 000 Computers, Information and General Reference > 004 Data processing computer science 
Faculties:  Faculty of Mathematics, Computer Science and Statistics 
Language:  English 
Date of oral examination:  27. March 2018 
1. Referee:  Tresp, Volker 
MD5 Checksum of the PDFfile:  58b1621eebcda76588bb8c5d2c2654b7 
Signature of the printed copy:  0001/UMC 25456 
ID Code:  22092 
Deposited On:  25. Apr 2018 13:59 
Last Modified:  25. Apr 2018 13:59 