Main Content

Word embedding layer for deep learning networks

A word embedding layer maps word indices to vectors.

Use a word embedding layer in a deep learning long short-term memory (LSTM) network. An LSTM network is a type of recurrent neural network (RNN) that can learn long-term dependencies between time steps of sequence data. A word embedding layer maps a sequence of word indices to embedding vectors and learns the word embedding during training.

This layer requires Deep Learning Toolbox™.

creates a word embedding layer and specifies the embedding dimension and vocabulary
size.`layer`

= wordEmbeddingLayer(`dimension`

,`numWords`

)

sets optional properties
using one or more name-value pairs. Enclose each property name in single quotes.`layer`

= wordEmbeddingLayer(`dimension`

,`numWords`

,`Name,Value`

)

[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." In *Proceedings of the thirteenth international conference on artificial intelligence and statistics*, pp. 249-256. 2010.

[2] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." In *Proceedings of the IEEE international conference on computer vision*, pp. 1026-1034. 2015.

[3] Saxe, Andrew M., James L. McClelland, and Surya Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks." *arXiv preprint arXiv:1312.6120* (2013).

`doc2sequence`

| `fastTextWordEmbedding`

| `tokenizedDocument`

| `trainWordEmbedding`

| `word2vec`

| `wordEncoding`

| `lstmLayer`

(Deep Learning Toolbox) | `sequenceInputLayer`

(Deep Learning Toolbox) | `trainNetwork`

(Deep Learning Toolbox)