In this blog post i will talk about attention the most important innovations in deep learning in the last few years. Attention started out in the field of computer vision as an attempt to mimic human perception. This is a quote from a paper on Visual Attention from 2014. It Read more…
there’s a whole class of NLP tasks that rely on sequential output, or outputs that are sequences of potentially varying length. For example: Translation: taking a sentence in one language as input and outputting the same sentence in another language. Conversation: taking a statement or question as input and responding Read more…
“Being able to predict the future is not always a good thing. Cassandra of Troy had the gift of foreseeing but was cursed by Apollo that no one would believe her predictions. Her warnings of the destruction of Troy were ignored and — well, let’s just say that things didn’t turn out Read more…
RNN if you look up the definition of the word Recurrent, you will find that it simply means occurring often or repeatedly. So why are these networks called Recurrent Neural Networks? It’s simply because with RNN’s, we perform the same task for each element in the input sequence. RNN’s also Read more…
The neural network architectures such as multi-layers perceptron (MLP) were trained using the current inputs only. We did not consider previous inputs when generating the current output. In other words, our systems did not have any memory elements. RNNs address this very basic and important issue by using memory (i.e. past inputs to the Read more…